Commit Graph
849 Commits
Author SHA1 Message Date
kevinveenbirkenbach fb125e163d Merge branch 'main' into feature/designer 2026-10-15 11:32:54 +02:00
kevinveenbirkenbach 51d75fb804 Merge pull request #908 from infinito-nexus/fix/env/distro-derived-keys/794
fix(env): derive the image and container names from the distro under test
2026-10-15 10:50:59 +02:00
kevinveenbirkenbach 2b43cd666f Merge remote-tracking branch 'origin/main' into feature/designer 2026-10-15 02:39:36 +02:00
kevinveenbirkenbachandClaude Opus 5.5 47565781ea style(python): import datetime as dt for ruff 0.17
ruff 0.17.0 adds `datetime` -> `dt` to the default aliases of
flake8-import-conventions, and the project selects ICN, so lint-python
failed with ICN001 in seven files.

The files now import `datetime as dt`. ruff's unsafe fix also renamed
two local names in utils/github/playwright_summary.py into the alias:
the `_to_iso_z` parameter and the parsed value in
`_parse_iso_timestamp`. Both are now called `moment`; with the bare fix,
nine tests of test_playwright_summary.py failed with UnboundLocalError
and AttributeError.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 01:24:31 +02:00
kevinveenbirkenbach b637742f96 Merge remote-tracking branch 'origin/main' into fix/env/distro-derived-keys/794 2026-10-15 01:07:53 +02:00
kevinveenbirkenbachandClaude Opus 5.5 8a29ba0bfd fix(images): leave eol roles out of image discovery
The scheduled mirror job has failed every day since 2026-09-25 on the same two
images: quay.io/minio/minio and quay.io/minio/mc answer `unauthorized` to
anonymous pulls ("2/197 failed", run 38352569503). Both belong to
web-app-minio, an eol role outside the lifecycle envelope that default.env
tests, so no CI deploy pulls them.

iter_role_images now skips every role that is_maintained() reports as
unmaintained, the check the version updaters already apply to the same list.
On the tree the discovery yields 187 images instead of 192; the five it drops
belong to the four eol roles web-app-confluence, web-app-jira, web-app-minio
and web-app-phpldapadmin. The mirror sync, resolver and wait read this list,
and their descriptions and mirror.md now say so.

The roles directory is resolved first: is_maintained reads a relative path as
a role name and would keep every role.

475ab9b386 on feature/store-i18n takes the same approach with a lifecycle
reader of its own; main already has is_maintained for it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 01:19:28 +02:00
kevinveenbirkenbachandClaude Opus 5.5 1079f48feb fix(env): derive the image and container names from the distro under test
A compose deploy job labelled centos or arch built its stack image from
pkgmgr-debian and ran it as infinito_nexus_debian (#794). distros.sh
exports INFINITO_DISTRO per iteration and sources load.sh again, but the
loader returned early because INFINITO_ENV_LOADED and the .env mtime were
unchanged. A full reload would not have helped either: the debian values
were already exported, so the restore step kept them as caller intent.

load.sh now records the distro it loaded for. Another INFINITO_DISTRO
defeats the early return and drops INFINITO_CONTAINER,
INFINITO_PARENT_IMAGE and INFINITO_IMAGE, the three keys the env handlers
derive from the distro. When the requested distro also differs from the
one .env was generated for, the loader sources a set that
`python -m cli.meta.env --output` generates for it in a private temp
directory and leaves .env untouched, so a local multi-distro run does not
switch the checkout's distro. A caller value set for the loaded distro
still wins.

Verified: tests/unit/python/scripts/meta/env/test_load.py and the new
distros.sh case pass. Against main's loader both fail with the symptom of
the issue: INFINITO_DISTRO=centos next to infinito_nexus_debian and
pkgmgr-debian:stable, and distros.sh handing debian, arch and centos the
same debian parent. shellcheck is clean.

Consequence: CI jobs labelled arch, ubuntu, fedora or centos now build
and deploy that distro, so a failure that shows up there was masked
before.

feature/store-i18n carries 3f973eb137, which resolves the parent image
for scripts/image/push.sh on its own, and 05e4dbafcb, which inserts the
INFINITO_INSTANCE slot check into the block this commit restructures. The
merge keeps the slot check ahead of the distro check.

Closes #794

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:30:48 +02:00
kevinveenbirkenbach 8c82e5262e Merge branch 'main' into feature/designer
# Conflicts:
#	roles/web-app-suitecrm/files/session.yaml
#	roles/web-app-suitecrm/meta/volumes.yml
2026-10-13 15:27:59 +02:00
kevinveenbirkenbachandClaude Opus 5.5 f7987679dd fix(ci): read every page of the job listing before reporting failed roles
report_failures asked `gh api --paginate` for the run's jobs with the jq
filter `[.jobs[] | {name, conclusion}]`. gh applies that filter to each page
and prints one array per page, so a run with more jobs than one page holds
produced two JSON documents, and json.loads stopped with "Extra data: line 2
column 1". Main run 38218402254 (113 jobs) failed its "Report main role
failures" job that way, and no issue was opened for its failed roles.

The listing now emits one job object per line and parses the lines one by
one, the same way logs/download/github.py lists jobs.

Verified: the old parse of the real output of run 38218402254 raises the
same "Extra data: line 2 column 1 (char 8491)" as CI; list_jobs on that run
returns all 113 jobs, 16 of them failed, grouped into 12 failed roles.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 00:46:28 +02:00
kevinveenbirkenbachandClaude Opus 5.5 e31b9557d2 fix(env): keep the host .env when the runner container runs make setup
The repository is bind-mounted into the infinito container, and every deploy
runs `make setup` there, which depends on `dotenv`. The generator therefore
rewrote the host's .env with the values the container sees: the host lost
INFINITO_RUNNING_ON_GITHUB=true that way, and the guide replay pulled
alpine:latest past the GHCR mirror.

compose.yml now marks the infinito service with INFINITO_DOTENV_HOST_OWNED,
and `python -m cli.meta.env` keeps an existing .env while that marker is
"true". A missing .env is still generated, and `make dotenv-force` (env -i)
still rewrites it. The key stays out of the env registry: only compose.yml
sets it, it is no configuration value, and a .env entry could freeze the
host .env.

test_distros now also strips INFINITO_ENV_LOADED. load.sh returns early
while that guard is set and the mtime of .env is unchanged, so the test only
reloaded the budget keys it strips because the container kept rewriting .env.

Verified: unit tests cover a kept .env, a missing one being generated, and a
rewrite without the marker. Local compose deploys print "Kept .env: the host
checkout owns it" where they printed "Wrote .env" before, the host .env
checksum stayed the same across a deploy, and the four keys holding host
paths resolve inside the container under the same paths.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 18:35:22 +02:00
kevinveenbirkenbachandClaude Opus 5.5 25a1f08ab7 feat(design): leave the design queue by switching the design service off
The design queue skipped a hand-kept UI_LESS_ROLES list in
cli/meta/roles/design. The decision now lives in each role: a web-app- or
web-svc- role without a UI to design sets design.enabled: false in its
meta/services.yml, and design_disabled() reads that flag. web-svc-design,
the provider, stays out by its entity name.

A new lint, test_design_off_reason.py, requires a design entry in every
web-app- and web-svc- role and, where the entry is switched off, a
"# nocheck: design-off  Reason: <text>" on the design: line or the comment
lines directly above it. The rule is in the suppression catalog, and
requirement 037 decision 13 describes it.

Twelve roles switch design off with a reason checked against their code:
asset and simpleicons were already off; file, onlyoffice, bridgy-fed and
mig were on and no longer receive the shared CSS and design script; cdn,
collabora, coturn, legal, mirror and xmpp had no entry. The four that were
on drop their bond and their design variant overrides, which the
disabled-bond and variants-static-override lints reject on a literal false.
The queue lists 77 candidates; HEAD's list also counted bridgy-fed, mig,
collabora and onlyoffice. Cosmos diagrams and the integration matrix are
regenerated.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 17:09:33 +02:00
kevinveenbirkenbachandClaude Opus 5.5 bc78780a08 fix(design-queue): count web-svc-legal as a role without its own UI
web-svc-legal has no vhost and no container; it only renders imprint.html
into the web root of web-svc-html, whose design spec styles and tests that
page. make design-queue still listed the role as new because it was missing
from UI_LESS_ROLES. The role now sits in that set, and its TODO.md, whose
only line recorded exactly this, is gone.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 11:08:53 +02:00
kevinveenbirkenbachandClaude Opus 5.5 0327114c44 fix(inventory): drop disable= toggles once their key leaves the list
Provisioning merges into host_vars and never deletes, so the false
enabled/shared toggles that apply_services_disabled writes for a key stayed
in the round inventory after a later run dropped that key from disable=.
That covered the provider's own entry too: with seaweedfs once disabled,
web-app-seaweedfs kept shared: false, the service registry offered no
seaweedfs provider, container_networks attached no consumer, and BookWyrm
ran USE_S3=true against http://seaweedfs:8333 without a route.

reconcile_services_disabled_from_env records the keys of each disable run
in the host_vars key _INFINITO_SERVICES_DISABLED. On the next run it deletes,
in every application block, the enabled/shared toggles that are still
exactly false for a recorded key the current disable no longer lists, drops
a block left empty, and rewrites or removes the record. It runs right after
ensure_host_vars_file, before credentials and the variant bake, so a variant
that wants a toggle off writes it again. With an unchanged or empty disable
it does nothing. An inventory written before this change has no record, so
its older toggles are only reverted after one run with the old list.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 11:07:21 +02:00
kevinveenbirkenbach 1fb98b5b76 Merge remote-tracking branch 'origin/main' into feature/designer 2026-10-10 10:59:33 +02:00
kevinveenbirkenbachandClaude Opus 5.5 1ba796d7f1 fix(ci): pull the images the runner uses itself from the GHCR mirror
The GHCR mirror covered role images only. The images the runner pulls on
its own went straight to Docker Hub: coredns from compose.yml, the cache
stack in compose/cache.override.yml and alpine:latest, the init image of
scripts/docker/cache/package-frontend/certs.sh. With the anonymous
githubactions quota spent, the guide step of Postmarks#0 (main run
37954203718) and Shopware#2 (#872) died on "toomanyrequests" while pulling
alpine:latest, after coredns had hit the same limit.

iter_runner_images (utils/docker/image/discovery.py) finds those images,
and the mirror sync and the fork-PR wait now cover them next to the role
images. On a real GitHub runner compose_file_args layers a generated
build/compose/runner-mirror.override.yml that points only the services of
the layered compose files at their mirror, and the certificate generator
gets the mirrored init image. Local runs and act are unchanged, and the
coredns literal stays in compose.yml for Dependabot. test-dns now waits for
images-mirror-missing like the deploy and workspace jobs, since it brings up
the stack too.

Verified with docker compose config on a simulated GitHub runner: the
service set is unchanged and only the runner images point at
ghcr.io/infinito-nexus/core/mirror/docker.io/..., and an existing role mirror
under that scheme pulls anonymously.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:37:41 +02:00
kevinveenbirkenbach 3c5305de5c Merge remote-tracking branch 'origin/main' into feature/designer 2026-10-09 19:06:42 +02:00
kevinveenbirkenbach 377f5a070a Merge remote-tracking branch 'origin/main' into feature/designer 2026-10-08 18:42:16 +02:00
kevinveenbirkenbachandClaude Opus 5.5 13901899d5 docs(update): state that a templated pin never moves backwards
The README said a templated pin moves whenever the assembled value
changes. newer_version() additionally rejects an assembled value that
sorts below the current pin, which
test_an_unchanged_or_older_templated_tag_reports_nothing covers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 16:28:30 +02:00
kevinveenbirkenbachandClaude Opus 5.5 4cba7e7c2f feat(update): read a version source from several documents, once per run
An http_regex source read one document, so a tag whose parts upstream
keeps in two files could only carry the second part as a literal. Every
pin also fetched its document on its own: a failed request left one pin
of a release behind while the others moved.

url now takes a list whose bodies are searched as one text, and every
document is fetched once per run, so all pins that read it resolve against
the same snapshot. The HTTP reads move into utils/update/fetch.py.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 11:26:13 +02:00
kevinveenbirkenbachandClaude Opus 5.5 1585e056fc feat(update): let a declared version source own the whole pin
A pin that names its own upstream in an update: block was still collected
by the image updater or, for a ref, by the repository updater. Those raise
it to the newest tag of its registry or repository, which for a component
of a release is not the release's value. A tag built from several upstream
values, such as <freeswitch>-<bigbluebutton>, could not be resolved at
all: the resolver takes one capture and keeps a pin on its suffix.

The image updater now skips a version and the repository updater a ref
that declares a source for itself. An http_regex source accepts a template
over the pattern's named groups, and a templated pin moves whenever the
assembled value changes, its suffix included.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 10:32:14 +02:00
kevinveenbirkenbachandClaude Opus 5.5 75460b95cd feat(update): leave an image pin to the version source it declares
A pin that names its own upstream in an update: block was still collected
by the image updater, which raises it to the newest tag of its registry.
For a component of a release bundle that tag is not the bundle's, so the
two updaters would propose different versions for the same line and such
a pin had to carry nocheck: docker-version.

The image updater now skips a version that declares a source for itself.
The source resolver alone bumps it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 10:30:48 +02:00
kevinveenbirkenbach fb8dd810c6 Merge remote-tracking branch 'origin/main' into feature/designer 2026-10-07 16:28:26 +02:00
kevinveenbirkenbachandClaude Opus 5.5 a43f15b539 Merge branch 'main' into feature/designer
Conflicts:
- roles/web-app-pihole/meta/services.yml: main dropped the unused redis
  entry, this branch changed the javascript entry directly below it. The
  redis entry is gone and the javascript entry keeps the flag bound to
  the SSO flag of the role and its nocheck reasons.
- README.md: the generated role table keeps the renamed web-svc-design
  role and lists Pi-hole without Redis and without the CDN.

The role diagram of Pi-hole is regenerated, because both sides removed
an edge and the index of the red link moved.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 17:09:52 +02:00
kevinveenbirkenbach 0622cce916 Merge remote-tracking branch 'origin/main' into fix/code-scanning-alerts/807 2026-10-06 09:18:00 +02:00
kevinveenbirkenbachandClaude Opus 5.5 3610e7b9ef fix(code-scanning): resolve the alerts that describe real code
Of the 76 open code scanning alerts, 39 point at code that can change.

svc-ai-mcp-adapter:
- A path argument that is empty or made only of dots is refused.
  Percent-encoding keeps a separator inside its segment, but "." and ".."
  need none, so an upstream that resolves dot segments could answer from a
  path the contract does not name.
- parse_duration splits on the units. The pattern it used before rescanned
  a long run of digits from every start position.

Tests and the README generators:
- Every Jinja environment built with autoescape=False and a "noqa: S701"
  uses select_autoescape now, as six test modules and
  utils/templating/jinja.py already did. It escapes templates named
  *.html, *.htm and *.xml, which none of these call sites loads.
- Dead code goes: a doubled "rel" block in the email lint, an unread
  _SKIP_DIRS, and a module that _resolve returned to no caller.
- Two "except ...: pass" blocks become contextlib.suppress and
  assertRaises.
- Nine test modules import unittest.mock one way instead of two, and one
  lookup test imports its module once.

Closes #807

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 09:18:00 +02:00
kevinveenbirkenbachandClaude Opus 5.5 89b95d751c fix(update): stop bumping pins of eol roles
The update bot opened a version PR for web-app-jira (#821) although the role is end of life: none of the updaters looked at the lifecycle. utils.update.base.is_maintained now answers that question once, and the collectors for image versions, declared version sources, pip pins, repository refs and the addon check skip a role whose lifecycle is eol.

The lint that requires every semver pin to be watched shares that scope, otherwise it would report the pins of the eol roles as unwatched. Every other stage, deprecated included, stays in the update scope.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 01:09:10 +02:00
kevinveenbirkenbach 396b8230eb Merge branch 'main' into feature/designer 2026-10-06 00:13:54 +02:00
kevinveenbirkenbachandClaude Opus 5.5 6f7eb6b8c3 docs(design): describe in-house carriers, role sync and the palette check
The design loop keeps apps deployed between roles and purges only when
the next deploy does not fit, by memory, storage or a published host
port.

A role pass applies the palette through the app's own configuration
first: a theming configuration, then an in-house theme or custom CSS
option, and the injected role stylesheet only for what neither reaches.
It sizes one logo slot per shape, iterates through design-sync,
compose-role-sync and compose-playwright instead of a second full
deploy, asserts with assertToken, assertLightAndDark and
assertReadable, provisions an account for the signed-in interface, and
checks the palette with a blue, a red and a green base color.

The requirement follows: Bootstrap mapping on opt-in, in-house
configuration first, compose-role-sync for a changed script, task or
template. web-svc-mirror serves no page of its own and leaves the queue
and the role list.

The README of web-svc-design links the overview of the review
galleries.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 22:12:12 +02:00
kevinveenbirkenbachandClaude Opus 5.5 4c955d080f feat(update): fail a declared update source whose pin is not a semver
The version-source resolver orders upstream versions by semver. A pin that
is not one, such as `latest` or a branch name, has no depth, so no candidate
ever matches it and the declaration never moves anything. Nothing reported
that: the watched-pins lint only looks at values that already are a semver.

test_version_keys now fails on an `update:` block whose pinned key does not
hold a semver. A monitored addon the resolver does not read is held to the
same rule as soon as it pins a `version`, because the catalog check skips a
non-semver pin just as silently. An addon without a `version` keeps tracking
the app default and is left alone.

The Stalwart README no longer asks for `webui_version` to be bumped by hand
next to the image tag. Its `update:` block names the upstream tags, the
server itself follows `releases/latest` of the bundle, and the resolver
plans v1.0.5 to v1.0.11 for it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 12:48:22 +02:00
kevinveenbirkenbachandClaude Opus 5.5 739c78bad1 feat(update): bump addon pins through their declared source
The version-source updater only read meta/services.yml. An addon pinned in
meta/addons/<id>.yml stayed where it was: the ONLYOFFICE connector for
Nextcloud sat on v10.1.2, which does not install on Nextcloud 35, until
someone bumped it by hand.

The updater now reads addon files too. An addon declares its source in the
`update:` block it already has, beside monitored, catalog and upstream_id.
A monitored addon of the github-releases catalog needs nothing more: its
upstream_id names the repository whose tags are its versions. The
update-version-sources job opens the PRs, one per role.

Every pinned addon is wired:

- nextcloud onlyoffice carries its own version pin now. Its archive URL
  stays a literal that carries the pin, because
  test_pinned_addon_archives_available reads it as written
- wordpress mcp-adapter, moodle auth_oidc and jellyfin sso-authentication
  follow the tags of their repository, moodle within its v4.5 series
- the three xwiki extensions read their Maven metadata
- pretix-oidc reads the PyPI simple index. The JSON API answers 404 behind
  the package cache the test stack routes pypi.org through
- joomla's plugin is built here and jellyfin's ldap-authentication is bound
  to the md5 of its archive; both are opted out with that reason

A bump rewrites the pin and the config.archive line together, and an
archive that does not carry the pin fails test_version_keys, so the two
cannot drift apart again. test_version_pins_are_watched covers addon pins
as well: a new one without a source or a reasoned opt-out fails the lint.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 02:50:48 +02:00
kevinveenbirkenbachandClaude Opus 5 81ab6e4ffc feat(update): let a version pin declare its own upstream source
The Docker updater watches image plus version, the repository updater
repository plus ref. A dozen pins fell outside both - an app_version, a
release, a tag in a registry neither speaks - so nothing ever told anyone
they had gone stale.

Such a pin now names its upstream next to itself in an `update:` block:
git_tags, registry_tags, npm, http_regex or a role-local script, with
`match` selecting a tag flavour and `strip` removing a prefix the pin does
not carry. utils/update/source.py resolves them beside the two existing
updaters, `python -m cli.contributing.update.source` bumps them, the
update-version-sources job opens one PR per role, and
tests/external/update/source warns while a bump is pending.

Two lints hold the rule: every semver pin is watched by one of the three
updaters or opted out with a reason, and a semver only ever sits behind
`version`, `<what>_version` or `ref` - which renames moodle's release to
app_version and simpleicons' local_version to local_tag, the latter being
the tag of a locally built image rather than an upstream version. A
declaration naming a key the entity does not carry, or a type the resolver
does not know, fails the lint instead of resolving nothing in silence.

Picked from feature/store-i18n (94d607fd0a) onto current main. Two things
differ from the original: svc-ai-lmstudio no longer carries an app_version
pin, so its declaration is dropped, and web-app-stalwart's webui_version,
added since, declares the git tags of the webui repository.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 02:50:48 +02:00
kevinveenbirkenbachandClaude Opus 5.5 5b002678df docs(mcp): remove the generated audit report
The report aggregated each role's meta/mcp.yml into one table, and its
freshness test failed on every image-version bump of an MCP-serving role,
because the updater edits services.yml and nothing regenerates the committed
report. The table added nothing the sources lack: scope and credential subject
were identical for every role, so it restated its inputs.

The generator, the report, its freshness test, make mcp-audit and the i8mcau
alias go, and make docs no longer calls the generator. The metadata stays in
each role's meta/mcp.yml, where test_mcp_schema.py enforces its shape, and
requirement 035 now says so. The classification and completeness lints stay,
because they check the lists in 035 against the roles, not the report.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 14:36:52 +02:00
kevinveenbirkenbachandClaude Opus 5.5 86cefeff26 feat(web-svc-design): derive one corporate design from a single base color
web-svc-css becomes web-svc-design and owns the configuration that lived in group_vars/all/13_design.yml. Consumers switch their css service key to design, and sys-front-inj-css becomes sys-front-inj-design. javascript stays an independent key; a role's optional design.js.j2 is injected together with the design and CSP-hashed like the functional script.

An in-repo OKLCH lookup replaces colorscheme_generator. From design.colors.base alone it derives surfaces, text, borders, the primary and status fills with their on-colors, hover and active states and a dedicated dark palette; a unit test proves WCAG AA, and AAA for body text, across six base colors in both modes. Semantic --design-* tokens replace --color-NN-SS in every role stylesheet. The shared element styles sit in the cascade layer infinito-design, so an app's own design system always wins and takes the palette through a variable mapping in its role stylesheet or through its own theming mechanism.

design.logo and design.title resolve globally from web-svc-design or per role from its design entry, and false or 0 disables the replacement. The design_assets action renders the logo lockup per slot as SVG, PNG and ICO, and the legacy title keys move into the design entries.

Three roles carry the design end to end. The dashboard takes logo, favicon and title from the generator and replaces the hard-coded port-ui colors. Gitea maps its theme variables onto the tokens and mounts the generated logo and a branded start page. Nextcloud receives primary color, logo, name and slogan through occ theming and ships no stylesheet.

make design-sync re-renders the static design files of a deployed app, make design-gallery captures before and after screenshots in light, dark, desktop and mobile, and make design-queue orders the roles that are due for a design pass. The i8-designer skill and docs/agents/action/design.md describe that loop.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 06:04:14 +02:00
kevinveenbirkenbachandClaude Opus 5.5 42bdf4321f docs(roles): stop testing deprecated roles
lifecycle.md said a deprecated role is still covered by the test suite while the tested envelope already left it out. A deprecated role is now untested: the matrix-deploy MUST NOT exercise it and a failure in it MUST NOT block a release, while planned, pre-alpha and eol stay open to best-effort runs. The complexity legend says untested too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-03 21:11:51 +02:00
Alejandro Roman 9df3287adf Merge remote-tracking branch 'upstream/main' into feature/role-stalwart-mail-swarm 2026-09-22 12:16:44 +04:00
kevinveenbirkenbachandClaude Opus 5 5b8465260f feat(ci): deploy only the variants a change reaches
An image bump in a provider redeployed every variant of every consumer,
including the variants that switch that provider off and therefore never
deploy the bumped image at all. The diff-derived whitelist now narrows each
affected role to the variants whose own round closure still reaches a seed,
emitted as `role#0,2` selection tokens. That closure is walked on the same
variant-merged services maps the matrix planner builds its rounds from, so the
narrowing follows a parent's variant choice across every generation of the
chain. It covers every auto-derived whitelist: the weekly update PRs,
Dependabot, and any other diff-driven run.

The narrowing never shrinks the matrix on partial information. A seed role
keeps every variant, because its own files changed. A role reached only
through a run_after edge keeps every variant too, because the round closure
does not follow that edge and a role whose hits are empty is therefore
unnarrowed rather than dropped.

A pinned token naming a role the discovery query does not return now warns
instead of aborting the run. The diff-derived whitelist legitimately names
roles the lifecycle envelope filters out, and once it pins variants such a
token would abort every chunk; role names are still checked against roles/
before the run starts, so a typo is caught one layer earlier.

The round's variant merge moves out of the deploy inventory package into
utils/roles/applications/variants.py. The CI selection has to build the same
maps, and importing the deploy package for them pulls in a module that reads
INFINITO_SRC_DIR at import time, which the runner host does not set.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-21 22:51:03 +02:00
Alejandro Roman 4b9a834546 Merge remote-tracking branch 'upstream/main' into feature/role-stalwart-mail-swarm 2026-09-21 03:25:20 +04:00
kevinveenbirkenbachandClaude Opus 5 673de05a60 fix(inventory): write only the service toggles a role's contract declares
`disable=` wrote both `enabled: false` and `shared: false` for every role that
declares the service, but a role qualifies with any one of `enabled`, `shared`
or `provides`. compare_application_keys rejects any host_vars key the role
contract does not carry, so a role declaring only `enabled` came back as a
validation error and the deploy never reached Ansible:

    Missing default for web-svc-cdn: services.dashboard.shared
    Missing default for web-svc-mirror: services.dashboard.shared
    [ERROR] Inventory validation failed. Aborting deployment.

_declared_toggles reads the role's meta/services.yml and writes only what is
there; a role carrying neither toggle is skipped instead of gaining an empty
block. The module already held this principle: find_roles_with_service excludes
pure metadata entries so `disable` "doesn't add bogus enabled: false overrides",
and this applies the same rule per flag rather than per role.

Delta measured over the disable list the compose iteration procedure prescribes
(matomo, dashboard, prometheus, email, css): 407 of 409 role/service pairs are
untouched because they declare both toggles, exactly 2 stop receiving a
fabricated key, and 0 fall out of the disable entirely. The 2 are the pairs the
validator rejected. Across all 266 roles with a meta/services.yml, 1224 service
blocks declare `enabled`, 1190 declare `shared`, 34 declare `enabled` alone and
none declares `provides` alone, so nothing loses its disable.

Proven by the deploy sequence rather than by a test: the first local deploy
aborted at inventory validation, the second passed into the Ansible run with the
same disable list. Two regression tests cover both shapes, and the negative
control holds: hard-coding ["enabled", "shared"] fails the new test.

Without this, `disable=` is unusable in every local deploy, which the iteration
procedure requires on every run.

Residual: the generator was exercised through compose-deploy only, not through
roundtrip or swarm-zombie, which reach the same function by other call paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 06:00:56 +02:00
kevinveenbirkenbachandClaude Opus 5 2567c5ece1 feat(ci): let a manual run set the chunk size the formula would derive
chunk_size() multiplied the runner slots a chunk may assume by the waves the
queue window affords, and nothing could say otherwise. The derivation is the
right default -- too few rows leaves runners idle, too many push a chunk's tail
past the queue cancel -- but a run that wants smaller chunks had to edit
default.env to get them.

INFINITO_CI_CHUNK_SIZE now takes precedence when it is greater than zero, and
zero keeps the derivation. The input travels the same path offset already
travels: entry-manual-steer.yml offers the field and maps empty to 0,
call-orchestrator.yml hands it to all four chunk blocks, call-test-deploy.yml
exports it as the env var. It is a string rather than a number for the reason
resume_from_chunk already documents: a workflow_dispatch number input reaches
the caller as a string, and a number-typed workflow_call input then makes
GitHub refuse the job before it exists.

Two places in call-test-deploy.yml set it, not one. The plan summary resolves
the chunk size itself, so exporting it only for the app resolution would print
a plan that disagrees with the matrix the run then deploys.

Measured against the live helper: 0 gives 80 rows over 3 chunks, 25 gives 25
over 4, and returning to 0 restores 80 and 3.

The gate found three faults in the first attempt and all three are in this
commit: the key sorted after INFINITO_CI_CONCURRENCY rather than before it, the
dispatch description ran to 81 characters against a limit of 80, and the
retrigger fixture did not carry the new input. That last one matters beyond the
test: it is the check that would otherwise let a retrigger drop the chunk size
silently. carried_inputs() itself needed nothing, since it derives from the
declared inputs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 11:25:12 +02:00
kevinveenbirkenbachandClaude Opus 5 66be8f6a0a test(lint): refuse a Debian codename in an image tag
A codename carries no ordering, so 'bookworm' and 'trixie' are two opaque
strings and no updater can tell that one supersedes the other. A pin that
names its Debian release that way is frozen on that release for as long as
it stands, and the freeze is invisible because the tag keeps resolving.
Numeric tags are comparable and stay inside the freshness pipeline, so the
rule is worth holding for the whole repository rather than fixing case by
case.

The check reads two shapes, because a codename reaches the tree in two
spellings. A 'name:tag' reference is recognised by the absence of a space
after the colon, which is what separates 'debian:bookworm' from the YAML
'image: python' and covers Dockerfile FROM lines and heredocs without a
second parser. A 'version:' value is read as a key, because '3.11-bookworm'
carries no colon at all.

False positives are kept out by two independent gates: a word-boundary
prefilter, so 'residual' and 'inside' are not read as 'sid', and a split of
the tag on '-', '.' and '_' that requires a whole component to be a
codename. That is what lets 'cache buster', '.stretched-link',
'fa-bullseye' and a tags list containing 'stretch' through. Every codename
occurrence in the repository was checked against the matcher by hand, and
the check was negative-controlled in both shapes, including an untracked
file.

Four references had to move for the rule to hold. meta/distros.yml takes
'debian:13' rather than '13.6': at depth 1 the updater can raise it across
a major, while depth 2 pins it inside the 13.x line forever. The whole
'make install' path was run in a debian:13 container to back that: all
twenty-four Depends and Recommends resolve on trixie, the docker repository
bootstrap derives the right suite and installs docker-ce-cli, and the
package builds and installs.

The workspace DiD probe moves to 'debian:13-slim'. An earlier version of
this change carried a suppression claiming the probe had to follow the
suite the apt proxy mirrors. That was wrong twice over: it cited a function
that has nothing to do with the probe, and the coupling does not exist,
because the cache frontend rewrites by path prefix and the suite never
enters a routing decision. Replaying the probe verbatim against
debian:13-slim fetches all three trixie suites through the
bookworm-configured proxies, so the suppression is gone rather than
reworded.

The rule ships with an escape hatch and no uses of it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 17:03:53 +02:00
kevinveenbirkenbachandClaude Opus 5 f0ecf3e337 style(update): apply the formatting the gate produced
The nocheck markers were written as long single lines; ruff splits the
call across three lines and leaves the marker behind the closing paren.
Committing the formatter's own output keeps the tree and the history in
step.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 21:51:25 +02:00
kevinveenbirkenbachandClaude Opus 5 94baeb7502 fix(lint): meet the repository rules the collection work broke
The gate flagged six of my own changes. INFINITO_ANSIBLE_COLLECTIONS_SOURCE
reached default.env without a handler, so the generator dropped it
silently; ansible.sh then carried a shell-side default for the same key,
a second source that drifts. Both are now one declaration, read as
required, so a missing key fails loudly instead of falling back.

File reads route through the session cache except where the same run
rewrites the file, which is the documented opt-out and now says so per
call. The updater reads yaml through utils.cache.yaml rather than calling
safe_load itself.

restore_collection_history.sh was committed without its executable bit,
and the new workflow's name was 31 characters where the dispatch menu
truncates at 25; the explanatory wording lives in run-name instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 21:47:10 +02:00
kevinveenbirkenbachandClaude Opus 5 623ecc0248 feat(update): retune the preferred collection source from measured latency
Which of the two sources is faster is not a constant. galaxy.ansible.com
degraded from 207s to 687s over one day and then served 502s outright,
while the pinned git fallback carried the same install in 31s; a week
later the order may be the other way round. The daily job times both and
swaps the default only when the preferred one's median reaches three
times the other's over twelve samples.

A failed install is recorded as None and ranks worse than any duration,
so an outage counts instead of dropping out of the statistic. Below a
full window nothing changes, which keeps a single bad day from flipping
the default.

History lives in an artifact rather than the Actions cache: a schedule
run resolves to the default branch and has read-only cache access there,
so it could not carry the window forward.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:59:45 +02:00
kevinveenbirkenbachandClaude Opus 5 e992175fcc feat(update): keep the pinned Ansible collections from ageing unnoticed
Pinning the collections froze them, and nothing thaws a pin: dependabot
reads pyproject.toml, the docker updater reads image tags, and neither
sees a version inside requirements/. The new job resolves upstream
releases with git ls-remote --tags and opens a PR the same way the pip,
image and repository-ref updaters do.

Both requirement files are bumped together, because the git fallback has
to install what the Galaxy file declares; moving only one would break
that. A collection whose two files disagree, or whose version is not a
semver, is left alone -- that covers the branch ref and the commit pin,
which state an intent a bumper must not guess.

The job calls pr.sh rather than role_prs.sh: the latter splits per
sub-directory and would open no PR at all for flat paths like
requirements/*.yml.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 20:58:35 +02:00
kevinveenbirkenbachandClaude Opus 5 05fcec52e7 fix(planner): give every variant-carrying app the round's index, not only the include
round_variants was built by walking round_include. When a primary's variant switches its couplings off the planner resolves no prerequisites, round_include collapses to the primary, and no dependency gets an index at all. derive_includes then evaluates them at base config, where an unrendered Jinja coupling counts as enabled, so the closure inflates to everything reachable instead of what the variant declares.

Measured: svc-ai-litellm round 1 deploys 10 apps instead of 32, web-app-bigbluebutton round 1 and web-app-peertube round 1 drop from 34 to 10 and 11. Every other round of every role measured is unchanged, including all fourteen nextcloud rounds. The apps that leave are not dependencies of the round: svc-ai-litellm declares litellm, lmstudio, ollama and postgres and no sso at all, while bigbluebutton and peertube declare sso but switch it off in variant 1 together with dashboard, matomo, css, logout and prometheus.

complexity/cli.py:300 already documents the rule this makes true: the planner hands each pulled-in dependency round_index if round_index is below its variant count else 0, and pulls_at_zero implements exactly that. The coverage model was describing behaviour the planner only delivered for apps inside the include.

init.py filters the printed variant map to the apps actually shown, because round_variants now carries 266 entries and round 1 would otherwise print 112 of them on one line while ten apps deploy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 13:29:43 +02:00
kevinveenbirkenbachandClaude Opus 5 e74a585402 fix(guide): ask the reader for the clearnet resolver a Tor node needs
svc-net-tor aborts at roles/svc-net-tor/tasks/router.yml:2 when networks.internet.dns is empty, and rightly so: dnsmasq runs with no-resolv, so an empty upstream leaves the node answering .onion and nothing else. That value is set in exactly two places, both under inventories/development/, and the documented production path reaches neither - the provisioner writes no networks key at all. A reader following the role README verbatim therefore cannot deploy a Tor node.

CI has been proving this since 2026-08-31 without anyone seeing it: the README replay rides on the role's smallest variant and is skipped when the deploy is red, so the only row that ever reached it is one where svc-net-tor itself went green. That happened first on Ubuntu, where the local-service defect abb6f37297 fixes never fired.

The template now emits the DNS assignment, the -e passthrough and the networks entry in --vars for a role whose tasks name networks.internet.dns, which today is svc-net-tor alone; the generator derives that rather than taking a declaration, so a second role asking for the value documents it by doing so. scripts/github/guide/compose_deploy.sh substitutes the placeholder from INFINITO_DNS_IP, the same SPOT inventories/development/default.yml reads, next to the existing domain substitution.

This follows 972596a758, which added the administrator key to the same block for the same reason. No fallback inside the role: defaulting to NETWORK_PUBLIC_DNS_RESOLVERS would route a Tor node's clearnet queries to Cloudflare without the operator choosing it, which is what the assert exists to prevent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 02:17:26 +02:00
kevinveenbirkenbachandClaude Opus 5 46110f4311 perf(swarm-test): rotate the round's credentials, not every block in the file
The matrix gate rotates credentials between the two deploy passes so PASS 2
has to carry fresh values. It rotated every application block in the host_vars
file, and on a matrix inventory that is not the round. Provision applies its
--include as a strict intersection (filters.py:57-62), so it generates
credentials for at most derive_includes, while mirror_overrides.py adds a block
per mirror artefact carrying only image and version. Rotating those costs one
credential subprocess each for values provision never wrote.

matrix.py now passes --include derive_includes(app_id, variants=round_variants),
the same call 02_provision_inventory.sh:11 feeds provision, from the same
round_variants the provision step got one call earlier. The rotation set is
therefore a superset of what provision generated: every database_password still
rotates, because utils/manager/inventory.py:236-242 writes it under an id from
that same filtered list. Measured for web-app-wordpress: 35 ids.

The scope stops at the credentials. reset_credentials takes credential_ids
beside application_ids because the two halves shared one list, and narrowing it
narrowed _drop_user_passwords too: that resolves through
required_user_policies(roles_dir, application_ids), so 60 of 66 declared users
would have stopped rotating, root, backup and every mcp-* bot among them. The
user half keeps the full document list and is byte-identical to before.

The test fixture declared the same users in both roles, which made the
regression test pass against the regression. web-app-keycloak now declares a
user web-app-a does not, so the assertion has something to bite on.

The CLI aborts when --include matches no block instead of rotating nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 20:32:02 +02:00
kevinveenbirkenbachandClaude Opus 5 c32bf320ff feat(ci): add a fourth deploy chunk so a short priority chunk strands no budget
A sweep may deploy 210 rows (available) in chunks of 80. Three declared
blocks hold exactly that when there is no priority line, but priority
rows get a chunk of their own that is never topped up with regular rows.
Nine priority rows therefore occupied a whole block, left the regular
rows 2 x 80 = 160, and 42 rows of budget went unused.

Adding a block to the YAML alone would not have changed that:
matrix.chunks_of handed the split slots.chunk_count(), which is
min(ceil(available / chunk size), INFINITO_CI_MAX_CHUNKS) and ignores
the priority seam, so the plan still stopped at three blocks.

call-orchestrator.yml now declares test-deploy-chunk-3, gated on chunks
0 to 2 like every chunk gates on all before it, and report-main-failures
and done need it. INFINITO_CI_MAX_CHUNKS is 4, one above what the budget
fills, and slots.chunk_blocks() returns it. chunks_of hands the split
every declared block; available still caps the rows, and a chunk whose
slice comes back empty skips its deploy. chunk_count() stays the "chunks
filled" figure of the budget report.

Tests: the split spends 9 + 80 + 80 + 41 = 210 rows with a spare block;
chunks_of plans three chunks from chunk_blocks=3 while chunk_count=2
(the previous wiring yields [1, 10]); an integration guard requires the
declared chunk jobs to equal INFINITO_CI_MAX_CHUNKS and to exceed
ceil(available / chunk size) by one, and fails on the stale value 3.

A shell that inherited INFINITO_CI_MAX_CHUNKS=3 keeps failing that guard
until it is restarted; make dotenv-force rewrites .env.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 23:46:48 +02:00
kevinveenbirkenbachandClaude Opus 5 c7d162fa88 fix(coredns): derive the domain regex from INFINITO_DOMAIN when rendering
INFINITO_DOMAIN_RE carried an already escaped regex through the env
chain: EnvBuilder.setdefault prefers a value already in the process env,
quote_for_dotenv escapes every backslash, and load.sh unescapes it again.
One inherited value with a doubled backslash was enough to write
"infinito\\\\.test" into .env, the rendered Corefile then matched a
literal backslash, and getent hosts test.infinito.test failed with rc 2.

The renderer now substitutes DOMAIN_RE = re.escape(INFINITO_DOMAIN) at
render time, so no backslash travels through the env or .env at all. The
template placeholder is renamed because it is a derived render value, not
an env key, and INFINITO_DOMAIN_RE is removed from default.env and the
passthrough keys, leaving INFINITO_DOMAIN as the single source.

A renderer test renders the real template and requires a single-escaped
match with no doubled backslash and no leftover placeholder. Verified
live: the Corefile renders ^infinito\.test\.$, test.infinito.test
resolves to 172.30.0.10 and test.infinitoXtest does not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 20:53:19 +02:00
kevinveenbirkenbachandClaude Opus 5 8434a31611 fix(ci): answer a role-only selection by existence, not by variant None
Rows are indexed by `(name, variant)` and a variant is an int. A token that
names only a role carries no variant, so the lookup ran with `None` and could
never hit, and every such token was reported as matching no discovered row.

That is the form `collapse_to_roles` produces, so it is the form a retriggered
priority line arrives in. Run 34348821577 shows the result: all twenty-one
priority entries carried

    priority: '<role>' matches no discovered row
              (no row in this run's mode and lifecycle envelope)

while baserow, checkmk, decidim and the rest went on to deploy fine. The
warning was not wrong about one entry and right about twenty; it was blind. A
role that genuinely has no row in the envelope looked exactly like a role with a
full set, which is how priority work goes missing without anyone being able to
see which entry did.

A token that narrows nothing beyond the role name is now answered by existence:
it asks for the role in whatever axes the rotation picks, so any discovered row
satisfies it. `Pin.pinned` already draws that line - "whether the token narrows
anything at all beyond the role name" - and pinned tokens keep the old path,
`check_pins` included.

Three tests come with it. There was already one for a bare role that matches
nothing, and none for a bare role that matches something, which is why the
lookup could stay broken. One of the new ones uses variant 4 so a future
implementation cannot pass by quietly guessing variant 0. Checked against HEAD:
the same input returns the spurious warning there, so the tests fail without
this change.

This restores the signal, not the lost jobs. The other silent path, `priority`
being truncated to the job budget in cli/meta/ci/chunks.py, is untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 01:44:59 +02:00