Commit Graph
94 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 68b0dfd508 feat(ci): drop the dot between the category glyphs and the role title
Job and run titles read `🐳🧅…𓏴𓏛·Nextcloud#2`; the `·` between the
hieroglyph block and the title only costs width. RoleDisplayName.encode
now joins the two directly (`𓏴𓏛Nextcloud#2`). Spaces inside a README
heading still become `·`, which keeps a display name one token in the
space-separated whitelist and priority inputs.

decode and decode_list still map the dotted form of older runs back to
the role id, so report_failures, the run tooling and selection tokens
pasted from an earlier run keep working. Old and new names cannot
collide: a glyph block holds only hieroglyphs and a title starts with a
letter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 17:16:18 +01:00
kevinveenbirkenbachandClaude Opus 5.5 621a492ec3 feat(ci): name the deploy chunks as numbered batches and star the priority ones
The chunk jobs read `🧱 Chunk N` and every priority row carried its own
trailing ⭐, so a priority sweep repeated the star down the whole job
list while the chunk that held those rows looked like any other.

- The four chunk jobs are named `0️⃣ Batch` to `3️⃣ Batch`, with ` ⭐`
  when the batch holds priority rows, and their discover job reads
  `🔍 Batch N ⭐` through the existing `marker` input.
- call-validation counts those batches once per run
  (scripts/github/resolve/priority_batches.sh, which runs
  `cli.meta.ci.matrix --priority-chunks`) and exposes the count as the
  `starred` output. chunks.plan cuts the priority chunks from the
  priority rows alone, so offset and the regular line cannot move them;
  the orchestrator hands validation the same sweep and chunk size it
  hands the chunk jobs.
- The row label drops its own star. report_failures and the job-name
  parsers still accept a starred title from an older run.
- The glossary names the unit `batch` (`#️⃣`) instead of `chunk` (`📦`),
  `ci plan` prints the batch column as the keycap the job name leads
  with, and the name lint accepts a leading keycap emoji.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 15:46:38 +01:00
kevinveenbirkenbachandClaude Opus 5.5 91022ad37a fix(ci): let only the primary app's declared variant bring the guide host
Every role's base services carry `docs.enabled: "{{ 'web-app-docs' in
group_names }}"`, and the static closure counts any "in group_names" flag
as possibly true (is_explicit_truth). A round therefore pulled web-app-docs
in through any member whose active variant did not pin docs off, and a
round hands every role its index or clamps it to variant 0, where 84 roles
declare docs on. The README replay and everything it drags in (dashboard,
matomo, keycloak, prometheus with its MCP partners) landed in 365 of 469
rounds instead of one per role, which ran CI run 40625679270 out of
resources: Listmonk#0 memory, OpenProject#0 I/O in the DR drill,
Nextcloud#0/#2 budget (4310-4539 s in the replay each, 60 s over),
BookWyrm#0 drill disk, Matrix#1/#3 pids in the replay; none of those
primaries declares docs in the failing variant.

The guide host now stays on only where the primary app's own variant
declares it (declares_guide, the predicate instructions.py already used);
every other role has it switched off (guide_off), in the planner's round
overrides and in the swarm include set. The host_vars bake drops a docs
edge whenever the round does not deploy web-app-docs. Measured for
listmonk, openproject, translate and mariadb: web-app-docs is deployed in
exactly each primary's declared guide round, swarm and planner agree in
all 12 rounds, and rounds shed up to ten apps (mariadb#1 19 -> 9).

test_guide_rides_the_primary_only pins that on real role data (it fails
with the rule disabled); test_variant_pin_required leaves the guide host
out of its clone comparison and no longer counts the docs key as a
pairing edge, because the replay no longer follows a partner's index.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 12:49:51 +01:00
kevinveenbirkenbachandClaude Opus 5.5 b87bf5f979 fix(ci): disable partners a tor-only node serves no network of
A variant that pairs a clearnet-only partner with the onion (Matrix
variant 2 with web-svc-coturn) could draw the tor-only network mode, and
the deploy's domain map then aborted with "web-svc-coturn: allows
['clearnet'] but the node runs in 'tor' mode". disabled_services() now
also emits every deployed partner with domains and a Tor bond whose
networks the node mode does not serve, mirroring utils/cache/domains.py,
so the inventory provision drops it the same way it drops Tor on
clearnet rows. Measured across every role: 8 tor-only rows change, all of
which failed at that point before; clearnet and multi rows are untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 03:16:53 +01:00
kevinveenbirkenbachandClaude Opus 5 677a63931c Merge branch 'feature/network/multi-network-serving' into feature/store-i18n
Resolves 68 conflicted files. The governing rule was our structure with their
network semantics: the tor boolean becomes a network mode (clearnet | tor |
multi) throughout the variant layer, while this branch's architecture and mesh
axes, its extracted modules and its i18n content stay.

Where the two sides covered different ground, both are kept rather than one
picked:

- ca_injected answers a ca_client on the deployment's TLS mode OR on an SSO
  issuer that resolves self-signed. 3a3e9ed2c9 records why: an issuer is not
  an object store, so keeping only one side drops the other's coverage.
- get_merged_domains keeps the loud refusal when svc-net-tor is deployed
  without a provisioned node, now gated on their tor_serves_host; their
  node_onion_of returns an empty string there, which would have served every
  onion-exclusive app on clearnet in silence.
- sso_cookie_domains stays the single source for oauth2-proxy's cookie scope.
  Their per-network branch omitted the issuer domain on the onion arm, which
  breaks the callback leg.
- The clearnet CSP batch carries --clearnet-timeout again; their batch dict
  hardcoded zero while the role still passes the flag.
- The dashboard keeps its i18n content dir, image, version and design
  resolution; their side reduced the logo sources to the bare service
  provider values.

Their side wins where it is measurably better: the onion mirror rewrite is
gated per vhost rather than per app, the Playwright onion decision reads the
pass's own domain instead of the role's canonical one, and the matomo
reinjection and dashboard config render per network sibling.

Eight files were broken by the merge without ever carrying a conflict marker,
because git marks only the lines both sides touched:

- utils/cache/domains.py lost the onionize block that sat between two hunks
- roles/sys-front-inj-all/templates/body_filter.lua.j2 called it
- web-app-bluesky, keycloak sso_proxy: the get_entity_name filter was renamed
- trigger/__main__.py, playwright_coverage.py and its test: the deleted tor
  module
- cli/administration/deploy/ci/timings.py read Label.tor
- plugins/filter/csp_filters.py called _aligned_url with four of six arguments
- tests/unit/python/utils/cache/test_domains.py referenced a renamed class
- tests/.../sys-ctl-hlth-csp/files/test_script.py mocked a function main no
  longer calls

Verified: the variant layer (155 tests, 65 subtests), ca_injected (18),
the CI layer (175), the domains cache including the restored guard (32),
retrigger inputs (9), timings and the CSP health script (53). make
quality-high clears autoformat; its remaining findings are tracked separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-23 09:53:41 +02:00
kevinveenbirkenbachandClaude Opus 5 74a91e7adf refactor(ci): declare the README replay as a service edge instead of a matrix flag
The replay used to be decided by the matrix: utils.roles.guide picked a
variant, instructions.mark wrote a flag, and the workflow appended
web-app-docs to the row's apps. Now each invokable role declares the
guide host in meta/services.yml with bond 0.5 behind the group gate and
switches it on in exactly one variant, and the deploy pulls it in
through that edge. The matrix only reads the declaration back.

web-app-docs was no provider at all: the registry requires shared,
provides or an alias on the role's own entity entry and it had none, so
the 188 declarations would have been decorative. Registering it brings
a structural contract with it -- a dynamically loaded service role must
set its run_once flag as its very first task or load/app.yml includes
it again -- so the role body moves into tasks/00_core.yml and main.yml
becomes the guard.

smallest_variant no longer counts the guide. It picks the variant that
then switches the guide on, which on the next read would make that
variant one service larger and no longer the smallest; without the
exclusion 30 of 188 roles disagreed with the file.

The job title loses the marker. The row is an ordinary deploy of its
variant, so nothing distinguishes it any more -- which is why the unit
test now asserts the label is left alone.

Measured against the previous tree, the sweep marks 68 rows where it
marked 88. Twenty roles declare the guide on a variant this sweep does
not run in the guide's mode, and by decision the declaration is
absolute: the file is the truth and those roles carry no replay in such
a run. The old code instead searched the deployed variants, which hid
the mismatch behind a fallback.

The same measurement found that the predicate first read the group gate
as a declaration -- the gate renders to a non-empty string, so every
variant looked like the declaring one. It now tests for a literal true.
Neither defect showed up in the unit tests, which check the new logic
rather than its difference to the old.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-23 02:30:56 +02:00
kevinveenbirkenbachandClaude Opus 5 d9f8af9b16 fix(ci): refuse a mesh pin the row cannot take
A selection token may pin the mesh state as &vpn or &direct, but check_pins
had no arm for it. pinned_vpn_states intersects the pin with the states the
mode allows and only swarm meshes, so a token like @compose+clearnet&vpn left
an empty state list: the row produced no job at all and the run reported green
for a combination that never deployed. That is the outcome _reject exists to
prevent, and every sibling axis already refuses an impossible pin.

The new arm asks whether any candidate mode keeps a state rather than whether
the pin is reachable, so a held vpn axis still overrides the pin instead of
being refused. That behaviour has its own test and is unchanged.

An earlier check was removed with the compose-mesh revert of 2026-10-16
because it rested on meshable() and smallest_variant(), which that revert
deleted. This one reuses pinned_vpn_states, applies to the swarm-only axis the
revert restored, and covers both mesh states instead of only &vpn.

pin_vpn and vpn_mode now reach check_pins from all three callers. The run's
vpn axis reaches the validation job through call-orchestrator and
call-validation, which passed every other axis but this one. The priority
procedure and its skill state all eight axes a token carries, not six:
architecture and mesh were pinnable and, for architecture, already validated,
yet neither was documented.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-21 18:13:54 +02:00
kevinveenbirkenbachandClaude Opus 5 78340e78be fix(ci): pin the matrix generator to ubuntu-26.04
main rejects a moving runner label since 52ad8549e0, because ubuntu-latest moved to 26.04 in the middle of run 39166850273 and every job that landed on the new image met a different OS. 9bdecaa794 pinned the three places on this branch that still carried the moving label, but the matrix generator was a fourth and kept emitting it for every amd64 row.

It was missed for the reason that commit names: the updater of b055eece05 raises a bare runs-on: ubuntu-NN.NN and leaves everything else alone, and here the label lives in a Python dict rather than on a runs-on line. That is the same blind spot 9bdecaa794 describes for the architecture ternary of call-images-build-ci.yml, which it also had to raise by hand. tests/lint/repository/test_workflow_runner_pinned.py does not reach this file either: it passed with ubuntu-latest in place.

The drift is visible in the runs. On this fork ubuntu-latest resolves to 24.04, whose system Python is 3.12, while main requests 26.04 and runs 3.14, and .github/workflows/README.md documents 26.04 for amd64. The branch is uniformly about twice as slow per ansible task as main at every percentile, and the interpreter is one of three candidates for that alongside the new controller-side code and the fork-versus-upstream difference. Pinning the generator makes the platforms comparable, which is what lets that question be answered at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-21 16:26:56 +02:00
kevinveenbirkenbachandClaude Opus 5 386fb8cdff fix(docs): stop re-exported types from being documented twice
autodoc documents whatever __all__ lists, an imported name included, and apidoc also writes a page for the submodule the name comes from. An annotation naming such a type resolves to two targets, which -W turns fatal: the documentation catalogs could not be extracted at all.

Only a type reaches that failure, because only a type appears in the annotations sphinx resolves. mesh re-exported Mesh, MeshMember and MeshSpec; libretranslate re-exported LibreTranslate and Outcome; the development inventory package still listed PlanEntry beside the DevInventorySpec that 7fee974c7f removed for this reason. Each now uses the redundant alias the repository already established, which keeps the name importable without listing it. Label in variant/axes.py had no importer and is gone.

Re-exporting a function stays allowed: the development inventory package re-exports its helpers so unit tests can patch them under the package path, and no function has ever produced a second target.

A lint pins the class, because the docstring 7fee974c7f relied on did not hold a third time. It reads the trees apidoc skips from APIDOC_EXCLUDED, which commands.py now names once instead of repeating the three paths inside the argv.

locale/ is the extract this unblocks: 26144 messages over 183 catalogs. 987761 translations carry over, 8893 lose a msgid that no longer exists, and 1136 keep their msgid but come back empty for the next i18n-translate to refill.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-21 02:39:36 +02:00
kevinveenbirkenbachandClaude Opus 5 fbd4b3a18d refactor(meta): label categories with one hieroglyph
A deploy job name repeats the category path on every row of the matrix. One Egyptian hieroglyph costs four UTF-8 bytes where the two Chinese characters it replaces cost six. Unicode names each sign by its Gardiner code, so the opaque value stays identifiable without a comment per entry.

A label has to stay one codepoint: a sign written with a variation selector costs seven bytes and is worse than what it replaced. tests/integration/meta/categories/test_glyph.py enforces that.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-20 23:17:19 +02:00
kevinveenbirkenbach fdb6e668b2 Merge branch 'main' into feature/network/multi-network-serving
# Conflicts:
#	utils/github/variant/axes.py
2026-10-20 22:56:45 +02:00
kevinveenbirkenbachandClaude Opus 5.5 6e776a5780 refactor(python): clear the open CodeQL code-quality alerts
- Declare `__all__` in every package `__init__.py` whose `PROJECT_ROOT`
  is imported by its modules (163 packages), so the constant counts as
  exported; list every public name where the package defines more.
- Empty the 15 package `__init__.py` files whose `PROJECT_ROOT` no module
  imports.
- Declare the public API of `utils.github.variant.axes` in `__all__`,
  re-exports included, instead of silencing them with `noqa: F401`; the
  list matches the one `feature/store-i18n` already carries.
- Use `import unittest.mock as mock` in `test_applications.py`, as the
  rest of the suite does.
- Spell `_NON_PRINTABLE` in `utils/yaml_bootstrap.py` as the equivalent
  BMP-only positive class; it matches exactly the same 2111 code points
  as the old negated class across U+0000..U+10FFFF.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 21:40:58 +02:00
kevinveenbirkenbachandClaude Opus 5.5 dd81049317 fix(variant): return the vpn pin match directly and give wazuh its meshes key
Follow-up to the main merge: ruff SIM103 rejected the if/return-True tail of
covers(), and the networks lint requires every role to declare meshes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 17:14:40 +02:00
kevinveenbirkenbachandClaude Opus 5 0c695a80d5 refactor(variant): split pin validation out of axis assignment
Our side added the architecture axis and main the vpn axis; combining
both put axes.py at 502 lines against the 500 the structure lint allows.

check_pins and row_architectures answer one question: may this row take
what the selection token demands of it. Assigning the axes of a row that
carries no pin is the other job, so the two separate cleanly and axes.py
drops to 397 lines.

axes.__all__ still exports both names, so cli/meta/ci/validate.py keeps
reaching them through axes. The test patches get_role_architectures,
which moved with them, so its patch target moves too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-19 02:32:32 +02:00
kevinveenbirkenbach 15930945fe Merge branch 'main' into feature/store-i18n 2026-10-19 01:54:37 +02:00
kevinveenbirkenbachandClaude Opus 5.5 7f28564548 Merge branch 'main' into feature/network/multi-network-serving
Combine main's WireGuard mesh axis with this branch's network axis in the CI
variant code. Every matrix row now carries both: network (clearnet, tor,
multi) from utils.github.variant.network and vpn (meshed or direct) from
utils.github.variant.vpn. Label parsing, which main moved to label.py, reads
the network glyph instead of the old tor flag and keeps the vpn glyph; the
selection token, plan table, matrix and trigger CLIs, the workflows and the
swarm routine forward both axes. The test of an incapable priority variant
now compares the set of network modes, since a swarm priority row also
covers both mesh arms.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-19 00:15:05 +02:00
evangelostsak 99fcca4d1b feat(ci): compose titles carry the direct arm glyph
A compose row never joins a WireGuard mesh, so its title shows 🛣️ like a
direct swarm row. The vpn axis, the planner and the deploy are unchanged.
2026-10-17 00:03:18 +02:00
evangelostsak 5d7327d911 revert(ci): compose titles carry no mesh glyph and take no mesh pin check 2026-10-16 21:26:16 +02:00
evangelostsak dc84a4c9ac revert(ci): the vpn axis covers swarm rows only again 2026-10-16 21:26:16 +02:00
evangelostsak 0b1bc82d57 refactor: drop rationale and narration from utils/github/variant/vpn.py 2026-10-16 19:10:01 +02:00
evangelostsak f5b2979c45 refactor: drop rationale and narration from utils/github/variant/selection.py 2026-10-16 19:10:01 +02:00
evangelostsak 216f6ea170 refactor: drop rationale and narration from utils/github/variant/label.py 2026-10-16 19:10:01 +02:00
evangelostsak aaecbccf8c refactor: drop rationale and narration from utils/github/variant/axes.py 2026-10-16 19:10:01 +02:00
evangelostsak 1860f587b4 feat(ci): show the mesh state on every compose title and refuse a mesh pin it cannot take
A compose row has two behaviours, with or without the mesh, but its title
showed three: no glyph on a role's larger variants, 🛣️ or 🚇 on its smallest.
No glyph and 🛣️ deploy the same thing, so every compose title now reads 🛣️
or 🚇 like a swarm title, and host titles stay without one. Only the smallest
variant still rotates onto the mesh.

A &vpn pin on a row no mode of which can take the mesh now aborts the run
instead of quietly deploying 🛣️, and a regular row pinned to the mesh only
rotates onto a mode that can carry it.
2026-10-16 14:48:36 +02:00
evangelostsak a206abd8d5 docs(ci): every swarm and compose row carries the mesh or direct glyph 2026-10-16 14:48:35 +02:00
evangelostsak 5cd28fd83a refactor(ci): drop the re-export of the removed mesh glyph modes 2026-10-16 03:40:16 +02:00
evangelostsak de804fe304 feat(ci): assign the vpn axis and its glyph to smallest compose variants
A compose row of any other variant keeps its title unchanged.
2026-10-16 03:40:16 +02:00
evangelostsak 1052be4754 feat(ci): compose rows of a role's smallest variant take the vpn axis
The same input and rotation as swarm. Only the smallest variant is meshable
in compose, so the CLI test of svc-net-wireguard runs once per role rather
than once per variant.
2026-10-16 03:40:16 +02:00
evangelostsak 1c5ba164f2 Merge remote-tracking branch 'upstream/main' into feature/svc-net-wireguard
No conflicts across 121 incoming commits. The one file both sides changed is
tasks/stages/01_constructor.yml: upstream parks an applications carrier for
the swarm bootstrap roles after the add_host tasks, and keys the carrier on
the tokens file as well. This branch's mesh bootstrap opens and drops its own
block-scoped carrier before that point, so both keep working as written.

Upstream also brings ruff 0.17.0, distro-derived env keys, a scoped act outer
teardown, the stalwart relay rule, the onion apex redirect and image and
version updates; ruff 0.17 reformats nothing on this branch.
2026-10-15 21:27:11 +02:00
kevinveenbirkenbach 1da2cef95e Merge remote-tracking branch 'origin/main' into feature/network/multi-network-serving
# Conflicts:
#	plugins/lookup/email.py
#	tests/unit/python/roles/web-app-prometheus/test_blackbox_targets.py
2026-10-15 16:45:18 +02:00
kevinveenbirkenbach 26a7a273ff Merge branch 'main' into feature/store-i18n 2026-10-15 04:49:45 +02:00
kevinveenbirkenbachandClaude Opus 5.5 47565781ea style(python): import datetime as dt for ruff 0.17
ruff 0.17.0 adds `datetime` -> `dt` to the default aliases of
flake8-import-conventions, and the project selects ICN, so lint-python
failed with ICN001 in seven files.

The files now import `datetime as dt`. ruff's unsafe fix also renamed
two local names in utils/github/playwright_summary.py into the alias:
the `_to_iso_z` parameter and the parsed value in
`_parse_iso_timestamp`. Both are now called `moment`; with the bare fix,
nine tests of test_playwright_summary.py failed with UnboundLocalError
and AttributeError.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 01:24:31 +02:00
evangelostsak a4048b43f8 fix(ci): rotate swarm rows over both mesh arms within one sweep
The mode rotation picks swarm for a row when (position + sweep) is odd, so
every swarm row of a sweep shares one position parity. wants_vpn rotated on
(position + sweep // 4) % 2, the same digit, so all swarm rows of a default
run landed on one arm: a sweep was entirely meshed or entirely direct and
never compared the two.

Reading the position's second digit (position // 2) splits the swarm rows of
every sweep about evenly between the arms, and sweep // 4 still flips each
row on its own schedule. Compose rows are untouched: they never take the
mesh.
2026-10-14 00:27:17 +02:00
evangelostsak cf9803a8a8 docs(ci): reduce the vpn variant docstrings to summary and Args
The axis is documented with the other axes, not per function.
2026-10-05 12:43:23 +02:00
evangelostsak ff75343aba fix(ci): stop a held mesh axis deleting priority rows
The pin narrowed the cross product after the axis was resolved, so a row
pinned against an enforced or disabled run lost every combination and left
the matrix entirely -- while a regular row with the same pin was simply moved
onto the held side. A priority row now takes the same precedence a regular
row already took: the run's axis first, the pin second.
2026-10-02 00:54:20 +02:00
evangelostsak 89ab678a19 feat(ci): take the run's mesh mode when assigning rows
Also moves the title codec to label.py and re-exports it, which is what keeps
the file under the line cap.
2026-09-26 08:31:11 +02:00
evangelostsak 85e9112409 refactor(ci): split the job-title codec out of the assigner
Reading a title back into its axes is the inverse of building one, and it kept
axes.py over the 500-line cap once the mesh axis landed. The names stay
importable from axes so no consumer moves.
2026-09-26 08:31:01 +02:00
evangelostsak 4be2a381ab feat(ci): let a run hold the mesh axis on one side
auto rotates, enforced meshes every swarm row, disabled meshes none. No
exclusive: every swarm row can take the mesh, so there is nothing to drop.

A run that holds the axis outranks a per-token pin, because an operator who
asked for a whole sweep on one side means it.
2026-09-26 08:30:44 +02:00
evangelostsak 997b2f5091 fix(ci): count the mesh state as a narrowing axis
Without it two tokens differing only in the mesh state compare equal and
collapse into one, and a bare role&vpn reads as narrowing nothing.
2026-09-26 02:11:40 +02:00
evangelostsak 61d5206e3b feat(ci): pin the mesh state from a selection token
role&vpn / role&direct, and the same state read back off a pasted job title.
Own separator rather than more + words: + is the onion axis, and one
separator meaning two things could not be read back unambiguously.
2026-09-25 22:34:09 +02:00
evangelostsak 3128d74ba5 feat(ci): assign the mesh axis to every deploy row
Entries carry vpn, labels carry the glyph, and the artifact slug carries the
state so a meshed and a direct run of one row cannot collide on one artifact
name. Priority rows cover both states; compose never takes the axis.
2026-09-25 22:34:04 +02:00
evangelostsak 6391761c05 feat(ci): add the WireGuard mesh deploy axis
A swarm deploy runs either over the mesh or straight over the underlay, and
both have to keep working: the mesh joins physically separated clusters, so a
single-site swarm has no use for it and must not depend on it.

Quarters on sweep // 4 so it flips in step with neither the mode rotation nor
the onion, and a row walks every mode/tor/vpn combination over eight sweeps.
2026-09-25 22:33:55 +02:00
kevinveenbirkenbachandClaude Opus 5.5 098b0caffc fix(ci): offer multi to bond-less rows whose deploy pulls Tor in
A role without a Tor bond only took clearnet rows, and clearnet rows disable
Tor. svc-ai-litellm (variants 0 and 2) and web-app-seaweedfs (variant 0) still
pull the Tor provider in through their dependencies, so they ran on a node
with Tor about half the time under the tor axis and never under the network
axis.

row_states now gives such a row clearnet and multi. Whether the deploy pulls
the provider in is read from the services closure the discovery row already
carries, which matched the deploy's own include resolution on all 33 rows
without a bond. A row whose deploy contains no Tor provider stays clearnet
only, because NETWORK_MODE=multi without the provider fails the deploy, and a
gate pinned false still means clearnet. The matrix and the selection
validator pass the same flag, so a +multi pin agrees with the matrix.
tor_capable lost its last caller and is gone.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 22:42:55 +02:00
kevinveenbirkenbachandClaude Opus 5.5 6314938133 feat(ci): replace the tor axis with a network axis of node modes
Every deploy row now takes a node network mode, clearnet, tor or multi,
instead of an onion yes or no. The workflow input `network`
(auto|clearnet|tor|multi) replaces `tor`, and INFINITO_NETWORK replaces
INFINITO_TOR.

utils/github/variant/network.py decides which modes a row can take: clearnet
only without a Tor bond or when the variant pins Tor off, no multi for a
single_mode role, no clearnet for the Tor provider, whose rows are therefore no
longer dropped when the rotation lands on clearnet. Priority rows cover every
mode and network combination, regular rows rotate the network on sweep // 2,
and clearnet rows carry disable=tor. Selection tokens pin the axis with
+clearnet, +tor or +multi, and multi rows wear the new glyph.

A row hands its mode to the deploy as the variable `network`, which
provisioning writes into host_vars as NETWORK_MODE. Without it the node keeps
deriving its mode from the deployed providers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 22:40:20 +02:00
kevinveenbirkenbachandClaude Opus 5 c7fb6e60c5 fix(ci): narrow a row's architectures by what its variant deploys
row_architectures intersected the run's pool with the declarations of
every role in the complexity closure. That closure follows every
conditional provider flag through the base graph, so tor -> web-svc-mirror
-> prometheus -> openwebui -> litellm -> lmstudio put the amd64-only
lmstudio into the closure of 93 of 128 roles, including the minimal
variants that never deploy it. A single amd64 pin would have withdrawn
arm64 from nearly the whole matrix.

The matrix now replaces a row's services with the include set that
plan_dev_inventory_matrix, the planner the deploy step itself uses,
resolves for that variant. Against CI run 35674334334 it matches every
data point: lmstudio for zammad#2, moodle#2 and xwiki#2, which pulled it
on arm64, and not for docs#0 or legal#0, which did not. With the pins of
the next commit, 126 of 128 roles keep arm64 rows instead of 34.

Without any architecture declaration the emitted matrix is byte-identical
to before under a fixed discovery seed. The planner adds about 37 s per
matrix build, and a role whose plan errors now stops the matrix instead of
only its own deploy job.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 16:01:09 +02:00
kevinveenbirkenbachandClaude Opus 5 9e8d190f14 fix(ci): narrow a row's architecture by its service closure too
A row drew its architecture from the run's pool narrowed by the role's
own meta/services.yml declaration only. The deploy also brings up every
role in the transitive service closure, so a single-architecture
provider still received rows on hardware it cannot run on.
row_architectures now intersects the pool with the declarations of the
role and of each closure role, which assign() passes from the
complexity row.

No role declares architectures today, so no current row changes. The pin
validator in cli/meta/ci/validate.py still checks the role alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 02:08:55 +02:00
kevinveenbirkenbach eea14a0cfb Merge branch 'main' into feature/store-i18n 2026-09-21 23:35:55 +02:00
kevinveenbirkenbachandClaude Opus 5 5b8465260f feat(ci): deploy only the variants a change reaches
An image bump in a provider redeployed every variant of every consumer,
including the variants that switch that provider off and therefore never
deploy the bumped image at all. The diff-derived whitelist now narrows each
affected role to the variants whose own round closure still reaches a seed,
emitted as `role#0,2` selection tokens. That closure is walked on the same
variant-merged services maps the matrix planner builds its rounds from, so the
narrowing follows a parent's variant choice across every generation of the
chain. It covers every auto-derived whitelist: the weekly update PRs,
Dependabot, and any other diff-driven run.

The narrowing never shrinks the matrix on partial information. A seed role
keeps every variant, because its own files changed. A role reached only
through a run_after edge keeps every variant too, because the round closure
does not follow that edge and a role whose hits are empty is therefore
unnarrowed rather than dropped.

A pinned token naming a role the discovery query does not return now warns
instead of aborting the run. The diff-derived whitelist legitimately names
roles the lifecycle envelope filters out, and once it pins variants such a
token would abort every chunk; role names are still checked against roles/
before the run starts, so a typo is caught one layer earlier.

The round's variant merge moves out of the deploy inventory package into
utils/roles/applications/variants.py. The CI selection has to build the same
maps, and importing the deploy package for them pulls in a module that reads
INFINITO_SRC_DIR at import time, which the runner host does not set.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-21 22:51:03 +02:00
kevinveenbirkenbachandClaude Opus 5 2a7455bd90 feat(ci): spread the deploy matrix over amd64 and arm64
Both architectures were claimed to work and only one was ever exercised. The
deploy matrix now carries an architecture the way it carries mode, tor, distro
and filesystem, so every run proves both.

A job cannot ask for an architecture the way it asks for a distro, because the
architecture is the machine rather than something the job installs on it. The
matrix therefore carries a runner label and `runs-on` reads it, and an
architecture without a label in RUNNERS is a build error rather than a row that
lands on the wrong hardware and still reports green. assert_architecture.sh
checks the runner it actually got, because a label that stops resolving to
arm64 would otherwise go unnoticed forever.

Assignment is rotation, not chance: two architectures against five distros and
fifteen distro/filesystem positions are coprime enough that every combination
comes round. A role narrows the pool through `architectures` in its
meta/services.yml; without one it takes whatever the rotation hands it. An
empty intersection aborts instead of silently removing the role from the run.

Images are built per architecture and pushed under an architecture-suffixed
tag, then merged into the plain tag by manifest_all.sh; pushing the plain tag
from both builds would let the last one win and send half the rows to the wrong
platform. cleanup.sh now protects the `ci-<sha>-<architecture>` children of a
protected tag, because the plain tag is an index over them and deleting a child
leaves it unpullable on that platform.

label.py is split out of axes.py, which the 500-line limit no longer fits.
Writing a job title and reading one back belong together: a retrigger that
recovers an axis wrongly replays a different deploy than the one that failed,
and a drifting artifact name makes the failure reporter download nothing. The
leading `.*` in LABEL_RE is greedy on purpose so it anchors on the last mode
glyph; a reusable-workflow caller path carries a glyph of its own, and
test_selection.py guards the case.

Verified: 313 unit tests and 48 subtests across utils/github/variant,
utils/roles and cli/meta/ci. Locally, a deploy pinned to arm64 reaches ok=1444
and its image builds log aarch64-linux-gnu paths.

Not verified here: actionlint over the seven changed workflows, and a full
`make test`. The matrix doubles the row count, so runtime and runner
availability change for every CI run from here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 07:02:44 +02:00
kevinveenbirkenbachandClaude Opus 5 4193da1cb2 fix(ci): replay the instructions on a row that runs in the guide's mode
The replay is a second deploy on the same runner, and mark() only ever checked the variant number of a row, never its mode. So the marker landed on whichever row of the smallest variant came first, which for 38 of 98 marked rows was a swarm one: that runner would have had to bring a compose stack up beside a swarm stack.

Two places needed it. The row now has to run in the mode guide_deployable names, and only rows in that mode count as deployed when the search looks for the smallest variant - without the second half the search settles on a variant this sweep runs in swarm alone and the marker falls away silently.

Measured over the full matrix: 98 marked rows become 71, swarm drops from 38 to 0, compose rises from 47 to 59. The 27 roles that lose the replay entirely are the ones this sweep deploys in no compose row of an eligible variant; getting them back means forcing such a row into the sweep, which is a change to matrix selection rather than to this marker.

mark() had no tests. It has eight now, including the two failure modes the old code showed under a negative control: a swarm row carrying instructions=compose, and a compose row left unmarked because a swarm-only variant won the search.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 22:21:13 +02:00