Commit Graph
90 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 677a63931c Merge branch 'feature/network/multi-network-serving' into feature/store-i18n
Resolves 68 conflicted files. The governing rule was our structure with their
network semantics: the tor boolean becomes a network mode (clearnet | tor |
multi) throughout the variant layer, while this branch's architecture and mesh
axes, its extracted modules and its i18n content stay.

Where the two sides covered different ground, both are kept rather than one
picked:

- ca_injected answers a ca_client on the deployment's TLS mode OR on an SSO
  issuer that resolves self-signed. 3a3e9ed2c9 records why: an issuer is not
  an object store, so keeping only one side drops the other's coverage.
- get_merged_domains keeps the loud refusal when svc-net-tor is deployed
  without a provisioned node, now gated on their tor_serves_host; their
  node_onion_of returns an empty string there, which would have served every
  onion-exclusive app on clearnet in silence.
- sso_cookie_domains stays the single source for oauth2-proxy's cookie scope.
  Their per-network branch omitted the issuer domain on the onion arm, which
  breaks the callback leg.
- The clearnet CSP batch carries --clearnet-timeout again; their batch dict
  hardcoded zero while the role still passes the flag.
- The dashboard keeps its i18n content dir, image, version and design
  resolution; their side reduced the logo sources to the bare service
  provider values.

Their side wins where it is measurably better: the onion mirror rewrite is
gated per vhost rather than per app, the Playwright onion decision reads the
pass's own domain instead of the role's canonical one, and the matomo
reinjection and dashboard config render per network sibling.

Eight files were broken by the merge without ever carrying a conflict marker,
because git marks only the lines both sides touched:

- utils/cache/domains.py lost the onionize block that sat between two hunks
- roles/sys-front-inj-all/templates/body_filter.lua.j2 called it
- web-app-bluesky, keycloak sso_proxy: the get_entity_name filter was renamed
- trigger/__main__.py, playwright_coverage.py and its test: the deleted tor
  module
- cli/administration/deploy/ci/timings.py read Label.tor
- plugins/filter/csp_filters.py called _aligned_url with four of six arguments
- tests/unit/python/utils/cache/test_domains.py referenced a renamed class
- tests/.../sys-ctl-hlth-csp/files/test_script.py mocked a function main no
  longer calls

Verified: the variant layer (155 tests, 65 subtests), ca_injected (18),
the CI layer (175), the domains cache including the restored guard (32),
retrigger inputs (9), timings and the CSP health script (53). make
quality-high clears autoformat; its remaining findings are tracked separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-23 09:53:41 +02:00
kevinveenbirkenbachandClaude Opus 5 74a91e7adf refactor(ci): declare the README replay as a service edge instead of a matrix flag
The replay used to be decided by the matrix: utils.roles.guide picked a
variant, instructions.mark wrote a flag, and the workflow appended
web-app-docs to the row's apps. Now each invokable role declares the
guide host in meta/services.yml with bond 0.5 behind the group gate and
switches it on in exactly one variant, and the deploy pulls it in
through that edge. The matrix only reads the declaration back.

web-app-docs was no provider at all: the registry requires shared,
provides or an alias on the role's own entity entry and it had none, so
the 188 declarations would have been decorative. Registering it brings
a structural contract with it -- a dynamically loaded service role must
set its run_once flag as its very first task or load/app.yml includes
it again -- so the role body moves into tasks/00_core.yml and main.yml
becomes the guard.

smallest_variant no longer counts the guide. It picks the variant that
then switches the guide on, which on the next read would make that
variant one service larger and no longer the smallest; without the
exclusion 30 of 188 roles disagreed with the file.

The job title loses the marker. The row is an ordinary deploy of its
variant, so nothing distinguishes it any more -- which is why the unit
test now asserts the label is left alone.

Measured against the previous tree, the sweep marks 68 rows where it
marked 88. Twenty roles declare the guide on a variant this sweep does
not run in the guide's mode, and by decision the declaration is
absolute: the file is the truth and those roles carry no replay in such
a run. The old code instead searched the deployed variants, which hid
the mismatch behind a fallback.

The same measurement found that the predicate first read the group gate
as a declaration -- the gate renders to a non-empty string, so every
variant looked like the declaring one. It now tests for a literal true.
Neither defect showed up in the unit tests, which check the new logic
rather than its difference to the old.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-23 02:30:56 +02:00
kevinveenbirkenbachandClaude Opus 5 d9f8af9b16 fix(ci): refuse a mesh pin the row cannot take
A selection token may pin the mesh state as &vpn or &direct, but check_pins
had no arm for it. pinned_vpn_states intersects the pin with the states the
mode allows and only swarm meshes, so a token like @compose+clearnet&vpn left
an empty state list: the row produced no job at all and the run reported green
for a combination that never deployed. That is the outcome _reject exists to
prevent, and every sibling axis already refuses an impossible pin.

The new arm asks whether any candidate mode keeps a state rather than whether
the pin is reachable, so a held vpn axis still overrides the pin instead of
being refused. That behaviour has its own test and is unchanged.

An earlier check was removed with the compose-mesh revert of 2026-10-16
because it rested on meshable() and smallest_variant(), which that revert
deleted. This one reuses pinned_vpn_states, applies to the swarm-only axis the
revert restored, and covers both mesh states instead of only &vpn.

pin_vpn and vpn_mode now reach check_pins from all three callers. The run's
vpn axis reaches the validation job through call-orchestrator and
call-validation, which passed every other axis but this one. The priority
procedure and its skill state all eight axes a token carries, not six:
architecture and mesh were pinnable and, for architecture, already validated,
yet neither was documented.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-21 18:13:54 +02:00
kevinveenbirkenbachandClaude Opus 5 78340e78be fix(ci): pin the matrix generator to ubuntu-26.04
main rejects a moving runner label since 52ad8549e0, because ubuntu-latest moved to 26.04 in the middle of run 39166850273 and every job that landed on the new image met a different OS. 9bdecaa794 pinned the three places on this branch that still carried the moving label, but the matrix generator was a fourth and kept emitting it for every amd64 row.

It was missed for the reason that commit names: the updater of b055eece05 raises a bare runs-on: ubuntu-NN.NN and leaves everything else alone, and here the label lives in a Python dict rather than on a runs-on line. That is the same blind spot 9bdecaa794 describes for the architecture ternary of call-images-build-ci.yml, which it also had to raise by hand. tests/lint/repository/test_workflow_runner_pinned.py does not reach this file either: it passed with ubuntu-latest in place.

The drift is visible in the runs. On this fork ubuntu-latest resolves to 24.04, whose system Python is 3.12, while main requests 26.04 and runs 3.14, and .github/workflows/README.md documents 26.04 for amd64. The branch is uniformly about twice as slow per ansible task as main at every percentile, and the interpreter is one of three candidates for that alongside the new controller-side code and the fork-versus-upstream difference. Pinning the generator makes the platforms comparable, which is what lets that question be answered at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-21 16:26:56 +02:00
kevinveenbirkenbachandClaude Opus 5 386fb8cdff fix(docs): stop re-exported types from being documented twice
autodoc documents whatever __all__ lists, an imported name included, and apidoc also writes a page for the submodule the name comes from. An annotation naming such a type resolves to two targets, which -W turns fatal: the documentation catalogs could not be extracted at all.

Only a type reaches that failure, because only a type appears in the annotations sphinx resolves. mesh re-exported Mesh, MeshMember and MeshSpec; libretranslate re-exported LibreTranslate and Outcome; the development inventory package still listed PlanEntry beside the DevInventorySpec that 7fee974c7f removed for this reason. Each now uses the redundant alias the repository already established, which keeps the name importable without listing it. Label in variant/axes.py had no importer and is gone.

Re-exporting a function stays allowed: the development inventory package re-exports its helpers so unit tests can patch them under the package path, and no function has ever produced a second target.

A lint pins the class, because the docstring 7fee974c7f relied on did not hold a third time. It reads the trees apidoc skips from APIDOC_EXCLUDED, which commands.py now names once instead of repeating the three paths inside the argv.

locale/ is the extract this unblocks: 26144 messages over 183 catalogs. 987761 translations carry over, 8893 lose a msgid that no longer exists, and 1136 keep their msgid but come back empty for the next i18n-translate to refill.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-21 02:39:36 +02:00
kevinveenbirkenbachandClaude Opus 5 fbd4b3a18d refactor(meta): label categories with one hieroglyph
A deploy job name repeats the category path on every row of the matrix. One Egyptian hieroglyph costs four UTF-8 bytes where the two Chinese characters it replaces cost six. Unicode names each sign by its Gardiner code, so the opaque value stays identifiable without a comment per entry.

A label has to stay one codepoint: a sign written with a variation selector costs seven bytes and is worse than what it replaced. tests/integration/meta/categories/test_glyph.py enforces that.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-20 23:17:19 +02:00
kevinveenbirkenbach fdb6e668b2 Merge branch 'main' into feature/network/multi-network-serving
# Conflicts:
#	utils/github/variant/axes.py
2026-10-20 22:56:45 +02:00
kevinveenbirkenbachandClaude Opus 5.5 6e776a5780 refactor(python): clear the open CodeQL code-quality alerts
- Declare `__all__` in every package `__init__.py` whose `PROJECT_ROOT`
  is imported by its modules (163 packages), so the constant counts as
  exported; list every public name where the package defines more.
- Empty the 15 package `__init__.py` files whose `PROJECT_ROOT` no module
  imports.
- Declare the public API of `utils.github.variant.axes` in `__all__`,
  re-exports included, instead of silencing them with `noqa: F401`; the
  list matches the one `feature/store-i18n` already carries.
- Use `import unittest.mock as mock` in `test_applications.py`, as the
  rest of the suite does.
- Spell `_NON_PRINTABLE` in `utils/yaml_bootstrap.py` as the equivalent
  BMP-only positive class; it matches exactly the same 2111 code points
  as the old negated class across U+0000..U+10FFFF.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 21:40:58 +02:00
kevinveenbirkenbachandClaude Opus 5.5 dd81049317 fix(variant): return the vpn pin match directly and give wazuh its meshes key
Follow-up to the main merge: ruff SIM103 rejected the if/return-True tail of
covers(), and the networks lint requires every role to declare meshes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 17:14:40 +02:00
kevinveenbirkenbachandClaude Opus 5 0c695a80d5 refactor(variant): split pin validation out of axis assignment
Our side added the architecture axis and main the vpn axis; combining
both put axes.py at 502 lines against the 500 the structure lint allows.

check_pins and row_architectures answer one question: may this row take
what the selection token demands of it. Assigning the axes of a row that
carries no pin is the other job, so the two separate cleanly and axes.py
drops to 397 lines.

axes.__all__ still exports both names, so cli/meta/ci/validate.py keeps
reaching them through axes. The test patches get_role_architectures,
which moved with them, so its patch target moves too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-19 02:32:32 +02:00
kevinveenbirkenbach 15930945fe Merge branch 'main' into feature/store-i18n 2026-10-19 01:54:37 +02:00
kevinveenbirkenbachandClaude Opus 5.5 7f28564548 Merge branch 'main' into feature/network/multi-network-serving
Combine main's WireGuard mesh axis with this branch's network axis in the CI
variant code. Every matrix row now carries both: network (clearnet, tor,
multi) from utils.github.variant.network and vpn (meshed or direct) from
utils.github.variant.vpn. Label parsing, which main moved to label.py, reads
the network glyph instead of the old tor flag and keeps the vpn glyph; the
selection token, plan table, matrix and trigger CLIs, the workflows and the
swarm routine forward both axes. The test of an incapable priority variant
now compares the set of network modes, since a swarm priority row also
covers both mesh arms.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-19 00:15:05 +02:00
evangelostsak 99fcca4d1b feat(ci): compose titles carry the direct arm glyph
A compose row never joins a WireGuard mesh, so its title shows 🛣️ like a
direct swarm row. The vpn axis, the planner and the deploy are unchanged.
2026-10-17 00:03:18 +02:00
evangelostsak 5d7327d911 revert(ci): compose titles carry no mesh glyph and take no mesh pin check 2026-10-16 21:26:16 +02:00
evangelostsak dc84a4c9ac revert(ci): the vpn axis covers swarm rows only again 2026-10-16 21:26:16 +02:00
evangelostsak 0b1bc82d57 refactor: drop rationale and narration from utils/github/variant/vpn.py 2026-10-16 19:10:01 +02:00
evangelostsak f5b2979c45 refactor: drop rationale and narration from utils/github/variant/selection.py 2026-10-16 19:10:01 +02:00
evangelostsak 216f6ea170 refactor: drop rationale and narration from utils/github/variant/label.py 2026-10-16 19:10:01 +02:00
evangelostsak aaecbccf8c refactor: drop rationale and narration from utils/github/variant/axes.py 2026-10-16 19:10:01 +02:00
evangelostsak 1860f587b4 feat(ci): show the mesh state on every compose title and refuse a mesh pin it cannot take
A compose row has two behaviours, with or without the mesh, but its title
showed three: no glyph on a role's larger variants, 🛣️ or 🚇 on its smallest.
No glyph and 🛣️ deploy the same thing, so every compose title now reads 🛣️
or 🚇 like a swarm title, and host titles stay without one. Only the smallest
variant still rotates onto the mesh.

A &vpn pin on a row no mode of which can take the mesh now aborts the run
instead of quietly deploying 🛣️, and a regular row pinned to the mesh only
rotates onto a mode that can carry it.
2026-10-16 14:48:36 +02:00
evangelostsak a206abd8d5 docs(ci): every swarm and compose row carries the mesh or direct glyph 2026-10-16 14:48:35 +02:00
evangelostsak 5cd28fd83a refactor(ci): drop the re-export of the removed mesh glyph modes 2026-10-16 03:40:16 +02:00
evangelostsak de804fe304 feat(ci): assign the vpn axis and its glyph to smallest compose variants
A compose row of any other variant keeps its title unchanged.
2026-10-16 03:40:16 +02:00
evangelostsak 1052be4754 feat(ci): compose rows of a role's smallest variant take the vpn axis
The same input and rotation as swarm. Only the smallest variant is meshable
in compose, so the CLI test of svc-net-wireguard runs once per role rather
than once per variant.
2026-10-16 03:40:16 +02:00
evangelostsak 1c5ba164f2 Merge remote-tracking branch 'upstream/main' into feature/svc-net-wireguard
No conflicts across 121 incoming commits. The one file both sides changed is
tasks/stages/01_constructor.yml: upstream parks an applications carrier for
the swarm bootstrap roles after the add_host tasks, and keys the carrier on
the tokens file as well. This branch's mesh bootstrap opens and drops its own
block-scoped carrier before that point, so both keep working as written.

Upstream also brings ruff 0.17.0, distro-derived env keys, a scoped act outer
teardown, the stalwart relay rule, the onion apex redirect and image and
version updates; ruff 0.17 reformats nothing on this branch.
2026-10-15 21:27:11 +02:00
kevinveenbirkenbach 1da2cef95e Merge remote-tracking branch 'origin/main' into feature/network/multi-network-serving
# Conflicts:
#	plugins/lookup/email.py
#	tests/unit/python/roles/web-app-prometheus/test_blackbox_targets.py
2026-10-15 16:45:18 +02:00
kevinveenbirkenbach 26a7a273ff Merge branch 'main' into feature/store-i18n 2026-10-15 04:49:45 +02:00
kevinveenbirkenbachandClaude Opus 5.5 47565781ea style(python): import datetime as dt for ruff 0.17
ruff 0.17.0 adds `datetime` -> `dt` to the default aliases of
flake8-import-conventions, and the project selects ICN, so lint-python
failed with ICN001 in seven files.

The files now import `datetime as dt`. ruff's unsafe fix also renamed
two local names in utils/github/playwright_summary.py into the alias:
the `_to_iso_z` parameter and the parsed value in
`_parse_iso_timestamp`. Both are now called `moment`; with the bare fix,
nine tests of test_playwright_summary.py failed with UnboundLocalError
and AttributeError.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 01:24:31 +02:00
evangelostsak a4048b43f8 fix(ci): rotate swarm rows over both mesh arms within one sweep
The mode rotation picks swarm for a row when (position + sweep) is odd, so
every swarm row of a sweep shares one position parity. wants_vpn rotated on
(position + sweep // 4) % 2, the same digit, so all swarm rows of a default
run landed on one arm: a sweep was entirely meshed or entirely direct and
never compared the two.

Reading the position's second digit (position // 2) splits the swarm rows of
every sweep about evenly between the arms, and sweep // 4 still flips each
row on its own schedule. Compose rows are untouched: they never take the
mesh.
2026-10-14 00:27:17 +02:00
evangelostsak cf9803a8a8 docs(ci): reduce the vpn variant docstrings to summary and Args
The axis is documented with the other axes, not per function.
2026-10-05 12:43:23 +02:00
evangelostsak ff75343aba fix(ci): stop a held mesh axis deleting priority rows
The pin narrowed the cross product after the axis was resolved, so a row
pinned against an enforced or disabled run lost every combination and left
the matrix entirely -- while a regular row with the same pin was simply moved
onto the held side. A priority row now takes the same precedence a regular
row already took: the run's axis first, the pin second.
2026-10-02 00:54:20 +02:00
evangelostsak 89ab678a19 feat(ci): take the run's mesh mode when assigning rows
Also moves the title codec to label.py and re-exports it, which is what keeps
the file under the line cap.
2026-09-26 08:31:11 +02:00
evangelostsak 85e9112409 refactor(ci): split the job-title codec out of the assigner
Reading a title back into its axes is the inverse of building one, and it kept
axes.py over the 500-line cap once the mesh axis landed. The names stay
importable from axes so no consumer moves.
2026-09-26 08:31:01 +02:00
evangelostsak 4be2a381ab feat(ci): let a run hold the mesh axis on one side
auto rotates, enforced meshes every swarm row, disabled meshes none. No
exclusive: every swarm row can take the mesh, so there is nothing to drop.

A run that holds the axis outranks a per-token pin, because an operator who
asked for a whole sweep on one side means it.
2026-09-26 08:30:44 +02:00
evangelostsak 997b2f5091 fix(ci): count the mesh state as a narrowing axis
Without it two tokens differing only in the mesh state compare equal and
collapse into one, and a bare role&vpn reads as narrowing nothing.
2026-09-26 02:11:40 +02:00
evangelostsak 61d5206e3b feat(ci): pin the mesh state from a selection token
role&vpn / role&direct, and the same state read back off a pasted job title.
Own separator rather than more + words: + is the onion axis, and one
separator meaning two things could not be read back unambiguously.
2026-09-25 22:34:09 +02:00
evangelostsak 3128d74ba5 feat(ci): assign the mesh axis to every deploy row
Entries carry vpn, labels carry the glyph, and the artifact slug carries the
state so a meshed and a direct run of one row cannot collide on one artifact
name. Priority rows cover both states; compose never takes the axis.
2026-09-25 22:34:04 +02:00
evangelostsak 6391761c05 feat(ci): add the WireGuard mesh deploy axis
A swarm deploy runs either over the mesh or straight over the underlay, and
both have to keep working: the mesh joins physically separated clusters, so a
single-site swarm has no use for it and must not depend on it.

Quarters on sweep // 4 so it flips in step with neither the mode rotation nor
the onion, and a row walks every mode/tor/vpn combination over eight sweeps.
2026-09-25 22:33:55 +02:00
kevinveenbirkenbachandClaude Opus 5.5 098b0caffc fix(ci): offer multi to bond-less rows whose deploy pulls Tor in
A role without a Tor bond only took clearnet rows, and clearnet rows disable
Tor. svc-ai-litellm (variants 0 and 2) and web-app-seaweedfs (variant 0) still
pull the Tor provider in through their dependencies, so they ran on a node
with Tor about half the time under the tor axis and never under the network
axis.

row_states now gives such a row clearnet and multi. Whether the deploy pulls
the provider in is read from the services closure the discovery row already
carries, which matched the deploy's own include resolution on all 33 rows
without a bond. A row whose deploy contains no Tor provider stays clearnet
only, because NETWORK_MODE=multi without the provider fails the deploy, and a
gate pinned false still means clearnet. The matrix and the selection
validator pass the same flag, so a +multi pin agrees with the matrix.
tor_capable lost its last caller and is gone.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 22:42:55 +02:00
kevinveenbirkenbachandClaude Opus 5.5 6314938133 feat(ci): replace the tor axis with a network axis of node modes
Every deploy row now takes a node network mode, clearnet, tor or multi,
instead of an onion yes or no. The workflow input `network`
(auto|clearnet|tor|multi) replaces `tor`, and INFINITO_NETWORK replaces
INFINITO_TOR.

utils/github/variant/network.py decides which modes a row can take: clearnet
only without a Tor bond or when the variant pins Tor off, no multi for a
single_mode role, no clearnet for the Tor provider, whose rows are therefore no
longer dropped when the rotation lands on clearnet. Priority rows cover every
mode and network combination, regular rows rotate the network on sweep // 2,
and clearnet rows carry disable=tor. Selection tokens pin the axis with
+clearnet, +tor or +multi, and multi rows wear the new glyph.

A row hands its mode to the deploy as the variable `network`, which
provisioning writes into host_vars as NETWORK_MODE. Without it the node keeps
deriving its mode from the deployed providers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 22:40:20 +02:00
kevinveenbirkenbachandClaude Opus 5 c7fb6e60c5 fix(ci): narrow a row's architectures by what its variant deploys
row_architectures intersected the run's pool with the declarations of
every role in the complexity closure. That closure follows every
conditional provider flag through the base graph, so tor -> web-svc-mirror
-> prometheus -> openwebui -> litellm -> lmstudio put the amd64-only
lmstudio into the closure of 93 of 128 roles, including the minimal
variants that never deploy it. A single amd64 pin would have withdrawn
arm64 from nearly the whole matrix.

The matrix now replaces a row's services with the include set that
plan_dev_inventory_matrix, the planner the deploy step itself uses,
resolves for that variant. Against CI run 35674334334 it matches every
data point: lmstudio for zammad#2, moodle#2 and xwiki#2, which pulled it
on arm64, and not for docs#0 or legal#0, which did not. With the pins of
the next commit, 126 of 128 roles keep arm64 rows instead of 34.

Without any architecture declaration the emitted matrix is byte-identical
to before under a fixed discovery seed. The planner adds about 37 s per
matrix build, and a role whose plan errors now stops the matrix instead of
only its own deploy job.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 16:01:09 +02:00
kevinveenbirkenbachandClaude Opus 5 9e8d190f14 fix(ci): narrow a row's architecture by its service closure too
A row drew its architecture from the run's pool narrowed by the role's
own meta/services.yml declaration only. The deploy also brings up every
role in the transitive service closure, so a single-architecture
provider still received rows on hardware it cannot run on.
row_architectures now intersects the pool with the declarations of the
role and of each closure role, which assign() passes from the
complexity row.

No role declares architectures today, so no current row changes. The pin
validator in cli/meta/ci/validate.py still checks the role alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 02:08:55 +02:00
kevinveenbirkenbach eea14a0cfb Merge branch 'main' into feature/store-i18n 2026-09-21 23:35:55 +02:00
kevinveenbirkenbachandClaude Opus 5 5b8465260f feat(ci): deploy only the variants a change reaches
An image bump in a provider redeployed every variant of every consumer,
including the variants that switch that provider off and therefore never
deploy the bumped image at all. The diff-derived whitelist now narrows each
affected role to the variants whose own round closure still reaches a seed,
emitted as `role#0,2` selection tokens. That closure is walked on the same
variant-merged services maps the matrix planner builds its rounds from, so the
narrowing follows a parent's variant choice across every generation of the
chain. It covers every auto-derived whitelist: the weekly update PRs,
Dependabot, and any other diff-driven run.

The narrowing never shrinks the matrix on partial information. A seed role
keeps every variant, because its own files changed. A role reached only
through a run_after edge keeps every variant too, because the round closure
does not follow that edge and a role whose hits are empty is therefore
unnarrowed rather than dropped.

A pinned token naming a role the discovery query does not return now warns
instead of aborting the run. The diff-derived whitelist legitimately names
roles the lifecycle envelope filters out, and once it pins variants such a
token would abort every chunk; role names are still checked against roles/
before the run starts, so a typo is caught one layer earlier.

The round's variant merge moves out of the deploy inventory package into
utils/roles/applications/variants.py. The CI selection has to build the same
maps, and importing the deploy package for them pulls in a module that reads
INFINITO_SRC_DIR at import time, which the runner host does not set.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-21 22:51:03 +02:00
kevinveenbirkenbachandClaude Opus 5 2a7455bd90 feat(ci): spread the deploy matrix over amd64 and arm64
Both architectures were claimed to work and only one was ever exercised. The
deploy matrix now carries an architecture the way it carries mode, tor, distro
and filesystem, so every run proves both.

A job cannot ask for an architecture the way it asks for a distro, because the
architecture is the machine rather than something the job installs on it. The
matrix therefore carries a runner label and `runs-on` reads it, and an
architecture without a label in RUNNERS is a build error rather than a row that
lands on the wrong hardware and still reports green. assert_architecture.sh
checks the runner it actually got, because a label that stops resolving to
arm64 would otherwise go unnoticed forever.

Assignment is rotation, not chance: two architectures against five distros and
fifteen distro/filesystem positions are coprime enough that every combination
comes round. A role narrows the pool through `architectures` in its
meta/services.yml; without one it takes whatever the rotation hands it. An
empty intersection aborts instead of silently removing the role from the run.

Images are built per architecture and pushed under an architecture-suffixed
tag, then merged into the plain tag by manifest_all.sh; pushing the plain tag
from both builds would let the last one win and send half the rows to the wrong
platform. cleanup.sh now protects the `ci-<sha>-<architecture>` children of a
protected tag, because the plain tag is an index over them and deleting a child
leaves it unpullable on that platform.

label.py is split out of axes.py, which the 500-line limit no longer fits.
Writing a job title and reading one back belong together: a retrigger that
recovers an axis wrongly replays a different deploy than the one that failed,
and a drifting artifact name makes the failure reporter download nothing. The
leading `.*` in LABEL_RE is greedy on purpose so it anchors on the last mode
glyph; a reusable-workflow caller path carries a glyph of its own, and
test_selection.py guards the case.

Verified: 313 unit tests and 48 subtests across utils/github/variant,
utils/roles and cli/meta/ci. Locally, a deploy pinned to arm64 reaches ok=1444
and its image builds log aarch64-linux-gnu paths.

Not verified here: actionlint over the seven changed workflows, and a full
`make test`. The matrix doubles the row count, so runtime and runner
availability change for every CI run from here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 07:02:44 +02:00
kevinveenbirkenbachandClaude Opus 5 4193da1cb2 fix(ci): replay the instructions on a row that runs in the guide's mode
The replay is a second deploy on the same runner, and mark() only ever checked the variant number of a row, never its mode. So the marker landed on whichever row of the smallest variant came first, which for 38 of 98 marked rows was a swarm one: that runner would have had to bring a compose stack up beside a swarm stack.

Two places needed it. The row now has to run in the mode guide_deployable names, and only rows in that mode count as deployed when the search looks for the smallest variant - without the second half the search settles on a variant this sweep runs in swarm alone and the marker falls away silently.

Measured over the full matrix: 98 marked rows become 71, swarm drops from 38 to 0, compose rises from 47 to 59. The 27 roles that lose the replay entirely are the ones this sweep deploys in no compose row of an eligible variant; getting them back means forcing such a row into the sweep, which is a change to matrix selection rather than to this marker.

mark() had no tests. It has eight now, including the two failure modes the old code showed under a negative control: a swarm row carrying instructions=compose, and a compose row left unmarked because a swarm-only variant won the search.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 22:21:13 +02:00
kevinveenbirkenbachandClaude Opus 5 7fd495cdfc feat(ci): replay a role's README instructions on its smallest deploy row
The instructions test held a job of its own that picked one random role per
run, so a README's Production block was proven roughly once per 265 roles.
The replay now rides on a deploy row instead: no extra runner, and every role
the sweep touches gets its documented one-command deploy exercised.

The row is the role's smallest variant -- fewest services enabled -- because
that is the shape the README describes, and it is chosen among the variants
the sweep actually deploys, so a row cut as redundant coverage cannot silently
drop the role from the test. A role qualifies only when its README carries a
Production block and the guide's own mode is one the role runs and tests.

The marked entry carries the guide mode in `instructions` and a 📖 in its job
title, between the variant number and a priority ⭐. The guide mode is
independent of the mode the rotation gave the row: a swarm row still replays
in compose. The step runs last in the deploy job, after the swarm teardown,
because the replay deploys the role a second time and needs the runner's
ports.

call-test-instructions.yml, cli.meta.ci.guide_select and the `instructions`
dispatch input go with it. `redundant()` moves next to the marker so the
chunker, the plan table and the marker share one definition.

The plan table trades its 📊 Weight column for 📖 Instructions; nothing read
the weight any more, so it leaves the matrix entries too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 16:57:04 +02:00
kevinveenbirkenbachandClaude Opus 5 66bde6af40 feat(ci): spread distro and filesystem per row
A run-wide draw proved one distribution and said nothing about the other four, while reporting green. Both are matrix-row axes now, assigned by the same deterministic rotation mode and tor already use, so one sweep exercises every distribution and every filesystem.

The two read the row's position like an odometer, the distro as the low digit. Turning both on the position directly covers only n of the n x m pairs whenever the pools are the same length, and no sweep unlocks it.

Both are part of a row's identity: glyphs in the job label, columns in the plan table, shards in the artifact name. Both are selectable through %distro and /filesystem, and a retrigger replays the distro off a failed title. It leaves the filesystem out, because a title states the kind the matrix assigned and a deploy may have fallen back from it.

A row's filesystem is a preference unless a human names it, on the run or in a token. Reading the matrix's own narrowing as a demand would fail rows for every transient condition the applying step is built to tolerate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 14:23:29 +02:00
kevinveenbirkenbachandClaude Opus 5 5137465c6f style: delete every comment the comments-valid policy rejects
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.

Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 03:22:02 +02:00
kevinveenbirkenbachandClaude Opus 5 e846063f5e feat(ci): cut redundant coverage from the deploy matrix
A clone and a row an earlier one already embeds prove nothing the sweep does
not already prove, but both still took a slot. Drop them when the regular line
is chunked, so the budget buys unique coverage only, and mark them in the plan
table with a status of their own instead of letting them read as 'beyond the
budget'.

A priority row is exempt: an operator who pins a clone asked for that deploy,
and dropping it would be the silent withdrawal 6605c7ef9 just removed.

The cut sits in chunks_of, not in entries_of, so the plan still lists every
candidate while no chunk can see one. 55 of 254 rows are cut on this branch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:02:26 +02:00