Commit Graph
830 Commits
Author SHA1 Message Date
evangelostsak 6df3288d04 docs(requirements): scope the logout criterion to what is proven 2026-09-08 21:24:43 +02:00
evangelostsak e117a576fa docs(requirements): check off the full variant matrix and the quality gate 2026-09-08 11:09:33 +02:00
evangelostsak 44dce31596 docs(requirements): check off swarm mode, placement, update order and README coverage 2026-09-08 05:26:41 +02:00
evangelostsak d348244494 docs(requirements): check off PKI, machine identity, persistence, restore and idempotency 2026-09-07 22:13:58 +02:00
evangelostsak 60859ca65f docs(requirements): add 032 OpenBao integration requirement 2026-09-07 00:20:17 +02:00
kevinveenbirkenbachandClaude Opus 5 95020d4fb4 fix(install): complete the ansible git fallback
requirements.git.yml carried 3 of the 7 collections the Galaxy file requests, and none of the transitive ones. When galaxy.ansible.com started returning 502, scripts/install/ansible.sh fell back to the git file, cloned its three collections, then resolved community.general's dependency on community.library_inventory_filtering_v1 against Galaxy again and died — five times over, taking make install and the whole CI workspace matrix with it.

All seven collections plus that dependency now resolve from git. Verified with an install run while galaxy.ansible.com was still 502: exit 0, seven collections installed, zero Galaxy contact.

A lint pins the parity so the fallback cannot rot back out of sync unnoticed; opt out per entry with the galaxy-git-parity nocheck marker.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 19:02:45 +02:00
kevinveenbirkenbachandClaude Opus 5 e4353d9742 refactor(ci): derive the manual cancel override from the event
force_cancel_in_progress existed so a manual sweep could supersede a running orchestrator on any ref. It was a flag every caller had to remember to pass, and only one ever did.

A called workflow inherits github.event_name from its caller, so the orchestrator can see for itself that a human dispatched the run. The input is gone; the group reads the event instead. Automatic triggers keep following the ref alone -- everything but main -- and a manual dispatch supersedes everywhere, main included, because someone typed it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 16:29:11 +02:00
kevinveenbirkenbachandClaude Opus 5 e3d91fd8b2 docs(ci): catalog the cancel entry point
Adds entry-cancel-superseded.yml to the workflow catalog and widens the cancel emoji's legend, which named only PR close and branch delete.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 16:29:11 +02:00
kevinveenbirkenbachandClaude Opus 5 653dc632e4 refactor(ci): key cancellation on the ref, not a variable
CI_CANCEL_IN_PROGRESS gated whether a new run cancels the one before it, and it had to be remembered per repository: unset upstream meant cancel, set to false meant do not, and a fork inherited neither. Cancellation now follows the ref. Every branch but main supersedes its own runs; main never does, because a half-finished pipeline on the default branch is worse than a queued one.

call-orchestrator.yml compares github.ref rather than github.ref_name: a called workflow inherits the caller's ref, which is refs/pull/<n>/merge for a pull_request caller. The manual entry workflow keeps overriding the rule through force_cancel_in_progress.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 16:29:10 +02:00
kevinveenbirkenbach f45df04bf1 test(docker): forbid ARG defaults in every Dockerfile
A defaulted build argument answers a question the caller failed to answer. The
build then succeeds against a base image, version or path nobody chose, and the
mistake surfaces as wrong behaviour at runtime instead of a failed build.

web-app-bridgy-fed shows the cost: its metadata declared python 3.14 while the
Dockerfile default silently built against 3.12. Both look plausible, so nothing
pointed at the drift. It and web-app-postmarks now take the base image from the
same services declaration the rest of the role reads.

The lint walks every tracked Dockerfile, template variants and paths outside
roles/ included, which the existing ARG-scope check does not. A second
assertion fails when the scan finds no Dockerfile at all, so the rule cannot
pass vacuously if the walk ever breaks.

Touching the two Dockerfiles brought their pre-existing narration under the
comment lint; what carried substance moved into the file header.
2026-08-25 00:25:00 +02:00
kevinveenbirkenbachandClaude Opus 5 2ca09f03b3 feat(skills): add i8-quality-high
i8-quality drives `make quality`; the gate that has to be green before a pull
request is `make quality-high`, which appends every lint target and runs about
four times as long. Nothing routed an operator there.

The skill leaves the fix loop to test-fix and pins only what differs: the run
goes to the background with its output teed to /tmp so the operator gets a
tail -f line, nothing else may run beside it because the wall-clock budgets
fail under self-made load, and the verdict comes from the two per-target tables
rather than the pipe exit code, which is unreliable here. docs and autoformat
write to the tree, so an unexpected diff after a pass is a finding.

Suppression is not a fix: a lint hit is resolved by deleting the offending
construct, and any nocheck marker needs the operator's explicit approval.

The cheatsheet row is not decoration - test_cheatsheet_is_skill_index requires
every i8- skill directory to appear in that table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 00:25:00 +02:00
kevinveenbirkenbachandClaude Opus 5 dbdf333e16 docs(workflow-loop): name what the local act loop cannot prove
The workspace suite diverges from a runner in two ways that make whole classes of defect unreproducible locally. INFINITO_DOCKER_VOLUME is a named volume locally and /mnt/docker on GHA (utils/env/handlers/infinito/docker_volume.py); compose down -v removes the former and cannot remove the latter, so nested Docker state is destroyed between tracks locally and carried forward in CI. And act sets ACT=true, so detect_gha_act resolves INFINITO_RUNNING_ON_GITHUB=false where a runner resolves true.

Both facts were already documented in isolation; their consequence was not, and it is the one that matters when someone reports a fix as verified because the local loop went green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 19:52:03 +02:00
kevinveenbirkenbachandClaude Opus 5 48845db979 refactor(profile): split runtime capabilities from runtime identity
One boolean answered five unrelated questions: whether the pull-through cache stack runs, whether image pulls take the GHCR mirror, whether the docker data root may be wiped, plus runtime naming and inventory resolution. The workspace is the environment where those answers diverge — it simulates a developer machine on a real runner — so getting one of them meant misreporting where the suite runs and silently paying for the other four.

is_ci() and runs_on_github() stay facts about the machine. cache_stack_enabled, image_mirror_enabled and docker_root_ephemeral become decisions that default off those facts and can be declared explicitly, so an environment states what it wants. Every default reproduces the previous condition exactly, including the distinction the old mirror and wipe gates drew between a real runner and act.

73ae40585 got the cache stack by flipping the identity flag and lost the docker-root wipe with it; the workspace now declares all three capabilities instead. Pairs with 4c068d1cf, which makes the resulting stray bind sources harmless rather than fatal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 19:52:03 +02:00
kevinveenbirkenbachandClaude Opus 5 e7fadf02fe ci(workflows): name workflows for GitHub's lists
Every scheduled workflow opened with an alarm clock, so the Actions list
read as a column of identical icons that said nothing about the run. The
clock now lives in run-name behind github.event_name == 'schedule', where
it marks the one thing it can mark: a run nobody triggered. The name says
what the run does, in two or three words.

The dispatch menu shows names in one narrow column and truncates them, so
a new lint caps a workflow_dispatch name at 25 characters and points the
longer, explanatory wording at run-name, which carries no such limit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 11:18:54 +02:00
kevinveenbirkenbachandClaude Opus 5 575a5a87ac refactor(layout): sort shipped scripts by language and split the unit suite
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).

This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.

Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:

  * 7 `src:` values built from a role variable holding a bare filename
    (svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
    The variable doubles as the destination name inside the container, so the
    language segment belongs at the call site, not in the variable.
  * `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
    through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
    18 roles ship exactly that and have no template fallback: the play would
    have died on each.
  * `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
    interface now reads `files/javascript/javascript.js`.
  * `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
    path through `path_absolute(... 'files', ...)`.
  * 4 `ansible.builtin.script:` invocations naming a bare command
    (pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
  * `dev-python/files/shell/install.sh` pointed at a sibling role's script.

Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 01:31:09 +02:00
kevinveenbirkenbachandClaude Opus 5 66bde6af40 feat(ci): spread distro and filesystem per row
A run-wide draw proved one distribution and said nothing about the other four, while reporting green. Both are matrix-row axes now, assigned by the same deterministic rotation mode and tor already use, so one sweep exercises every distribution and every filesystem.

The two read the row's position like an odometer, the distro as the low digit. Turning both on the position directly covers only n of the n x m pairs whenever the pools are the same length, and no sweep unlocks it.

Both are part of a row's identity: glyphs in the job label, columns in the plan table, shards in the artifact name. Both are selectable through %distro and /filesystem, and a retrigger replays the distro off a failed title. It leaves the filesystem out, because a title states the kind the matrix assigned and a deploy may have fallen back from it.

A row's filesystem is a preference unless a human names it, on the run or in a token. Reading the matrix's own narrowing as a demand would fail rows for every transient condition the applying step is built to tolerate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 14:23:29 +02:00
kevinveenbirkenbachandClaude Opus 5 acc9e25c0f refactor(volumes): one canonical mount surface, context-wired lookups
The mount pipeline was split across two layers: compose_volumes and
container_volumes lived as filter modules that ansible's filter loader
rejected on every start ("has no attribute 'FilterModule'", two warnings
per run), while the same-named lookups wrapped them thinly. The filter
bodies now live inside their lookups (the tor_socks precedent), the
filter files are gone, and the render tests moved next to the lookup
tests.

As lookups they see the templating context, so the application_id term
every call site passed back to its own role is gone too: both lookups
read application_id from context and fail loudly when it is missing.
107 templates and nfs_prep.yml drop the term; the calls collapse to
lookup('compose_volumes') and lookup('container_volumes', service_name).

The container_volumes call is now canonical - four leading spaces,
service_name in the service position, | indent(4) - enforced by a new
lint (test_container_volumes_call_shape). Normalizing the 85 literal
call sites surfaced that erpnext keyed its meta/volumes.yml mounts by
container name instead of compose service key, the only role to do so;
its meta now uses the service keys. wordpress's column-0 call, which
rendered its first line at top level, is indented like every other.

Template logic that only restated meta/volumes.yml is deleted: mailu fed
compose_volumes an extra_volumes dict whose eight names were looked up
from the very meta file compose_volumes reads itself; zammad's init and
railsserver extra mounts were the volumes.yml entries minus the two
services, which the meta now lists (the when-gated OIDC binds included).
Conditionality that meta cannot express stays in code, but in role-local
lookup plugins instead of template loops: matrix_bridge_mounts renders
the per-bridge registration mounts, nextcloud_extra_volumes the
feature-gated whiteboard/recording volumes (type: volume entries are
declared unconditionally by compose_volumes, so gating must happen
before it).

Parity is proven, not assumed: every mechanically rewritten template is
byte-identical to HEAD after the transformation regexes (95/95), and the
semantic roles render identically through the real functions against
HEAD's meta - zammad across six sso/tls polarities, all seven erpnext
services, mailu's top level. make test-lint passes complete.

Also: suitecrm's compose template pins container_extra_hosts with
host_alias (the SOCKS proxy URL host.docker.internal never resolved in
compose+tor), and the five disabled mautrix bridge addons use the
empty-default config lookup so templating the applications dict stops
emitting 125 errors per batch in every CI job.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 23:28:16 +02:00
kevinveenbirkenbachandClaude Opus 5 596b10c403 test(lint/csp): demand reason for unsafe-* grants
unsafe-inline re-admits the inline execution an XSS needs and unsafe-eval re-admits string-to-code, so a CSP relaxation must be verified against the running app rather than copied from another role. The new lint requires a nocheck marker plus a Reason on the flag's own line, so a marker cannot drift to the neighbouring directive.

The 197 pre-existing grants across 70 roles are checked out with a TODO: they were never verified and still have to be.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 23:28:14 +02:00
kevinveenbirkenbachandClaude Opus 5 5137465c6f style: delete every comment the comments-valid policy rejects
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.

Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 03:22:02 +02:00
kevinveenbirkenbachandClaude Opus 5 4b28f0b271 docs(test-e2e-cli): name the path the framework actually asserts
run_one.yml stats roles/<role>/files/test/test.sh and fails when it is absent, but the runner guide and the nfs-server TODO both said files/test.sh. The TODO was the costlier of the two: following it would have produced a script the asset check never finds. It now also names templates/test.env.j2, without which discover_cli_roles never picks the role up at all and the test stays silently unrun.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:02:34 +02:00
kevinveenbirkenbachandClaude Opus 5 476120c42f test(healthcheck): require a probe for every declared image, from either side
A container without a healthcheck counts as healthy the moment its process starts. That is how a service which came back empty passes a restore drill: the drill asks the host's health service, and the host says yes.

The image side is read from the registry rather than by pulling: the manifest names the config blob, the blob carries the HEALTHCHECK. The answer is three-valued on purpose. A mapping means the image ships a probe, an empty mapping means it definitely ships none, and None means the question could not be answered. Only the empty mapping is evidence; reading None as absent turns one network hiccup into a hundred false findings.

Probes address the GHCR mirror first and fall back upstream only for images the mirror does not carry, and a second test requires mirror plus cache to answer more often than upstream, so the fallback cannot quietly become the rule and spend the anonymous rate limit again.

Detection spans the four places a probe can come from: the services.yml declaration, a literal block in a template, a partial pulled in by include or lookup('template'), and a HEALTHCHECK in a Dockerfile the role builds its own image from. Without the last three the sweep reported nextcloud, every redis sidecar and openldap as gaps they are not.

The opposite direction gets its own lint: a declared healthcheck that no template renders is dead configuration. Fixing it surfaced that the shared resolver silently dropped every service_name call site, because it accepted only quoted literals and a role var resolves to a bare key. And the literal-protocol lint could never be suppressed on a wrapped value, since autoformat moves the marker to the value's last line while the offence is reported on its first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:02:32 +02:00
kevinveenbirkenbachandClaude Opus 5 37dc6b3406 feat(recover): replay a generation's database dumps
A backup generation stores databases as sql dumps, not as a file tree
(--dump-only-sql), and the recover chain only ever restored trees:
`git grep sql HEAD -- cli/administration/recover utils/recovery` was
empty. An operator running `recover full` after a real loss got their
files back, their databases untouched, and an exit code of 0.

Adds the missing leg as a fifth recoverer, planned right after the
volume steps so the dumps land once the volumes are back. Three
properties make it a restore rather than an attempt:

- It refuses while a consumer of that database runs. `--empty`
  pre-cleans the schema in one psql session and replays in the next; a
  booting consumer recreates it in between and the dump's own CREATE
  TABLE fails. That is what turned CI red with `relation
  "ar_internal_metadata" already exists`.
- The engine is read, not guessed: the service key in the consuming
  role's meta/services.yml, the same value that fed the backup seed.
  The two previous call sites each sniffed it from the container image
  and could disagree.
- The dump's own header states which server it came out of, and that is
  compared against the running engine before the pre-clean drops
  anything. Forward across a major version is the upgrade path and
  stays allowed; backward is refused. The version deliberately does not
  travel in databases.csv: that file is written at seed time and would
  state the version deployed back then, and a generation copied to
  another host carries its header along while a csv row stays behind.

Mind the header formats, they are not symmetric - postgres puts
`Dumped from database version` on line 7, while mariadb's line 2 opens
with mariadb-dump's own version and only the tab-separated `Server
version` line names the server. Both are pinned as test fixtures,
captured from postgres:17-alpine and mariadb:11 themselves.

baudolo is untouched: nothing beyond the credentials it already carries
is read from databases.csv, and no upstream format changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:02:31 +02:00
kevinveenbirkenbachandClaude Opus 5 450350de6b docs(ci): describe sweeps, chunks and priority coverage
Replaces the deploy-line sequencing section, which documented the
three-mode split, the serial/parallel decision and mode_fail_fast --
the only place any of that was written down.

Records what a run now does instead: why chunks exist at all (the queue
clock), how the chunk size and count are derived, what priority means
now that a named role is deployed in every combination it can take
rather than sampled, and how the mode and tor rotations are decoupled
for the regular rows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 06:09:05 +02:00
kevinveenbirkenbachandClaude Opus 5 c541e7c22f docs(credentials): name the path the code actually reads
Eight prose references still pointed at the pre-rename root, and one of them
did more than mislead a reader: the addons lint told an author to reach a
secret through lookup('config', application_id, 'credentials.<name>') at the
exact moment it rejected their literal, so the guidance produced the next
defect. The check itself only tests whether the value is templated, so no
enforcement changes here.

variants.md gains the reason rather than a corrected list. A variant entry is
baked into host_vars raw, which makes its top-level key the inventory path, so
a key no meta topic produces is silently discarded. The previous list drifted
precisely because it stated the set without stating why it was the set.

design/variants.md was wrong twice over: the old root, and an argument order in
lookup('config', 'credentials', '<provider>') that could never have resolved.
The erpnext key was checked against svc-db-mariadb/vars/main.yml rather than
prefixed on faith.

Left alone on purpose: test_credentials_roundtrip and the layout lint name
meta/schema.yml and schema/main.yml as history and as the legacy directory the
lint forbids. Both are correct.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 02:02:27 +02:00
kevinveenbirkenbachandClaude Opus 5 f713ba8350 refactor(credentials): address them at secrets.credentials, like every other meta file
meta/schema.yml became meta/secrets.yml, but its content still landed at
applications.<app>.credentials — the one meta file whose payload did not sit
under a root named after the file. Every other topic obeys that rule:
meta/volumes.yml goes to volumes, meta/rbac.yml to rbac. Credentials now go to
secrets.credentials.

The path exists exactly once, in utils/manager/credential_key.py, and
OVERRIDE_SECTION is derived from its two segments rather than spelled beside
them, so the dotted form used by --set and the nested form used by the
inventory cannot drift. Six consumers read it: the loader, the generator, the
CLI override parser, the two emit helpers, and the four lookup validators.

--set now accepts exactly one key shape,
applications.<app>.secrets.credentials.<key>. The three short forms are
rejected instead of guessed at: the same short key names different credentials
on different apps. That also retired primary_app_id, which only existed to
disambiguate them.

Three defects surfaced while wiring this up. _apply_one_role_schema still wrote
to the old root, so an app's own credentials would have moved while its
providers' stayed behind. web-app-pgadmin reads through
lookup('applications', ...).credentials.<key>, an access path no regex over
lookup('config', ...) could see, and would have deployed with an empty postgres
password. And _META_NON_GENERIC_TOPICS still listed 'schema' — a bare topic
name, invisible to any path search — so the generic file-root loop assigned the
raw schema under secrets and the follow-up write landed inside the cached
document, truncating keycloak's four credentials to one for every later
reader.

tests/integration/ansible/lookups/config/test_credentials_roundtrip.py pins both
directions of the seam and the cache invariant that last defect broke.

Existing inventories still carry the pre-rename root; migrating them is
deliberately out of scope here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 16:51:56 +02:00
kevinveenbirkenbachandClaude Opus 5 e96c752549 Merge fix/playwright/env-template-gate into feature/svc-net-tor
75 commits: the socialhome role reimplemented against the project it actually
deploys, native-login specs for funkwhale and shopware, a window.open IdP
handoff and a settings-link fallback in the shared persona helpers, a swarm
converge budget that gives up on a terminal task, and a long tail of role
fixes.

Eleven conflicts. The shared persona helpers (keycloak.js, logout.js) take the
incoming structure — they gained real behaviour this branch never had — with
this branch's resolveTimeout convention re-applied over it and
confirmKeycloakLogoutIfPrompted re-attached at every return point; without it
the Keycloak SSO session outlives the logout. playwright.config.js keeps this
branch's version: both sides set actionTimeout, but ours routes through an env
var and 15s is too short behind an onion. socialhome's spec takes the incoming
content — 2dbd6eb10 found the role described the wrong upstream project
entirely — and keeps gotoOnion and expectHstsWhenTls, since demanding HSTS over
a plaintext onion is a false failure. Both READMEs were regenerated rather than
merged by hand.

Three collisions arrived with no conflict marker at all, because each side
changed a different place in a different file:

- group_vars/all/18_swarm.yml declared swarm_stack_converge_retries twice.
  Same value, so nothing breaks today; editing the documented one would have
  silently changed nothing.
- svc-net-tor and web-app-socialhome both claimed 192.168.74.0/24. Socialhome,
  the newer role, moves to 192.168.75.0/24.
- web-app-socialhome/meta/schema.yml arrived as a new file, so ef5ea7cb5's
  rename never saw it. Its credential would not have resolved and the role
  would have deployed with an unusable administrator account.

The incoming specs also needed this branch's conventions: raw timeouts wrapped,
four page.goto calls routed through gotoOnion, a request.get given an
onion-aware timeout, and 21 roles marked playwright-service-flag over the tor
block this branch added to them.

make quality green on the merge result.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 14:20:19 +02:00
kevinveenbirkenbachandClaude Opus 5 ef5ea7cb55 refactor(roles): name the credential file meta/secrets.yml
meta/schema.yml held credential definitions and their runtime values, but the
name said 'schema' and read like structural metadata next to services.yml and
volumes.yml. Nothing in the file is a schema in the sense the other meta files
use the word, and a contributor had to open it to learn it holds secrets.

62 role files renamed. ROLE_FILE_META_SCHEMA becomes ROLE_FILE_META_SECRETS and
keeps its role as the SPOT the path is built from.

Two readers bypassed that constant and spelled the name themselves:
utils/cache/applications.py built meta_dir / 'schema.yml' twice, and
plugins/lookup/application_gid.py carries the filename in the marker list it
detects application roles by. Left behind, the rename would not have raised —
credentials would have resolved empty and the role would have stopped counting
as an application.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 12:08:06 +02:00
kevinveenbirkenbachandClaude Opus 5 7f81d26340 fix(lint): close the blank line the rerun doc left behind
3a1ef70cb removed the rerun workflow's rows from the actions configuration
page and left two blank lines at the end of the file, which MD012 rejects.
That is what failed the Lint job on 777f7f0fa.

The two Python files carry the reflow ruff 0.16.2 produces; 6320da703
brought that version in with the main merge. Committing it keeps the tree
matching the formatter, since scripts/lint/python.sh runs ruff format in
write mode and would otherwise rewrite them on every run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 11:08:49 +02:00
kevinveenbirkenbach 8435eda8b3 fix(package-cache): use Fedora mirror redirector 2026-08-12 03:42:38 +02:00
kevinveenbirkenbachandClaude Opus 5 3a1ef70cbb chore(ci): drop the cancelled-run rerun automation
The hourly sweep fought the rest of the CI control plane instead of
helping it: it preempted itself (17 of the last 29 runs concluded
cancelled 1-2s after their successor was created, for 8s of work behind
a 74-80min queue wait) and shipped MAX_ATTEMPTS=0, which the script
reads as no ceiling while its own README promised the opposite. The 24h
age cutoff could not bound it either, because every rerun refreshes the
run's updatedAt and so keeps it inside the window.

Removes the workflow, the two scripts it was the sole consumer of, its
unit test, and the CI_RERUN_* documentation that would otherwise point
at a deleted file. The workflow is disabled in both origin and fork so
no sweep fires before this lands.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 20:20:20 +02:00
kevinveenbirkenbachandClaude Opus 5 ac7b98f342 refactor(workspace): split the environment suite into base, compose and swarm tracks
scripts/tests/environment/ becomes scripts/tests/workspace/ with one
subdirectory per track. base/ prepares the workspace and is a prerequisite;
compose/ and swarm/ are independent of each other, so CI can run them as
separate jobs with a budget each. 00_orchestrator.sh runs the tracks named
by INFINITO_WORKSPACE_TRACKS and aborts on an unknown one.

The job count stays at five: cli.meta.ci.dev_runtime matrix assigns exactly
one track per distro and flips the split with the run number, so two
consecutive runs cover both tracks on every distro without doubling the
matrix. The workspace_track input (rotate|compose|swarm) pins it from manual
steer through the orchestrator for a targeted re-run.

Job names carry the track glyph from utils/symbol_glossary and the image
shortened to its last two segments, so quay.io/centos/centos:latest reads
as centos:latest.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 22:57:05 +02:00
kevinveenbirkenbachandClaude Opus 5 4693a81958 fix(lint): repair the two findings the full gate surfaced
Both are fallout from earlier commits in this branch.

4b10d6b93 moved redirect_uris.py from the keycloak role's filter_plugins to
plugins/lookup, but the requirements doc still linked the old path, which
test_relative_links flagged.

test_primitives.py reads back a file that download_file just wrote into the
test's own TemporaryDirectory. test_no_uncached_file_reads asks for
utils.cache.files.read_text, but its own message names this case as the
opt-out -- "tempdirs, synthetic fixtures written inside the same test" -- and
read_text is wrong here anyway: the payload is bytes, and a session-shared
cache would defeat the point of asserting what was just written. Same marker
as tests/integration/meta/workflows/test_subset_roles.py:40.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 10:58:12 +02:00
kevinveenbirkenbach d235dd7d7e Merge branch 'feature/swarm-nfs' into feature/svc-net-tor 2026-08-07 17:28:13 +02:00
kevinveenbirkenbachandClaude Opus 5 c74b10177e docs(ci): name the rerun sweep by its current file
The link targets were updated with the rename, the link texts were not, so four places advertised a workflow file that no longer exists. No lint catches this: the path resolves, only the word beside it is wrong.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 17:19:46 +02:00
kevinveenbirkenbachandClaude Opus 5 71888d86af refactor(ci): prefix workflows by trigger
The workflow directory mixed entry points, reusable stages and scheduled jobs into one flat list sorted by topic, so the first question about any file - what starts it - was the one the name did not answer. Every workflow now carries one of three prefixes, derived from its on: block: cron- for a schedule trigger, call- for one only reachable through workflow_call, entry- for everything a repository event or a human starts. Scheduled workflows additionally carry an alarm clock as the first character of their name.

Two lints keep it that way. One derives the expected prefix from the triggers and compares it against the file name; it flagged all 17 offenders before the rename and none after. The other resolves every .github/workflows/<file> string in the repository against the directory, which is what a rename silently breaks in Markdown links, Python constants and shell API URLs alike.

Renaming pulled 69 pre-existing comments into the comment lint's changed-file scope. The vendored CodeQL template got the nocheck marker its scorecard sibling already carried, two workflow headers moved above name: where they count as file headers, and the remaining banners were deleted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 17:05:51 +02:00
kevinveenbirkenbachandClaude Opus 5 2e3438cc99 feat(ci): rerun the failed jobs of cancelled runs
A run cancelled by a concurrency group loses the work it had already finished. An hourly sweep now walks every branch, and when its latest run ended cancelled it restarts only the failed jobs through gh run rerun --failed, keeping the successful ones.

CI_RERUN_INTERVAL_MINUTES, CI_RERUN_MAX_ATTEMPTS and CI_RERUN_MAX_AGE_HOURS tune the sweep. A cron expression cannot read a repository variable, so the workflow is triggered every full hour and a gate lets only the hours through that open the configured window; the gate reads the hour, not the minute, so a late start still counts for its hour. An unset attempt ceiling means no ceiling, bounded only by the 50 reruns GitHub allows per run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 17:05:51 +02:00
kevinveenbirkenbach 342de43111 Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
# Conflicts:
#	roles/svc-ai-ollama/tasks/01_manager_ops.yml
#	roles/web-app-decidim/templates/compose.yml.j2
#	roles/web-app-joomla/templates/compose.yml.j2
#	roles/web-app-matrix/files/flavor/ansible/runner/bootstrap.service
#	roles/web-app-matrix/templates/flavor/compose/services.yml.j2
#	roles/web-app-nextcloud/files/playwright/_shared.js
#	roles/web-svc-cdn/tasks/01_assets.yml
#	scripts/tests/deploy/swarm/utils/trigger_units.sh
#	scripts/tests/deploy/utils/filesystem/docker_dataroot.sh
2026-08-05 02:32:58 +02:00
kevinveenbirkenbachandClaude Opus 5 f9dea42c68 fix(package-cache): survive a flaky apt mirror
A debian:bookworm workspace job died on `apt-get update`: the mounted
sources list carried the Ubuntu entry too, Nexus had auto-blocked
archive.ubuntu.com, and apt fails the whole update on one bad source.

Three holes, three plugs:

- One sources list per INFINITO_DISTRO under compose/package-cache/apt/,
  selected by the compose overrides. A Debian runner no longer fetches
  an Ubuntu suite it has no use for. The non-apt distros get an empty
  file so the bind mount has a source to point at.
- autoBlock off. While a remote is auto-blocked Nexus answers `404
  Remote Auto Blocked` to every client instead of falling back to the
  copy it already has cached.
- Mirror failover in the frontend. Every apt suite gets a second proxy
  repo on an independent upstream; a 5xx from the primary re-runs the
  request against it. Nexus has no group repository type for apt, so
  the retry lives in nginx.

Measured on a live Nexus 3.91.1: dead remote without a cached copy
answers 502, with a cached copy 200 regardless of metadataMaxAge. The
frontend serves 149 kB of real bookworm `Release` from the mirror repo
when the primary is dead, and the new regex locations produce upstream
request lines byte-identical to the `rewrite … break` form they replace,
including `%3a` epochs in pool paths.

INFINITO_CACHE_PACKAGE_APT_LIST is gone; the mount interpolates
INFINITO_DISTRO directly.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 19:33:04 +02:00
kevinveenbirkenbachandClaude Fable 5 65d2aa028e feat(ci): emoji entry names and a symbol-framed manual run title
The three CI entry workflows drop the "⚡ CI:" prefix for one emoji each -
🕹️ Manual, 🔀 Pull Request, 📤 Push - completing the trigger triad by symbol.
The manual run-name loses its prose prefix and separator dots; the segments now
frame themselves: 🕹️ for the dispatch, 🐧 before the distro sweep, ⭐ for the
priority line, 🎯 for a verbatim whitelist, 🔀 for diff derivation.

  before: Manual CI triggered: debian arch; web-app-x ⭐; diff-derived (origin/main)
  after:  🕹️ 🐧 debian arch ⭐ web-app-x 🔀 diff (origin/main)

Dropping the dots removes the fixed literal after inputs.distros that
utils/github/run_name.py cut the distro list at - the list is itself
space-separated, and the next segment starts with a branch-dependent emoji.
value_from_title therefore derives the terminator dynamically when the framing
literal is whitespace-only: _openings_after collects what any later template
segment can render first (the leading literal of each quoted string in later
expressions, format placeholders stripped, plus later inter-expression
literals) and cuts at the earliest occurrence. The SPOT property holds: the
openings are read from the workflow's own run-name, so a future emoji change
flows into the parser without touching it.

Titles from before this change no longer match the template and yield '' - the
documented unknown-title behaviour, so a retrigger of a pre-rename run falls
back to the workflow's distro default instead of guessing.

The dispatch CLI is unaffected (it targets entry-manual.yml by file name), job
names and the orchestrator are untouched, and no workflow_run trigger
references the old names. Required status checks configured against the old
workflow names in repository settings are the one surface this repo cannot
see - re-check them after the rename lands.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 13:27:34 +02:00
kevinveenbirkenbachandClaude Opus 5 c7ddf6eb24 Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Three conflicts, each resolved by combining both sides rather than picking one.

compose.yml passes both env vars through to the container: swarm-nfs adds
INFINITO_SWARM_MGR_IP, this branch adds INFINITO_TOR_SOCKS_PORT, and neither
substitutes for the other.

daemon.json.j2 takes the data-root block from swarm-nfs and keeps HEAD's dns
key. The incoming side writes networks.internet.dns directly; lookup(
'container_dns') already returns that value as its clearnet entry and prepends
the docker0 bridge on a Tor node in compose mode, so the incoming form is a
strict subset and dropping it loses nothing.

The WordPress CA-trust addon spec gains skipUnlessServiceEnabled("sso") from
swarm-nfs alongside HEAD's gotoOnion and resolveTimeout, and asserts on the
Plugin Name the mu-plugin header declares - the mustuse screen renders that,
not the filename - with resolveTimeout applied to the literal the incoming
side left bare.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 00:47:10 +02:00
kevinveenbirkenbachandClaude Opus 5 673e994aa7 feat(lint): pin type and resolver on every container inspect
An untyped inspect searches every docker object kind, and a role's overlay network carries the same entity name as its main container, so the call can resolve the network and report a missing State key instead of naming the container. That is how the openresty assert died on every swarm job of run 30709070050.

Extend container-exec-resolver to cover inspect, stripping raw markers and quoted spans first so the flag walker survives Go template format strings. Add container-inspect-type, scoped to roles/ because the collision follows the entity naming convention and the outer act namespace carries no role networks. Sweep 40 call sites; suppress the delegate_to-resolved CIDs and the compose co-location probes with a stated reason.

Move the five container rules into tests/lint/ansible/shell/container/ and drop the container_ prefix from their filenames. Register both markers in the suppression docs, where the resolver marker was missing entirely.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 20:51:56 +02:00
kevinveenbirkenbachandClaude Opus 5 0bed81c048 Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Twelve conflicts, resolved by intent rather than by side.

Six shared one shape: the incoming side adds the new rescue_index.sh call but
drops this branch's `# nocheck: shell-or-true` markers. The markers carry a
lint contract here, so the incoming structure was taken and the markers put
back — dedicated.sh, guide/one.sh, guide/host_deploy.sh, swarm/routine/00_one.sh
and trigger_units.sh, where the incoming version additionally writes the full
journal to a file instead of only echoing a tail.

web-app-matrix keeps HEAD's _synapse_extra_args accumulator (the superset: CA
mount plus the onion case) but adopts the incoming improvement inside it — the
inlined regex_replace gives way to lookup('subnet_gateway').

handlers/__init__.py drops rescue_diagnostics_dir from the old import group
(the incoming side moved it under .infinito.rescue, keeping both would import
it twice) and keeps tor_socks_port, a HEAD-only handler ORDERED_HANDLERS still
needs.

svc-bkp-volume-2-local takes the incoming side throughout: both branches landed
the same feature, but swarm-nfs evolved it — the knob moved from the global
STORAGE_BACKUP_SNAPSHOT_MODE to services.volume-2-local.snapshot_mode, plus the
subvolume helper, 04_snapshot_subject.yml, meta/packages.yml and baudolo 3.2.2.
Three files that left behind were removed: files/baudolo_snapshot.py and the two
worktree tests at their pre-refactor paths, all byte-identical to their moved
counterparts.

Six follow-on failures the merge produced, fixed at the root:

- STORAGE_BACKUP_SNAPSHOT_MODE lost its last reader with the config-lookup
  design and is gone from group_vars.
- utils/github/variant_bundle_limits.py sat next to variant_bundles.py, which
  the prefix lint forbids; the pair now lives in utils/github/variant/ as
  bundles.py and bundle_limits.py, with the -m entry point, the ruff
  per-file-ignore, the design-doc link and the mirroring test path updated.
- The incoming WordPress addon spec navigated with a raw page.goto and carried
  three bare timeout literals; both route through gotoOnion and resolveTimeout
  now, which is what makes navigation survive onion transport.
- Six unmarked `|| true` got a concrete reason each: docker.socket is absent on
  distros that ship docker without it, the loop device is already detached when
  the probe failed before attaching, and modprobe zfs may fail because the
  /dev/zfs check decides.

make quality green across all four targets. The cold-domains budget test that
had to be waived here passes on its own now — 19d56f07e raises it to 100s.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 03:00:02 +02:00
kevinveenbirkenbachandClaude Opus 5 6c816e893e refactor(github): group the variant modules and drop the blind-except
variant_bundle_limits.py next to variant_bundles.py formed the prefix group
'variant', which the structure lint reads as a folder in disguise. It stayed
green locally because that lint asks git ls-files: the new file was untracked
through every quality run and only became visible once committed. CI saw it
immediately - the sole failure of run 30675284663.

Both move to utils/github/variant/ as bundles.py and limits.py, and the test
mirrors along.

The BLE001 per-file ignore is removed rather than repointed. parse_size raises
humanfriendly.InvalidSize for a malformed string and TypeError only for a
non-string, and the value reaching it is always str - so the catch-all was
never needed. Measured, not assumed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 02:46:36 +02:00
kevinveenbirkenbachandClaude Opus 5 e595ddc194 docs(skills): add i8-merge-sibling for pulling and merging a sibling branch
Five requests in one session ran the same chain by hand: fetch, merge, resolve,
gate, wait for the commit decision. The conflict doctrine that emerged from
those merges is now written down instead of re-derived each time — union
additive lists, prefer the SPOT variable over an inlined duplicate, keep the
conventions this branch's lint enforces, and take the side that fails loudly.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 18:14:28 +02:00
kevinveenbirkenbach e04c448cdd Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Three conflicts, none resolved by picking a side wholesale:

- Makefile: took the incoming ACT_INPUTS, which routes through SWARM_DISTROS
  instead of inlining the same expression a second time.
- test-e2e-playwright/02_run_one.yml: took the incoming rc handling. An absent
  rc now counts as failure; the previous default(0) let an aborted Playwright
  run pass silently, contradicting the debug message right above it.
- web-app-peertube spec: combined both. Kept gotoOnion and resolveTimeout,
  which this branch's lint enforces, and adopted the incoming retry-inside-poll
  for the OIDC button plus its networkidle wait.
2026-07-30 15:38:15 +02:00
kevinveenbirkenbachandClaude Opus 5 f3963dcbdd test(storage): gate min_storage against the image size it is declared for
An over-declared min_storage makes a host look too small: it feeds the storage guard, the runner packer and the sufficient-storage filter, so a deploy that would have fit prunes a cache it did not need to lose. Nothing checked the declared number against reality.

The new external test measures what a pull of the service's own image:version actually transfers - config blob plus every layer blob, resolved for linux/amd64 - and fails when min_storage exceeds it by more than 1GB. A service whose data volume genuinely dominates keeps its value by carrying '# nocheck: min-storage-headroom' together with a 'Reason:', the same two-marker shape meta/volumes.yml already uses for nfs: false.

Sizes come from the repo's own registry client rather than 'docker buildx imagetools': make test-external runs inside the compose container, which has no docker socket. manifest_transfer_size reuses the existing bearer-challenge flow, keeps the tri-state contract of manifest_exists (an indeterminate registry warns instead of failing), and resolves a multi-platform index to its amd64 manifest - the case a raw jq over '.layers' would miss entirely. Distinct image refs are measured concurrently through resolve_max_fetch_workers().

A min_storage that parses out of the YAML but cannot be located in the file text fails the test, so a declaration can never silently lose its marker resolution.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 15:08:51 +02:00
kevinveenbirkenbach 5dbbaca4a1 Merge branch 'feature/swarm-nfs' of github.com:kevinveenbirkenbach/infinito-nexus-core into feature/svc-net-tor 2026-07-29 23:23:56 +02:00
kevinveenbirkenbachandClaude Opus 5 bc98f16c07 docs(ci): realign the workflow table with the actual inputs
Three rows disagreed with their workflow files: ci-orchestrator still advertised a distros default that 0f7334b3d removed, test-deploy-swarm omitted distros entirely although it is its only required input, and test-instructions listed one input with a svc-db-postgres default where the workflow declares four on both triggers and picks a random role when role is empty. test-dns is left alone -- its 'required on call; default debian on manual' already describes the real split.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 23:23:54 +02:00
kevinveenbirkenbachandClaude Opus 5 f6c5f38760 docs(ci): realign the workflow table with the actual inputs
Three rows disagreed with their workflow files: ci-orchestrator still advertised a distros default that 0f7334b3d removed, test-deploy-swarm omitted distros entirely although it is its only required input, and test-instructions listed one input with a svc-db-postgres default where the workflow declares four on both triggers and picks a random role when role is empty. test-dns is left alone -- its 'required on call; default debian on manual' already describes the real split.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 22:47:31 +02:00
kevinveenbirkenbach 3f84626883 Merge branch 'feature/swarm-nfs' of github.com:kevinveenbirkenbach/infinito-nexus-core into feature/svc-net-tor 2026-07-29 16:30:43 +02:00