Commit Graph
380 Commits
Author SHA1 Message Date
kevinveenbirkenbach feb5630f19 Merge remote-tracking branch 'origin/main' into feature/network/multi-network-serving 2026-10-20 17:14:44 +02:00
kevinveenbirkenbachandClaude Opus 5.5 b055eece05 feat(ci): keep GitHub-hosted runner labels current through the updater
The workflows pin `runs-on: ubuntu-NN.NN`, so no job moves to a new Ubuntu
release behind our back, but nothing raised the pin either. A sixth job of
`🔄 Update versions` closes that: `utils/update/runner.py` reads the image
table of actions/runner-images, takes the newest x64 Ubuntu label whose
image carries no preview or beta badge, and raises every older bare
`runs-on: ubuntu-NN.NN` to it; arch-suffixed labels stay untouched. The job
opens the change as one PR over `.github/workflows`, so its CI run tests
the new image before it reaches main.

This commit carries the updater's first result: every job now runs on
ubuntu-26.04 (49 `runs-on`).

actionlint 1.7.12, the newest release, does not know `ubuntu-26.04` yet
(rhysd/actionlint#682, fix proposed in rhysd/actionlint#743), so
`.github/actionlint.yaml` declares the label until a release knows it.

act maps each `ubuntu-NN.NN` label the workflows use to the local platform
image, derived from `.github/workflows`, instead of a fixed list that did
not know ubuntu-26.04.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-19 16:56:52 +02:00
kevinveenbirkenbach 25d8982774 Merge branch 'main' into feature/store-i18n 2026-10-19 16:53:07 +02:00
kevinveenbirkenbachandClaude Opus 5 ae06cb4b9b fix(i18n): take a catalog's language from its path, not its header
reason() calls itself the single source of truth for both ends of the
pipeline, and the write path honoured that: client.py:228 passes the
directory code into unmask(). Only damaged() read catalog.locale, so
for 50 catalogs the two ends judged against different languages.

44 catalogs carry no locale header, 8 of them non-Latin and therefore
silently past the script check. 6 more name a code their directory
disagrees with, so tl was judged as fil, bh as bho, tw as ak.

The language becomes a required parameter; both production callers
already held the code. Measured over every catalog, header language
against path language, one moves: core/av from 0 to 29. Those 29 hold
pseudo-Norwegian and a degeneration loop in a Cyrillic catalog, and
make i18n-prune has since cleared them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-19 12:56:40 +02:00
kevinveenbirkenbachandClaude Opus 5 5647a2d2bd style: take the line breaks autoformat wrote
cfe20f0726 and 4915a401d4 wrapped three statements more cautiously than
the formatter wants. Without this the lint phase rewrites the same diff
into the tree on every run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-19 03:41:36 +02:00
kevinveenbirkenbachandClaude Opus 5 cfe20f0726 fix(swarm-test): one watcher, and a wedge it can actually tell apart
The main merge left two step watchers: run.run_watched from main and the
newer watchdog.run_watched from here. Their disk and memory floors are
byte-identical (6144M, 6% of RAM, abort 75), so run.py now keeps only
run_step and takes the watcher from watchdog. mesh.py reaches it through
run_step and matrix.py already imported it directly.

That alone would have been a regression. The watchdog aborts on io60
alone, which its own docstring says cannot separate a wedged mount from
a busy disk, and the six mesh steps had no I/O kill before: a healthy
sleep on a host with 126G free and no task in D state died at rc=75
after 10s where the old watcher let it finish. The first sample also
lands 10s in, so 50 of the 60 seconds it averages belong to whatever ran
before -- an image pull could abort its successor before that step read
a byte.

The ceiling now takes three things together: a task in D state on the
same sample, WEDGE_SAMPLES breaches in a row, and a step old enough to
own a whole STALL_WINDOW_S of the average. Both refuted scenarios are
regression tests. A real wedge is caught about a minute later than
before; the disk floor that covers ENOSPC is an instantaneous read and
is untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-19 03:21:57 +02:00
kevinveenbirkenbachandClaude Opus 5 0c695a80d5 refactor(variant): split pin validation out of axis assignment
Our side added the architecture axis and main the vpn axis; combining
both put axes.py at 502 lines against the 500 the structure lint allows.

check_pins and row_architectures answer one question: may this row take
what the selection token demands of it. Assigning the axes of a row that
carries no pin is the other job, so the two separate cleanly and axes.py
drops to 397 lines.

axes.__all__ still exports both names, so cli/meta/ci/validate.py keeps
reaching them through axes. The test patches get_role_architectures,
which moved with them, so its patch target moves too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-19 02:32:32 +02:00
kevinveenbirkenbach 15930945fe Merge branch 'main' into feature/store-i18n 2026-10-19 01:54:37 +02:00
kevinveenbirkenbachandClaude Opus 5.5 7f28564548 Merge branch 'main' into feature/network/multi-network-serving
Combine main's WireGuard mesh axis with this branch's network axis in the CI
variant code. Every matrix row now carries both: network (clearnet, tor,
multi) from utils.github.variant.network and vpn (meshed or direct) from
utils.github.variant.vpn. Label parsing, which main moved to label.py, reads
the network glyph instead of the old tor flag and keeps the vpn glyph; the
selection token, plan table, matrix and trigger CLIs, the workflows and the
swarm routine forward both axes. The test of an incapable priority variant
now compares the set of network modes, since a swarm priority row also
covers both mesh arms.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-19 00:15:05 +02:00
kevinveenbirkenbachandClaude Opus 5.5 d2b33a3b2a fix(compose): keep dynamic addresses off the stack's static IPs
compose/cache.shared.override.yml attaches every worktree stack's
infinito service to the primary's network (infinito-nexus-core_default,
172.30.0.0/24) without an ipv4_address. The default network declared
no ip_range, so Docker IPAM handed out the lowest free address and the
worktree stacks took 172.30.0.2, .3, .8, .9 and, whenever the primary
container was down, .10, the primary's INFINITO_IP4. The primary's
`make compose-up` then failed with "Address already in use", and with
it `make quality-high` on main.

The default network now declares ip_range: ${INFINITO_IP_RANGE}, the
upper half of INFINITO_SUBNET. Every static address of the stack (.1,
.4 to .6, .10, .53) sits in the lower half, so a container without a
static address can no longer take one.

INFINITO_IP_RANGE is derived from the final INFINITO_SUBNET after
custom.env is applied, because a worktree pins its subnet there and
`make dotenv-force` regenerates without it in the environment. Existing
worktrees therefore get their own slot's range without a new pin.

A lint checks that every static IP in default.env and every
172.30.${slot}.N pin in scripts/system/worktree/up.sh lies below the
pool.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-18 21:44:16 +02:00
kevinveenbirkenbach da449f8894 Merge pull request #323 from evangelostsak/feature/svc-net-wireguard
feat(svc-net-wireguard): Dockerized WireGuard role + test-e2e-cli DinD e2e
2026-10-18 19:50:42 +02:00
kevinveenbirkenbach af24e1f920 Merge branch 'main' into feature/network/multi-network-serving 2026-10-18 15:28:01 +02:00
kevinveenbirkenbach 415542c754 Merge branch 'main' into feature/store-i18n 2026-10-18 14:00:06 +02:00
kevinveenbirkenbach 5601592872 Merge pull request #989 from infinito-nexus/feature/fuzz/yaml-bootstrap-differential
feat(fuzz): fuzz the bootstrap YAML reader against PyYAML weekly
2026-10-18 13:29:43 +02:00
kevinveenbirkenbach 4a469e6476 Merge branch 'main' into feature/store-i18n 2026-10-18 13:20:06 +02:00
kevinveenbirkenbachandClaude Opus 5 9ecedfac92 test(lint): read the files these tests write without the session cache
Both new tests read a file their own run has just written, which the uncached-read lint flags because utils.cache.files.read_text is an lru_cache with no mtime invalidation. The two cases want different answers. The rendered inventory is YAML, so load_yaml_any reads and parses it the way the lint asks and needs no exception at all. The healthcheck probe writes its request through bash inside the test, so a cached read would compare against the previous run's bytes and the assertion would pass without proving anything; that one keeps a cache-read directive naming why.

make test-lint went from 2 failed to green, and the first attempt is the reason the inventory test carries no directive: suppressing the read there only moved the complaint to yaml.safe_load, which the right reader answers in one call.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-18 10:57:20 +02:00
kevinveenbirkenbachandClaude Opus 5 ee9cbc8d5b feat(tools): pin the lane's judge model in its own inventory
The oracle asks a chat model whether a translation states its source. llama3.2:1b cannot answer that: on the suite's own fixture it scored an unrelated text at the middle level instead of the bottom one, and asked as a yes-or-no question it called two identical texts different. With it the suite reached no verdict at all, reporting that the judge read none of the 47 languages the catalogs hold machine translations for. Qwen2.5-3B answers both correctly, and the same run then judged 248 samples.

The pin lives in the lane inventory rather than in AI_LOCAL_MODELS, because a judge that can separate a faithful rendering from an unrelated text is a demand no deployment makes, and the default should not grow a gigabyte to satisfy a check. The litellm config builds its routes from the merged application config, so a lane override reaches the model list and its window without touching the repository default.

LANE_MODEL_ALIAS repeats the one name in utils/inventory/tools.py, because ensure() asks whether the lane already serves before the deploy that writes the rendered inventory, so the probe cannot read it from there. A lane serving one model while the probe expects another never reports ready and redeploys on every call, so a test renders the template and holds the two names equal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-18 10:30:22 +02:00
kevinveenbirkenbachandClaude Opus 5 2f461b75bf fix(healthcheck): end the probe request at the header terminator
The generated tcp and http_status probes built their request with echo -e, and echo appends a newline, so the bytes on the wire ended with a stray bare LF after the header terminator of a Connection: close request. uvicorn's h11 reader treats that as trailing data and answers 400 before the handler's 200 reaches the socket, so the probe's grep for a 2xx or 3xx status line never matched and the container stayed unhealthy for good.

In compose the name resolves straight to the container, so an unhealthy but running service still served and only the backup drill noticed, failing with litellm is running/unhealthy after 1200s. In swarm a task joins the service VIP only once it is healthy, so the litellm service sat at 0/1 with no backend and every consumer of http://litellm:4000 got ECONNREFUSED, which is where the openwebui model publishing died.

The tcp flavor had been masking the same malformed request all along, because its grep for HTTP/1 matches 400 Bad Request as happily as 200.

printf is a bash builtin like echo and the probe already requires bash for /dev/tcp, so nothing new is demanded of the image. The new test runs the generated request through bash and compares the bytes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-18 09:51:34 +02:00
kevinveenbirkenbachandClaude Opus 5 12ed8a763c fix(i18n): translate through the gateway and let a backend declare what it serves
make i18n-translate addressed the engine directly, so a run could never reach more than the 47 languages Argos ships models for, while meta/languages.yml names 183. service_url now resolves web-svc-translate, which routes a pair the engine does not serve to a chat model, and the deploy brings both roles up.

The gateway advertised only what its backends declared, and the chat backend declared nothing, so /languages reported 51 targets of 183 and the readiness wait spun to its hour-long timeout without translating a line. The chat backend now declares every language the table names.

Routing offered libretranslate for every pair, because an undeclared catalogue read as any pair, so a language it holds no model for was answered with HTTP 400 instead of falling through to the next backend. serves() now reads the backend's own /languages, which catalogue() already exposed for the advertisement. Declaring the pairs in the role was the first attempt and carried a second copy that was already stale: the engine reports zh-Hans where the role's list says zh.

A small model copies the prompt markers back, echoes the source line and labels its answer with the target language. All three reached the catalog, and an entry carrying its own source scores below the oracle's middle level. clean() strips them, and only from an answer that carried scaffolding, because on its own a short word and a colon is a translation, not a label.

server() had lost its redeploy parameter while cli/build/i18n kept passing it, so every run raised TypeError before translating a line and the only symptom was an empty log.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-18 09:47:53 +02:00
kevinveenbirkenbachandClaude Opus 5 52d356544b refactor(i18n): let the table name languages and the gateway pick the engine
meta/languages.yml carried `libretranslate: true|false` per entry, which
answered two different questions with one field: which languages the
repository translates, and which engine renders them. The first is the
table's business; the second is not. The gateway offers every backend
that serves a pair, falls through when one refuses or mangles the
answer, drops an engine whose failures pile up, and settles a request
class on the engine that wins its comparisons - and `signature()` is
"{source}:{target}:{length bucket}:{markup|plain}", so that learning is
already per target language. A static column beside it is a second
opinion that goes stale the moment the router learns something.

So the field is gone, 184 lines, and the table now holds name, endonym
and direction. Every language it names carries a catalog in every
domain.

What one engine can do is declared with that engine instead:
services.libretranslate-engine.serves, 47 codes. Not a detour - both
readers run before any engine answers. The lane prefetches one model per
code, and the readiness probe is itself the thing that would have asked.

`translatable()` split along that seam. Catalog-scoped callers take
`domain_languages()`, engine-scoped ones take the new
`served_languages()`: the CI matrix, extraction and translation the
first, the model prefetch and the tuning sweep the second. Collapsing
the two is what this change is guarding against - `baked_languages()`
renders identically before and after, verified by diffing both outputs.

`translate()` also stops dying on a language whose catalog has not been
extracted yet. It reads a missing catalog as nothing pending rather than
raising, because 136 docs catalogs do not exist until the next
extraction and refusing those languages would quietly shrink the set the
repository promises to translate.

The oracle suite follows: tests/oracle/test_translation_score.py asks
utils/i18n/judge.py about every language instead of routing a laya
checkpoint per language. It cannot run yet - the tools lane carries no
LiteLLM - and its skip message says exactly that.

test_catalogs_complete_on_gpu now measures against the table rather than
one engine's coverage. It is live on a GPU host and will report the 136
missing catalogs as the gap they are, which is the point: narrowing it
to the engine would hide exactly the languages worth watching.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-18 01:26:35 +02:00
kevinveenbirkenbachandClaude Opus 5 2261bbabb0 feat(i18n): ask a chat model how faithfully a translation states its source
The oracle suite needs a verdict in every language meta/languages.yml
names, and laya cannot give one. Measured against the suite's fixture,
its english checkpoint separates a faithful rendering from an unrelated
text in 17 of 47 languages and its multilingual checkpoint in none,
answering around 1.5 either way. One judge for all of them beats a route
per language that is noise in most of them.

The question keeps the shape the checkpoint answered: the levels are
written out and the model names an index over them, never a bare
probability. A probability carries this deployment's calibration, and a
fixed threshold on it would encode that calibration into the suite;
"below the middle level" instead means what the middle level says.

Three refusals are deliberate. temperature 0, so the same comparison
does not come out differently on a rerun. An answer naming no index
raises instead of scoring zero, which would read a refusal as "states
something else". An index outside the scale raises instead of being
clamped, because a model answering 7 has not placed the pair on this
scale at all.

The request goes to the model gateway rather than to a model server, so
it carries a consumer identity and resolves through the gateway's model
list like every other model call here.

Nothing imports this yet: the oracle suite moves to it once the tools
lane carries LiteLLM and a loaded model, which is what lets the judge be
exercised against a real one for the first time.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-17 23:53:25 +02:00
kevinveenbirkenbachandClaude Opus 5.5 088c8be14f fix(ci): pull workspace images through the GHCR mirror like the deploy jobs
The workspace jobs pulled every image anonymously from Docker Hub and
two of them failed on its rate limit in run 39059128678 (#990): the
fedora swarm drill on busybox:1.38 on all three nodes, the centos
compose drill while building tor-smtp_custom. The regular deploy jobs
pull through the GHCR mirror, but the mirror was tied to being a real
GitHub runner: Profile.image_mirror_enabled() returned
runs_on_github(), and the swarm routine generated mirrors.yml only when
cli.meta.runtime reported github, which is act inside make
swarm-zombie.

The mirror is now its own capability, INFINITO_GHCR_MIRROR (next to
INFINITO_GHCR_MIRROR_PREFIX), written as a run fact like
INFINITO_CACHE_STACK: true on a real GitHub runner,
false everywhere else, and a non-empty exported value wins for the run.
Profile.image_mirror_enabled() reads the declaration and keeps
runs_on_github() as the fallback, and 02_provision_inventory.sh gates
on the same value instead of the runtime name. make swarm-zombie
forwards it into act through ACT_ENV, since act ignores the host env.

call-test-workspace.yml declares INFINITO_GHCR_MIRROR=true and passes
INFINITO_GHCR_MIRROR_PREFIX into the workspace container. Nothing else
of the workspace changes: is_ci, runs_on_github, the cache stack and
the docker root stay local-like. The step that loads the rescue paths
now removes the .env it generated on the runner, because on debian
rows load.sh sourced it inside the container and the run inherited
INFINITO_RUNNING_ON_GITHUB=true.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-17 22:40:13 +02:00
evangelostsak 0c9ad653f7 Merge remote-tracking branch 'upstream/main' into feature/svc-net-wireguard 2026-10-17 22:19:57 +02:00
kevinveenbirkenbach 1d7bc1b385 Merge branch 'main' into feature/network/multi-network-serving 2026-10-17 21:56:04 +02:00
kevinveenbirkenbachandClaude Opus 5 676a91773e fix(ci): sample I/O pressure, and give swarm a post-mortem at all
Two jobs of run 39040406916 died with the runner's own annotation, 'The
hosted runner lost communication with the server'. Both left every
diagnostic step pending, so no artifact exists for either. For the swarm
one the in-band sampler ruled starvation out — 9324M of 15947M available,
stall60 0.0%, 25 seconds before the cut — and nothing in the repository
can tell the remaining causes apart.

A stuck NFS mount wedges tasks in uninterruptible D state, which shows as
I/O pressure while memory pressure stays flat, and this drill runs its own
NFS node. /proc/pressure/io and /proc/pressure/cpu were read nowhere in
the repository. They are now sampled beside memory, with the D-state count,
every ten seconds instead of thirty, and an I/O stall at or above half the
minute aborts the step while the runner is still alive to upload the
banner.

cpu reads 'some', not 'full': the full line is pinned at 0.00 system-wide,
measured 13.50 against 0.00 on this host, so reading it like memory would
have shipped a metric that can never move.

Dump runner state was switched off for swarm, so the one post-mortem that
survives a step failure never ran there. It runs now, and the swarm drill
gets the step timeout its compose sibling has had all along — 350, the
number test_distro_budget_fits_timeout already derives the distro budget
from.

The sampler moved to its own module: the additions took matrix.py past the
file-size cap, and the lint asks for a split rather than a marker.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-17 21:31:12 +02:00
kevinveenbirkenbachandClaude Opus 5 685c6b50ec test(images): require a mount for every VOLUME an image declares
An unmounted VOLUME does not fail: docker creates an anonymous volume for
it, the container works, and the gap shows up later as a volume nobody can
attribute. sys-ctl-hlth-volumes fails the deploy over exactly that, and its
whitelist keys on volume ids, which are new on every deploy, so the gap
cannot be suppressed there either. e0b973f9bb fixed one such case in
web-svc-xmpp by hand; this is the check that would have found it.

Attribution is the whole difficulty. meta/services.yml declares an image
under a key, and the compose file often runs it under another name:
web-app-jitsi declares the image under 'jitsi' and renders the service as
'web', web-app-erpnext runs one image as seven services, and the two RDBMS
consumer keys name no service of their own because the container comes from
sys-svc-rdbms. The sweep therefore unions the entry key with its
proxy.service_key and delegates the RDBMS keys to their provider role. A
first version keyed on the entry alone and reported six roles that mount
every path correctly, web-svc-xmpp among them.

Reading a config blob section is now its own module beside the registry
client, because the two readers were the same walk twice and registry.py
sat three lines under the file-size cap.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-17 21:17:25 +02:00
kevinveenbirkenbach f79ddac220 Merge branch 'main' into feature/store-i18n 2026-10-17 20:51:29 +02:00
kevinveenbirkenbach 41e3146197 Merge remote-tracking branch 'origin/main' into feat/fuzz/yaml-bootstrap-differential 2026-10-17 20:41:07 +02:00
kevinveenbirkenbachandClaude Opus 5 2f9a53f073 fix(tools): reach the lane from the stack, and judge every role it serves
Three faults, one mechanism each, that together let the oracle suite report
a verdict it never reached.

The address. lane_url handed out the runner's compose service name, but the
stack's containers resolve through the deployment's CoreDNS, which never
learns one: getent hosts tools answers nothing inside them, so every check
that asked the lane from there skipped itself. The bring-up now writes the
runner's address, picked as the one inside INFINITO_SUBNET because the
runner sits on every network it was ever brought up on, and the checks read
it from there. It is rewritten on both paths, because a restart gives the
runner a new one.

The readiness. serving judged the engine alone, so a lane whose engine make
i18n had already brought up counted as up while System One was down. It now
probes every role the caller names, which is why the runner republishes
System One's port: a container IP is unreachable from the host that decides
whether to deploy.

The verdict. A silent lane made the suite skip, and a skip exits green.
Only a deploy that is already running leaves the stack alone and exits 0
now; a crashed deploy and a deployed-but-silent role both exit 1.

The override's third :? guard is added to the gate that decides whether the
file is layered in at all, next to the one that was already missing. Without
it an .env written before these keys aborts every compose command, down to
compose-down, instead of dropping the lane. A test now keeps that list and
the override's guards in step.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-17 20:27:54 +02:00
kevinveenbirkenbachandClaude Opus 5.5 0cb8bae88d perf(cache): let nested lookups reuse the unrendered applications view
Every cold render of the applications SPOT walks the templates of all
269 roles, and those templates call lookup('config'), lookup('tls'),
lookup('domain') and others. Each of these re-enters
get_merged_applications while the render guard is set, and each one
rebuilt the unrendered view from scratch: a deepcopy of all role
defaults plus the override merge, about 8 ms, never cached. A local
cold render of the development inventory made 351 such calls, 2.82 s
of a 6.86 s render.

The outer render now keeps a copy of its unrendered view and its cache
key on the render guard, taken before the inventory volumes are
popped, so it is the same view a nested call used to build. A nested
call with the same key returns it; any other key still builds its own.
The guard drops it when the render ends.

Measured on the same inventory: the nested calls took 0.07 s instead
of 2.82 s and the cold render 3.60 s instead of 6.86 s. The rendered
SPOT is unchanged: the old and new dumps differ in 69 user passwords,
which the development inventory regenerates on every read (two runs of
the old code differ in 68), and in one absolute checkout path, the only
other difference.
In CI each cold render took 40 to 77 s and the heaviest swarm cell of
#950 missed its budget by a few minutes (#984).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-17 19:52:34 +02:00
kevinveenbirkenbachandClaude Opus 5.5 32740d860f fix(utils): stop the bootstrap YAML reader from accepting what PyYAML rejects
utils/yaml_bootstrap.py promises that load_block(text) either equals
yaml.safe_load(text) or raises BootstrapYamlError. A fuzzer comparing
both found two inputs that broke the promise:

- str.splitlines() also breaks lines at \x0b, \x0c and \x1c to \x1e,
  which PyYAML's reader rejects as non-printable. "\x1e/: R" came back
  as {"/": "R"}, and a non-printable character inside a comment line was
  skipped with the comment.
- An indented " ---" on line 1 was taken as the document marker, while
  PyYAML reads it as a plain scalar and rejects the mapping that follows.

load_block now refuses any character outside PyYAML's own printable
class before it splits lines, and takes "---" as the document marker
only at column 0. The line breaks splitlines() still honours (\r, \r\n,
NEL, LS, PS) are line breaks for PyYAML too, so splitting stays as is.
The eight inputs join STRUCTURES in the divergence test; against the
previous reader, four of them diverge and fail it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-17 18:46:16 +02:00
kevinveenbirkenbachandClaude Opus 5 b033aab42b fix(backup): carry extended attributes through the restore
A recover mirrored a snapshot with rsync -a, which transfers no extended attributes, so a restored volume came back byte-identical while every application that keeps identity in xattrs read the tree as unknown. OpenCloud posixfs is the case that surfaced it: after the DR drill its storage log reports node.Xattrs on the platform administrator space as no data available, the showcase files are assimilated as new objects, and the graph call that lists drives returns no personal drive, so the design spec dereferences undefined. Measured here, source tagged with user.oc.name, on tmpfs and on ext4 alike: rsync -a returns 0 and drops the attribute, rsync -aHX and rsync -aHAX return 0 and carry it. The shared restore flow and the NFS half of this repo now use -aHAX, the form ensure_subject_subvolume.sh already uses, continuing what 69321d4135 started for numeric ownership.

The backup half still has the same gap: backup-docker-to-local 7.0.2 builds its command as rsync -aP --no-D --delete --delete-excluded with no flag passthrough, so until it carries -X and the pin moves, a snapshot holds no attributes for this restore to put back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-17 14:16:29 +02:00
evangelostsak e62ed8c56f Merge remote-tracking branch 'upstream/main' into feature/svc-net-wireguard 2026-10-17 09:34:00 +02:00
evangelostsak b3a9e140af test(ci): the compose title carries the direct arm glyph 2026-10-17 00:03:18 +02:00
kevinveenbirkenbach f8311d0780 Merge branch 'main' into feature/network/multi-network-serving 2026-10-16 22:36:07 +02:00
kevinveenbirkenbachandClaude Opus 5.5 8e7dd508c9 style(tests): apply the formatter to the network_served tests
The autoformat step of the gate wrapped the long assertions that
660ad2ff16 committed unformatted. No behaviour change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 21:40:19 +02:00
evangelostsak 9c93e32a73 revert(ci): the axis tests cover the swarm-only vpn axis again 2026-10-16 21:26:16 +02:00
evangelostsak 779e995bf3 refactor: drop rationale and narration from tests/unit/python/utils/tests/swarm/test_write_extras.py 2026-10-16 19:10:01 +02:00
evangelostsak 2a60fd1f32 refactor: drop rationale and narration from tests/unit/python/utils/tests/swarm/test_mesh_steps.py 2026-10-16 19:10:01 +02:00
evangelostsak 2698bb7996 refactor: drop rationale and narration from tests/unit/python/utils/github/variant/test_axes.py 2026-10-16 19:10:01 +02:00
evangelostsak d4179b747a test(ci): compose titles state their mesh and impossible mesh pins abort 2026-10-16 14:48:36 +02:00
kevinveenbirkenbachandClaude Opus 5.5 660ad2ff16 fix(web-app-seaweedfs): keep the frontend on every node that serves clearnet
The frontend and its sso, logout, dashboard and prometheus consumers were
gated on 'svc-net-tor' not in group_names, written when a Tor node served
onion only. On a multi node that removed the clearnet filer vhost, so the
storage specs of every SeaweedFS consumer failed on a clearnet pick with
ERR_SSL_UNRECOGNIZED_NAME_ALERT on filer.<domain>.

lookup('network_served', <network>, *role_ids) answers whether the node's
resolved NETWORK_MODE serves a network and every named role is deployed.
The gates use it instead of restating the mode resolution and the provider
check, so the frontend now drops only on a tor-only node. is_explicit_truth
registers the lookup in GATED_LOOKUPS so the static closure still reads
those flags as dynamic, and the dynamic-flag and universal-logout checks ask
is_explicit_truth instead of matching "in group_names" themselves.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 13:35:29 +02:00
kevinveenbirkenbach 338a2ffa7f Merge branch 'feature/designer' into feature/store-i18n 2026-10-16 13:19:15 +02:00
kevinveenbirkenbachandClaude Opus 5.5 55c95e6960 feat(networks): hand a role's non-canonical siblings over to its canonical URL
Some servers build every absolute URL from one site URL, so a second
network cannot be answered under its own origin. reachability.handover
keeps the siblings but turns each one outside the canonical network into a
302 to the same path on the canonical URL (basic and ws_generic vhosts).
The redirect target comes from the canonical domain, not the application
id, because lookup('tls', application_id) follows the network of the vhost
being rendered and would point the sibling at itself.

The flag also drops Onion-Location for the role (Tor Browser would bounce
between the two), skips those siblings in the CSP crawl, expects 302 from
the webserver health check and keeps Playwright on the canonical domain.
The key needs a nocheck reason like single_mode.

Mattermost opts in: it builds every OAuth callback from its one SiteURL, so
an onion login came back on the clearnet callback and failed with 400.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 12:37:16 +02:00
kevinveenbirkenbach 78bdb6e515 Merge branch 'main' into feature/designer
# Conflicts:
#	docs/contributing/artefact/files/plugins/lookup/applications.md
2026-10-16 11:44:29 +02:00
kevinveenbirkenbachandClaude Opus 5.5 67f6c8d7e8 fix(cache): keep the applications carrier valid across token store writes
Since 778c18fb23 the carrier key holds the tokens file's mtime and size.
sys-token-store changed that stat on every inclusion (00_core.yml
touched the file) and on every persisted token, and nothing re-parked a
carrier afterwards. Every later task's fresh worker then rendered the
applications SPOT cold, 30 to 80 s each. On main fea6f31d9f (run
38812964308) seven heavy jobs logged 194 to 287 such tasks and ran out
of the 20400 s budget; the same jobs before 778c18fb23 logged 65 to 83,
all of them real waits.

- 00_core.yml touches the store with modification_time and access_time
  preserved, so it only creates a missing file. In swarm the controller
  sees the store over NFS with attribute caching, and the touch alone
  made the second pass go cold without any write.
- write.yml re-parks the carrier when the persist or the controller
  mirror changed the store. Both run once, so every host that ran the
  include gets the same verdict.
- tasks/utils/setup/role.yml re-parks after every role on every host,
  so hosts that did not run the writing role recover too. Where the
  key still matches this serves the carried payload; measured 0.1 s per
  role in a local compose deploy.
- The tokens file enters the key as the SHA-256 of its content,
  memoised per inode, mtime and size, so a touch no longer changes the
  key. A stat that still matches serves the memo (1.67 us against
  1.41 us for the old stat signature), and a stale stat keeps key and
  YAML cache on the same old content instead of pairing new content
  with an old key.
- applications_carrier opens the tokens file before it computes the
  key, so on the NFS-mounted controller the parked key reflects the
  revalidated attributes rather than a cached stat that changes a
  minute later.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 11:11:47 +02:00
evangelostsak e00008070d Merge remote-tracking branch 'upstream/main' into feature/svc-net-wireguard 2026-10-16 08:58:56 +02:00
evangelostsak a473d4e03d test(ci): the vpn axis rotates on smallest compose variants only 2026-10-16 03:40:16 +02:00
evangelostsak 759bb237b4 test(mesh): hub, spoke and full-mesh members listen and dial as planned 2026-10-16 03:40:15 +02:00
kevinveenbirkenbach 810c8bc413 Merge branch 'main' into feature/store-i18n 2026-10-15 23:51:34 +02:00