Commit Graph
110 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 fbd4b3a18d refactor(meta): label categories with one hieroglyph
A deploy job name repeats the category path on every row of the matrix. One Egyptian hieroglyph costs four UTF-8 bytes where the two Chinese characters it replaces cost six. Unicode names each sign by its Gardiner code, so the opaque value stays identifiable without a comment per entry.

A label has to stay one codepoint: a sign written with a variation selector costs seven bytes and is worse than what it replaced. tests/integration/meta/categories/test_glyph.py enforces that.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-20 23:17:19 +02:00
kevinveenbirkenbach d0b45f8b2d Merge branch 'main' of github.com:infinito-nexus/core into feature/store-i18n
# Conflicts:
#	.github/workflows/call-images-build-ci.yml
#	.github/workflows/call-test-deploy.yml
#	.github/workflows/call-test-install-pkgmgr.yml
#	.github/workflows/call-test-workspace.yml
#	roles/svc-bkp-local-2-device/README.md
#	roles/svc-storage-nfs-client/README.md
#	roles/web-app-pixelfed/meta/services.yml
#	scripts/tests/deploy/act/workflow.sh
#	tests/unit/python/utils/recovery/test_base.py
2026-10-20 20:05:15 +02:00
kevinveenbirkenbachandClaude Opus 5.5 21eba1e88f fix(backup): carry hard links, ACLs and xattrs on every backup and recover hop
baudolo 7.1.x writes volume generations with `rsync -aHAX`, so the
producing host keeps `user.*` extended attributes (OpenCloud keeps its
storage metadata in `user.oc.*`). Every later copy dropped them again,
because none of the other rsync calls passed `-X` (nor `-A`/`-H`):

- NFS export snapshots (svc-bkp-nfs-2-local `script.sh`),
- the backup host's pull (svc-bkp-remote-2-local `pull_specific_host.py`),
- the device sync (svc-bkp-local-2-device `script.py`),
- every recover into a live target (`utils/recovery/base.py`, used by the
  volume, NFS and device recover scripts),
- the `recover full` and `recover remote` tree pulls.

A disaster recovery through the documented chain therefore restored
OpenCloud without `user.oc.*`, silently: the copies succeeded and the
dirval stamp does not cover attributes.

All of these now run `rsync -aHAX` (`-aHAXP` for the device sync). A
target filesystem without xattr support makes rsync exit non-zero as soon
as a source file carries attributes, so the hop fails loudly instead of
dropping them; there is no per-device opt-out. The single-image pull in
`recover remote` keeps `-a`. The provider's ssh-wrapper needs no change:
its glob accepts any server flags and its character guard allows them.

Two tests write a `user.oc.id` attribute and assert it survives the NFS
snapshot and the recover; with the old `-a` both fail with
`OSError: [Errno 61] No data available`.

Closes #1021

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 18:20:46 +02:00
kevinveenbirkenbachandClaude Opus 5.5 310663ae8c feat(compose): recreate the stack network without losing volumes or worktree stacks
When the IPAM config of a compose project's network changes (#999 added
`ip_range`), compose recreates the network on the next `up`. With a
foreign endpoint attached, as every worktree stack is on the primary's
network, compose first stops the project's own containers, then fails
with `error while removing network: ... has active endpoints`, and a
repeated `up` or `up --force-recreate` does not recover. On 2026-10-19
this took the shared registry/package cache and coredns down for every
local session.

`make compose-network-recreate` (alias `i8conere`) runs the sequence that
recovered it: detach every running container of another project from the
network, `docker compose down --remove-orphans` without `-v` so the
caches keep their volumes, the normal `up`, then attach the detached
containers again, which draw addresses from the dynamic pool. If `up`
fails, the still-detached containers are listed instead of reattached.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 13:45:37 +02:00
kevinveenbirkenbachandClaude Opus 5 5647a2d2bd style: take the line breaks autoformat wrote
cfe20f0726 and 4915a401d4 wrapped three statements more cautiously than
the formatter wants. Without this the lint phase rewrites the same diff
into the tree on every run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-19 03:41:36 +02:00
kevinveenbirkenbachandClaude Opus 5 4915a401d4 fix(ci): replay the mesh state of the row that failed
failed_selections reproduced mode, onion state, distro and architecture
but not the mesh axis. These tokens feed the priority line, where an
unpinned axis offers both states, so a failed direct swarm row came back
as itself and as a meshed sibling: two jobs to reproduce one.

Only a mode the axis covers takes the pin. pinned_vpn_states returns an
empty list for a compose row pinned to the mesh, and an empty list drops
the row from the matrix -- the class ff75343aba already had to fix once
for held runs.

The fixture had drifted from what the matrix emits: row_label built
titles with no mesh glyph while axes.assign writes one for every swarm
and compose row, so parse_label fell back to its False default and the
new &direct assertion would have been right by accident. row_label and
deploy_job_name now place the glyph under the same condition as
axes.assign, and a meshed row has a case of its own, which nothing
covered before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-19 03:25:39 +02:00
kevinveenbirkenbach 15930945fe Merge branch 'main' into feature/store-i18n 2026-10-19 01:54:37 +02:00
kevinveenbirkenbachandClaude Opus 5 67dd278d0c fix(deploy): forward the guide host into the container Ansible runs in
e17d182d0a gave web-app-docs a switch that turns the language assertion
off on a row where docs only rides along, and read it from the
guide_host environment variable. The deploy workflow sets that variable
on the runner, but a compose deploy runs Ansible inside the infinito
container, where only _FORWARDED_ENV_KEYS arrives. guide_host was not in
it, so the lookup resolved empty, the comparison against the row's own
id was true everywhere, and the switch never fired.

Run 39137606155 shows both halves: the passenger rows 117711219652 and
117711220306 carry guide_host: web-app-docs and each spent about 7000 s
of a 20400 s budget in that one assertion, while the docs row itself,
117711219660, carries an empty guide_host and therefore keeps running
it. Forwarding the variable preserves that split rather than silencing
the assertion everywhere.

APP_ID is in the tuple for the same reason and feeds the guide_role
lookup from the same workflow step.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-19 01:10:57 +02:00
kevinveenbirkenbach da449f8894 Merge pull request #323 from evangelostsak/feature/svc-net-wireguard
feat(svc-net-wireguard): Dockerized WireGuard role + test-e2e-cli DinD e2e
2026-10-18 19:50:42 +02:00
kevinveenbirkenbach 4a469e6476 Merge branch 'main' into feature/store-i18n 2026-10-18 13:20:06 +02:00
kevinveenbirkenbachandClaude Opus 5 6ecfccfc38 fix(tools): publish the model gateway in the lane and judge the lane by what it serves
The lane ran litellm but never republished its port, so a host-side probe hit the compose proxy on the same number and read 502. The lane could therefore never report ready, and the oracle suite skipped itself to a green exit instead of asking a judge. A handler reads the port from the role the way translate.py already does, the runner forwards it, and TOOLS_LANE_KEYS takes it up: compose_file_args layers the override in without checking its variables, so an .env written before the key existed would abort every compose command rather than merely drop the lane.

The gateway's master key is pinned in the lane inventory for the reason the System One key already is: the provisioner would otherwise generate it into a vault the checks cannot read back, and the oracle judge is not a deployed consumer with a virtual key of its own.

serving() stops treating reachability as readiness. A lane brought up before AI_LOCAL_MODELS named a different model keeps serving the old one, and a gateway holding a key these checks cannot use looks healthy from outside while refusing every call they make. It now asks /models with the bearer token and requires the configured alias, and requires the gateway to offer every language the catalogs name, because that advertisement is what the translation run waits on. Without this the lane waved itself through and no change to its roles ever shipped.

Two lane-scoped overrides come with it. The ollama role caps itself at four CPUs, which left the host at 361 percent while twenty cores idled, so the lane gives it the host count the way the engine entry above it already does. And the lane pins the router to its declared order: the decider may send a pair the engine serves to the chat model, which a batch over 183 catalogs cannot pay for. Deployments keep the decider.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-18 10:06:48 +02:00
kevinveenbirkenbachandClaude Opus 5.5 088c8be14f fix(ci): pull workspace images through the GHCR mirror like the deploy jobs
The workspace jobs pulled every image anonymously from Docker Hub and
two of them failed on its rate limit in run 39059128678 (#990): the
fedora swarm drill on busybox:1.38 on all three nodes, the centos
compose drill while building tor-smtp_custom. The regular deploy jobs
pull through the GHCR mirror, but the mirror was tied to being a real
GitHub runner: Profile.image_mirror_enabled() returned
runs_on_github(), and the swarm routine generated mirrors.yml only when
cli.meta.runtime reported github, which is act inside make
swarm-zombie.

The mirror is now its own capability, INFINITO_GHCR_MIRROR (next to
INFINITO_GHCR_MIRROR_PREFIX), written as a run fact like
INFINITO_CACHE_STACK: true on a real GitHub runner,
false everywhere else, and a non-empty exported value wins for the run.
Profile.image_mirror_enabled() reads the declaration and keeps
runs_on_github() as the fallback, and 02_provision_inventory.sh gates
on the same value instead of the runtime name. make swarm-zombie
forwards it into act through ACT_ENV, since act ignores the host env.

call-test-workspace.yml declares INFINITO_GHCR_MIRROR=true and passes
INFINITO_GHCR_MIRROR_PREFIX into the workspace container. Nothing else
of the workspace changes: is_ci, runs_on_github, the cache stack and
the docker root stay local-like. The step that loads the rescue paths
now removes the .env it generated on the runner, because on debian
rows load.sh sourced it inside the container and the run inherited
INFINITO_RUNNING_ON_GITHUB=true.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-17 22:40:13 +02:00
kevinveenbirkenbachandClaude Opus 5 2f9a53f073 fix(tools): reach the lane from the stack, and judge every role it serves
Three faults, one mechanism each, that together let the oracle suite report
a verdict it never reached.

The address. lane_url handed out the runner's compose service name, but the
stack's containers resolve through the deployment's CoreDNS, which never
learns one: getent hosts tools answers nothing inside them, so every check
that asked the lane from there skipped itself. The bring-up now writes the
runner's address, picked as the one inside INFINITO_SUBNET because the
runner sits on every network it was ever brought up on, and the checks read
it from there. It is rewritten on both paths, because a restart gives the
runner a new one.

The readiness. serving judged the engine alone, so a lane whose engine make
i18n had already brought up counted as up while System One was down. It now
probes every role the caller names, which is why the runner republishes
System One's port: a container IP is unreachable from the host that decides
whether to deploy.

The verdict. A silent lane made the suite skip, and a skip exits green.
Only a deploy that is already running leaves the stack alone and exits 0
now; a crashed deploy and a deployed-but-silent role both exit 1.

The override's third :? guard is added to the gate that decides whether the
file is layered in at all, next to the one that was already missing. Without
it an .env written before these keys aborts every compose command, down to
compose-down, instead of dropping the lane. A test now keeps that list and
the override's guards in step.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-17 20:27:54 +02:00
evangelostsak 1343f77929 refactor: drop rationale and narration from tests/unit/python/cli/administration/inventory/mesh/test_write.py 2026-10-16 19:10:01 +02:00
evangelostsak 277a2aad76 refactor: drop rationale and narration from tests/unit/python/cli/administration/inventory/mesh/test_transport.py 2026-10-16 19:10:01 +02:00
evangelostsak 7c85cedc8e refactor: drop rationale and narration from tests/unit/python/cli/administration/inventory/mesh/test_plan.py 2026-10-16 19:10:01 +02:00
evangelostsak 814fc8b336 refactor: drop rationale and narration from tests/unit/python/cli/administration/inventory/mesh/test_inventory.py 2026-10-16 19:10:01 +02:00
evangelostsak a88f69bd35 refactor: drop rationale and narration from tests/unit/python/cli/administration/inventory/mesh/test_full_mesh.py 2026-10-16 19:10:01 +02:00
evangelostsak c95e5d06c4 refactor: drop rationale and narration from tests/unit/python/cli/administration/inventory/mesh/test_cli.py 2026-10-16 19:10:01 +02:00
evangelostsak 617879f680 test(mesh-cli): full meshes peer every member with every other
Covers the declarations read from every host, the refusal of differing ones,
every peer pinned to its /32, no hub anywhere, the host of one mesh holding no
peer of the other, and a second run leaving every file alone.
2026-10-16 03:40:15 +02:00
evangelostsak f534c973a7 test(mesh-cli): an inventory without a hub host has no hub 2026-10-16 03:40:15 +02:00
evangelostsak b1279525bf test(mesh): the transport switch under a baked layout 2026-10-14 15:03:41 +02:00
evangelostsak 56ce03d370 test(mesh): the CLI reads its meshes from host_vars
Mesh names are asserted structurally: the baked layout names every mesh in
every file, so a substring match would pass on a host holding no interface.
2026-10-14 15:03:41 +02:00
evangelostsak a0a7436d19 test(mesh): the writer under a layout baked into host_vars 2026-10-14 15:03:41 +02:00
evangelostsak 54e237bddb test(mesh): plan against a topology instead of raw groups 2026-10-14 15:03:41 +02:00
evangelostsak e30b17276c test(mesh): declarations, pool and membership read from host_vars
Covers per-role and per-host membership, a host two roles put into one mesh,
the hub never being a spoke, an undeclared mesh being refused, and the
example's members matching its table.
2026-10-14 15:03:41 +02:00
evangelostsak 31cf7362d5 test(mesh): remove the contract tests with the contract 2026-10-14 15:03:41 +02:00
evangelostsak e9dbab5835 test(mesh): the contract shape carries the primary flag 2026-10-14 11:23:23 +02:00
evangelostsak 05dc4af041 test(mesh): the controller joins the first declared mesh by default 2026-10-14 11:23:23 +02:00
evangelostsak a521937794 test(mesh): no two interfaces of a host route one prefix
Covers the swarm lab under the three-mesh example, where the NFS server is a
spoke of two meshes, and generated topologies of one to eight meshes.
2026-10-14 11:23:23 +02:00
evangelostsak 06cc872c1c test(mesh): pin how the primary mesh is chosen 2026-10-14 11:23:23 +02:00
evangelostsak e043ad0c00 test(mesh): zip the planned meshes strictly against their specs
The any-number-of-meshes test zipped the specs against the planned meshes
without strict=, which ruff reports as B905 and the lint-python job failed on.
Both sequences come from the same declarations, so strict=True also turns a
plan that drops a mesh into a failure instead of a silently shorter loop.
2026-10-14 00:27:17 +02:00
kevinveenbirkenbachandClaude Opus 5 944a228ffb fix(tests): import the mock submodule the deploy-container suite uses
test_container.py decorates with @unittest.mock.patch but imports only
unittest. A submodule is not bound as an attribute by importing its
package, so the name resolves only while some other module in the same
interpreter has already imported unittest.mock.

That makes the defect invisible in a full run, where another suite
imports it first, and fatal under a narrow INFINITO_TEST_PATTERN or on
an xdist worker that receives this file alone: collection dies with
AttributeError: module 'unittest' has no attribute 'mock'.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-14 00:00:41 +02:00
evangelostsak b777b65e84 test(mesh): hold the planning guarantees at any mesh count
The guarantees were asserted against the declared swarm and data pair,
so they proved two meshes work rather than that the planner works. A
deployment declares its own, and the review asks for any number of them with
the nodes arranged however the client wants.

The same invariants now run over generated topologies: one to eight meshes,
three arrangements each, hosts dealt out so some belong to several meshes and
some to none. 2247 subtests, four seconds, and the planner needed no change to
pass them.
2026-10-13 19:36:43 +02:00
kevinveenbirkenbach de3127683f Merge branch 'feature/designer' into feature/store-i18n 2026-10-13 12:56:06 +02:00
kevinveenbirkenbachandClaude Opus 5 7c13e96c96 Merge branch 'main' into feature/store-i18n
Twelve files conflicted; most resolutions keep both sides rather than pick
one.

cli env.py and compose.py: both branches extend the same spots. The tools
override is appended before write_override, because that call filters the
mirror override to the compose files it is handed. compose.py keeps this
branch's CACHE_ROLE_SHELL_DIR, since the cache scripts moved into a role and
main's literal scripts/docker/cache path is their old home, and takes main's
mirrored init image alongside it.

utils/docker/image/discovery.py: this branch's version_variants expansion
plus main's whole runner-image block, which runner_mirror imports.

sys-svc-compose network create: main's side wins on substance and this
branch's on structure. main carries eda6249d7b, the fix for a probe that
matched an id prefix and let a sibling shadow the exact name; this branch had
refactored the task into create/overlay.yml and create/bridge.yml but brought
back the docker_network_info probe that fix replaced. The subfiles stay and
now probe through the exact-name lookup. main's half also read
SWARM_CLUSTER_AVAILABLE, which this branch removed in favour of
IS_SWARM_MODE, as the merge base confirms, so the gates follow this branch.
The lookup's import follows this branch's get_entity_name to entity_name
rename.

web-app-pixelfed: the custom entrypoints stay removed, since the new image
brings its own and the variables behind them no longer exist, and main's
host_alias=true comes along.

web-app-suitecrm: main's session.yaml and its config/packages path, because
only that version wires framework.session and only that directory is read by
Symfony.

web-app-odoo: main's onionOptions import is required by the test.use below
it; this branch's test-seaweedfs registration stays.

Four defects surfaced that no conflict marker shows.

The auto-merge of web-app-suitecrm/vars/main.yml duplicated
SUITECRM_SESSION_CONFIG_HOST, which ansible-lint rejects. Removing it exposed
the second: SUITECRM_SESSION_CONFIG_DOCKER still named config/session.yaml
while the mount had moved to config/packages, so the freshness comparison in
tasks/01_manager_ops.yml would have tested a path nothing writes.

svc-db-elasticsearch carried environment: on two include_tasks, which Ansible
accepts on include_role only. ELASTIC_PW now sits on the five tasks that read
it, which is also the tighter scope for a credential.

The openclaw templates render NETWORK_TRUSTED_PROXY_CIDRS while their test
renderer passes only its explicit kwargs, so seven tests raised Undefined.
That predates the merge: the template line is on this branch and not on main,
and both sides share the same test file.

test_runner_mirror neutralises four environment variables under clear=False
and missed three, so the gpu and tools overrides leaked in from the container
and the assertion on the layered list broke. It now neutralises all seven.

make quality-high passes both phases.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-13 10:57:22 +02:00
evangelostsak 34b49036d7 Merge remote-tracking branch 'upstream/main' into feature/svc-net-wireguard 2026-10-12 17:52:36 +02:00
kevinveenbirkenbachandClaude Opus 5.5 0327114c44 fix(inventory): drop disable= toggles once their key leaves the list
Provisioning merges into host_vars and never deletes, so the false
enabled/shared toggles that apply_services_disabled writes for a key stayed
in the round inventory after a later run dropped that key from disable=.
That covered the provider's own entry too: with seaweedfs once disabled,
web-app-seaweedfs kept shared: false, the service registry offered no
seaweedfs provider, container_networks attached no consumer, and BookWyrm
ran USE_S3=true against http://seaweedfs:8333 without a route.

reconcile_services_disabled_from_env records the keys of each disable run
in the host_vars key _INFINITO_SERVICES_DISABLED. On the next run it deletes,
in every application block, the enabled/shared toggles that are still
exactly false for a recorded key the current disable no longer lists, drops
a block left empty, and rewrites or removes the record. It runs right after
ensure_host_vars_file, before credentials and the variant bake, so a variant
that wants a toggle off writes it again. With an unchanged or empty disable
it does nothing. An inventory written before this change has no record, so
its older toggles are only reverted after one run with the old list.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 11:07:21 +02:00
kevinveenbirkenbachandClaude Opus 5.5 1ba796d7f1 fix(ci): pull the images the runner uses itself from the GHCR mirror
The GHCR mirror covered role images only. The images the runner pulls on
its own went straight to Docker Hub: coredns from compose.yml, the cache
stack in compose/cache.override.yml and alpine:latest, the init image of
scripts/docker/cache/package-frontend/certs.sh. With the anonymous
githubactions quota spent, the guide step of Postmarks#0 (main run
37954203718) and Shopware#2 (#872) died on "toomanyrequests" while pulling
alpine:latest, after coredns had hit the same limit.

iter_runner_images (utils/docker/image/discovery.py) finds those images,
and the mirror sync and the fork-PR wait now cover them next to the role
images. On a real GitHub runner compose_file_args layers a generated
build/compose/runner-mirror.override.yml that points only the services of
the layered compose files at their mirror, and the certificate generator
gets the mirrored init image. Local runs and act are unchanged, and the
coredns literal stays in compose.yml for Dependabot. test-dns now waits for
images-mirror-missing like the deploy and workspace jobs, since it brings up
the stack too.

Verified with docker compose config on a simulated GitHub runner: the
service set is unchanged and only the runner images point at
ghcr.io/infinito-nexus/core/mirror/docker.io/..., and an existing role mirror
under that scheme pulls anonymously.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 10:37:41 +02:00
kevinveenbirkenbachandClaude Opus 5 d3610af31f fix(disable): turn off a service key that only a variant bakes
`disable=lam` aborted the local deploy of web-app-openbao with "`disable`
conflicts with the current inventory state", and web-app-moodle carries the same
construction in two of its variants.

The writer and the conflict check disagreed on scope. apply_services_disabled
walked `find_roles_with_service`, the roles whose meta/services.yml declares the
key, so neither role was in that set for `lam` and nothing was written.
find_services_disabled_conflicts walks every deployed app's host_vars instead,
found the `lam` block the variant had baked, and refused the run. Dropping
`lam` from the list is not an answer either: the round then deploys web-app-lam
although ldap is off.

The writer now also flips a key it finds in host_vars under an app whose role
does not declare it, and only the flags that are already in that block. It never
creates a key or a flag, which is what `_declared_toggles` exists to prevent:
compare_application_keys rejects a host_vars key the role contract does not
carry. Overwriting a value the variant already wrote introduces nothing, so that
rule is untouched; test_skips_app_whose_role_does_not_define_service still
passes and pins it.

The alternative the TODO proposed, declaring `lam` in meta/services.yml with the
`'web-app-lam' in group_names` flags, is the worse trade: a consumer entry is
resolved through the service registry, so it would bind web-app-lam and its
whole closure into every round that carries openbao or moodle, for a key both
roles read only to gate a Playwright spec.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-09 22:48:06 +02:00
evangelostsak ff1db88753 revert(swarm-test): remove the SSH-on-underlay measurement arm
The arm existed to answer one question: how much of the meshed arm's
cost is the connection plugin and how much is the tunnel. Runs 37592061960
(docker), 37753506661 (SSH on the underlay) and 37621556682 (SSH over
WireGuard) answered it on web-app-gitea#0, debian/ext4:

  pass 1  5964s / 8511s / 8832s
  pass 2  5319s / 7472s / 7705s

Leaving ansible_connection: docker costs +4700s, the tunnel +554s - 89.5%
against 10.5%. The answer is recorded; the scaffolding does not need to ship.

Reverts d792519725, 1b28f87b37, 89b8a8c8d1, 0470491e1a, 529c5ace43,
dd3d10a9a2, c20dd0e5f5, 458d8956e1, 63ade1e03b, d281555945 and 4adfb6e3d2.
95cfc8b7b7 is already superseded by bbfd82f89, which sets the profiling limit
upstream.
2026-10-09 12:12:03 +02:00
evangelostsak 4adfb6e3d8 test(mesh-cli): cover the addresses the transport dials
One test asserts ansible_host lands on the given address, the other
that a missing member raises. Either would have caught the UNREACHABLE run
before it was dispatched.
2026-10-08 10:58:23 +02:00
kevinveenbirkenbachandClaude Opus 5 f2fedfc677 Merge branch 'main' into feature/store-i18n
Fourteen real conflicts plus the 66 generated READMEs.

utils/cache/files.py and its test: both sides added a skip-dir rule, resolved
as a union so the root `build/` prune and the per-root walk key coexist.

plugins/lookup/database.py and postgres_libraries.py: main's added lookups kept,
with get_entity_name renamed to entity_name as this branch renamed it.

roles/web-app-{mailu,matrix}/meta/services.yml: main's `covers: [email]` and its
MAIL_PROVIDER-gated email entry, plus this branch's gitlab/jira/n8n/jitsi
consumers, which the merge base carried on neither side.

group_vars/all/17_resource.yml: this branch's lookup('resource', ...) refactor;
main's single added literal is already covered by _POSTGRES_MAX_LOCKS_PER_TRANSACTION.

docs/requirements/035-mcp-proxy-expansion.md: this branch moved the design into
ADR 0003, so the requirement keeps only the user story and the criteria. Main's
delta inside the moved block was two new roles, carried into the ADR audit:
web-app-stalwart as adapter_candidate, svc-net-tor-smtp as privileged_control_plane.
The no_surface total said 99 while its own subcategories summed to 112.

inventories/development/default.yml: biber keeps no explicit email. Both sides fixed
the same CI failure; this branch fixed it in _apply_account_defaults, which renders
biber@{{ DOMAIN_PRIMARY }} (utils/cache/users/__init__.py:68) — byte-identical to
the address main hardcoded, so the inventory line is a second copy of one fact.

roles/web-app-{gitlab,keycloak}/files/playwright/playwright.spec.js and
roles/web-app-gitlab/templates/playwright.env.j2: union of both import sets, with
requireDotenvValue kept over decodeDotenvQuotedValue for CANONICAL_DOMAIN because
an empty value must fail rather than match every substring.

The 66 role READMEs take this side: aceb5295ef moved Cosmos, Quick Setup and
Credits to the documentation build, and test_readme.py now rejects them in a
committed README. Main's prose edits inside them are carried over by hand (the
Mailu deprecation banner, Mailu -> Stalwart in erpnext and zammad), and the two
READMEs main added fresh are stripped of the three generated sections.
roles/web-app-sphinx/README.md stays deleted; 6eb9a59d14 merged that role into
web-app-docs. The root README's generated role table was regenerated from the
merged metadata.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-04 01:57:17 +02:00
evangelostsak 25c7fdd1a5 test(mesh): pin the switched host to a tty-free transport
The carriage returns a pty adds are invisible until a role parses its own
output, so the absence of the tty is asserted where the transport is
written rather than left to the next role that reads stdout.
2026-10-03 23:20:26 +02:00
kevinveenbirkenbachandClaude Opus 5 0d2fadce0c feat(cache): serve every checkout's own cache declarations
A checkout that declared a new cache host rendered the artefacts and reached nothing: the Nexus proxy repository, the frontend's leaf certificate and the nginx upstream map all live in the running stack, which only the owning checkout set up. cache-apply installs all three and reloads the frontend, and compose up runs it for a checkout that merely joins the network, so the manual call is left for a declaration changed between two bring-ups.

The map source is no longer selectable. INFINITO_CACHE_CONF_SOURCE existed to let a worktree read the primary checkout's map instead of its own, but the frontend mounts one map and the install replaces it, so the checkout that applied last wins either way: the knob chose which file to point at, never who prevailed. Dropping it also retires the git subprocess that resolved the owning checkout on every env build.

Two checkouts sharing a stack therefore overwrite each other's map, which the documentation now states instead of implying a selector could prevent it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-03 21:40:34 +02:00
evangelostsak e456b1eee6 Merge remote-tracking branch 'origin/main' into feature/svc-net-wireguard
One conflict, in the MCP exclusion list of requirement 035. Both sides
edited the same line: main added svc-net-tor-smtp, this branch replaced the
three svc-net-wireguard-{core,plain,firewalled} entries with the single
svc-net-wireguard it deploys. Resolved to the union of both — 24 entries —
which is what the merged tree actually holds.

Everything else merged clean and was checked for overlap rather than
assumed: no reference anywhere still names the three replaced roles;
matrix.py kept main's PRIMARY_APPS alongside the reordered round, and the
bootstrap pass does not need it, since only one template reads it and
group_vars defaults it empty; and main's ansible.cfg addition sits in
[defaults], so the ssh_args the mesh tool derives its own connection
settings from are untouched.
2026-10-03 02:29:45 +02:00
evangelostsak 152588b511 perf(mesh): retune the multiplexer for the hosts the mesh switches
The pass that runs over the mesh reaches every host by SSH where the
pass before it used the container connection, and it pays that difference
once per task. Measured in a container: an exec through a warm multiplexer
costs 7ms, a fresh connection 85ms, and 165ms once sudo asks for the
password every task demands. The pass runs thousands of tasks.

Two settings threw the multiplexer away. A persist window of 20s closes the
master whenever a host waits longer than that for its next task, which in a
linear play is most of them; and the named ControlPath points at a directory
nothing creates, so a master that cannot open its socket reconnects per task
without reporting it.

Neither is retuned for the repository. The tool that moves a host onto the
mesh now writes that host's own ssh_args, derived from the repository's
value so every other option is kept and only the window and the socket
differ. A host the mesh never touches keeps exactly what it had.

Confirmed against the built command: a switched host gets ControlPersist=600s
and a ControlPath under the directory Ansible creates, while a host beside it
keeps the repository's own.
2026-10-02 16:35:29 +02:00
evangelostsak dc14e8a3a8 fix(mesh): keep every member on the address it already holds
The offset came from the spoke's position in a sorted list, so a host that
sorts earlier renumbered every member behind it. `wg syncconf` carries keys
and peers onto a live interface and never its address, so a renumbered member
keeps answering on the old address while the peers and the inventory have
moved to the new one -- and the address it gave up can be handed to a new
member while the old owner still holds it.

The planner now reuses the addresses already on disk and allocates only an
unseen member, from the lowest free offset. A value outside the subnet, a
duplicate, and the hub's own offset are refused, so a mirrored host_vars
cannot hand a spoke the hub's address. Rotation keeps addresses: it replaces
identities, not addresses.

The routing assertions re-planned the mesh and compared the result with
itself, so a writer that stored the mesh subnet in place of the routed range
would have passed; they now read the entry off disk, which is what the role
reads.
2026-10-02 00:54:20 +02:00
kevinveenbirkenbachandClaude Opus 5 d88fe1d83a test(lint): let the legacy-path guard see the trees that still named the old layout
`config/main.yml` exists in no role: f65f4d60fb migrated the last one. A guard
against the string already exists, `TestNoLegacyPathReferences`, and it names
`config/main.yml` among its LEGACY_TOKENS. It scanned plugins, utils, cli,
roles, scripts, filter_plugins and lookup_plugins, which is every tree except
the two the stale references actually lived in.

So nine of them survived in `tests/`, and the guard reported green the whole
time. The code underneath was already correct everywhere: `test_csp_consistency`
and `test_role_wiring` iterate ROLE_FILE_META_SERVICES, and only their docstrings
and one assertion message still said `config/main.yml`. Had any of them globbed
the old path the test would have gone vacuous rather than red, which is the
failure mode worth naming here.

`tests` and `tasks` are now scanned. The targets come from
`utils/roles/mapping.py`, not from guesswork: `config/main.yml` is
`meta/services.yml`, `schema/main.yml` is `meta/secrets.yml` (confirmed at
`cli/administration/inventory/provision/reset.py:67`, which reads exactly that
file), and `users/main.yml` is `meta/users.yml`.

Scanning `tasks` exposed a second dead pointer. The migration tool became a
package on 2026-09-14, but EXEMPT_FILES still named `migrate_meta_layout.py`, a
file that no longer exists, so the exemption was inert and only invisible
because `tasks` went unscanned. Two dead pointers covering for each other. The
exemption is now a directory, so a new module in that package cannot fall out of
it. `test_no_direct_calls.py` named the same dead path in prose; that lint
allows through nocheck markers at the call site, so nothing was broken there.

`docs/administration/configuration/email.md` was the worst of them, because it
instructed operators rather than describing tests: it told them to write
`compose.services.email` into `roles/<application_id>/config/main.yml`. The
consumer, `plugins/lookup/email.py:255`, reads `entry["services"]["email"]` and
says in its own comment that there is no `compose.services` envelope. The role
snippet, the inventory snippet and the precedence list now match the reader.
`group_vars/all/TODO.md` pointed at the same dead file.

Verified by watching the guard fail, twice. Re-introducing `config/main.yml` in
a test file reports it; pointing the new directory exemption at a wrong path
reports `tasks/utils/migrate_meta_layout/credentials.py`, which proves both that
`tasks` is scanned and that the exemption is the only thing letting the
migration tool through. Restored, 11 passed.

The changelogs under `docs/changelog/` keep their mentions, and `.md` stays out
of the scanned suffixes for that reason.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-30 21:27:40 +02:00
kevinveenbirkenbachandClaude Opus 5 058e76e58c fix(test): read the two planner call sites through the shared cache
`test_no_uncached_file_reads` requires every read under tests/, utils/, cli/,
plugins/ and library/ to go through `utils.cache.files.read_text`, which keeps
one decoded read per file for the whole pytest session. The test added in
3b85e4e1e1 called `Path.read_text` directly and so read init.py and
deploy/cli.py past that cache.

The rule allows `# nocheck: cache-read` only for a read outside the project
tree, which is how bfdf3dee79 handled a tempdir. These are real source files,
so the reader is the answer and the marker would have been the convenient
wrong one; e5430ad4ca is the matching precedent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-30 02:34:39 +02:00