02_provision_inventory.sh copies the manager's host_vars to every node,
the backup host included, so a row with network=multi handed bkp-01
NETWORK_MODE=multi as well. The backup host is deployed from its own
backup.yml, which has no svc-swarm-node group and no Tor provider, so
tor_serves_host is false there and resolve_node_mode raised at the first
tls lookup: "NETWORK_MODE 'multi' needs svc-net-tor with a node onion on
this node" (01_constructor.yml, pixelfed swarm drill on the multi axis,
after a green deploy round and Playwright pass).
extend_inventory already prepares the backup host's own inventory after
the copy; it now also drops NETWORK_MODE from that host's host_vars, so
the backup host derives clearnet as a host without Tor does. The
remote-2-local pull reaches the manager over SSH and needs no onion.
Rows on clearnet or without a mode resolve as before; only the tor and
multi rows, which raised, change.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Declare `__all__` in every package `__init__.py` whose `PROJECT_ROOT`
is imported by its modules (163 packages), so the constant counts as
exported; list every public name where the package defines more.
- Empty the 15 package `__init__.py` files whose `PROJECT_ROOT` no module
imports.
- Declare the public API of `utils.github.variant.axes` in `__all__`,
re-exports included, instead of silencing them with `noqa: F401`; the
list matches the one `feature/store-i18n` already carries.
- Use `import unittest.mock as mock` in `test_applications.py`, as the
rest of the suite does.
- Spell `_NON_PRINTABLE` in `utils/yaml_bootstrap.py` as the equivalent
BMP-only positive class; it matches exactly the same 2111 code points
as the old negated class across U+0000..U+10FFFF.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
baudolo 7.1.x writes volume generations with `rsync -aHAX`, so the
producing host keeps `user.*` extended attributes (OpenCloud keeps its
storage metadata in `user.oc.*`). Every later copy dropped them again,
because none of the other rsync calls passed `-X` (nor `-A`/`-H`):
- NFS export snapshots (svc-bkp-nfs-2-local `script.sh`),
- the backup host's pull (svc-bkp-remote-2-local `pull_specific_host.py`),
- the device sync (svc-bkp-local-2-device `script.py`),
- every recover into a live target (`utils/recovery/base.py`, used by the
volume, NFS and device recover scripts),
- the `recover full` and `recover remote` tree pulls.
A disaster recovery through the documented chain therefore restored
OpenCloud without `user.oc.*`, silently: the copies succeeded and the
dirval stamp does not cover attributes.
All of these now run `rsync -aHAX` (`-aHAXP` for the device sync). A
target filesystem without xattr support makes rsync exit non-zero as soon
as a source file carries attributes, so the hop fails loudly instead of
dropping them; there is no per-device opt-out. The single-image pull in
`recover remote` keeps `-a`. The provider's ssh-wrapper needs no change:
its glob accepts any server flags and its character guard allows them.
Two tests write a `user.oc.id` attribute and assert it survives the NFS
snapshot and the recover; with the old `-a` both fail with
`OSError: [Errno 61] No data available`.
Closes#1021
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The workflows pin `runs-on: ubuntu-NN.NN`, so no job moves to a new Ubuntu
release behind our back, but nothing raised the pin either. A sixth job of
`🔄 Update versions` closes that: `utils/update/runner.py` reads the image
table of actions/runner-images, takes the newest x64 Ubuntu label whose
image carries no preview or beta badge, and raises every older bare
`runs-on: ubuntu-NN.NN` to it; arch-suffixed labels stay untouched. The job
opens the change as one PR over `.github/workflows`, so its CI run tests
the new image before it reaches main.
This commit carries the updater's first result: every job now runs on
ubuntu-26.04 (49 `runs-on`).
actionlint 1.7.12, the newest release, does not know `ubuntu-26.04` yet
(rhysd/actionlint#682, fix proposed in rhysd/actionlint#743), so
`.github/actionlint.yaml` declares the label until a release knows it.
act maps each `ubuntu-NN.NN` label the workflows use to the local platform
image, derived from `.github/workflows`, instead of a fixed list that did
not know ubuntu-26.04.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Combine main's WireGuard mesh axis with this branch's network axis in the CI
variant code. Every matrix row now carries both: network (clearnet, tor,
multi) from utils.github.variant.network and vpn (meshed or direct) from
utils.github.variant.vpn. Label parsing, which main moved to label.py, reads
the network glyph instead of the old tor flag and keeps the vpn glyph; the
selection token, plan table, matrix and trigger CLIs, the workflows and the
swarm routine forward both axes. The test of an incapable priority variant
now compares the set of network modes, since a swarm priority row also
covers both mesh arms.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
compose/cache.shared.override.yml attaches every worktree stack's
infinito service to the primary's network (infinito-nexus-core_default,
172.30.0.0/24) without an ipv4_address. The default network declared
no ip_range, so Docker IPAM handed out the lowest free address and the
worktree stacks took 172.30.0.2, .3, .8, .9 and, whenever the primary
container was down, .10, the primary's INFINITO_IP4. The primary's
`make compose-up` then failed with "Address already in use", and with
it `make quality-high` on main.
The default network now declares ip_range: ${INFINITO_IP_RANGE}, the
upper half of INFINITO_SUBNET. Every static address of the stack (.1,
.4 to .6, .10, .53) sits in the lower half, so a container without a
static address can no longer take one.
INFINITO_IP_RANGE is derived from the final INFINITO_SUBNET after
custom.env is applied, because a worktree pins its subnet there and
`make dotenv-force` regenerates without it in the environment. Existing
worktrees therefore get their own slot's range without a new pin.
A lint checks that every static IP in default.env and every
172.30.${slot}.N pin in scripts/system/worktree/up.sh lies below the
pool.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The workspace jobs pulled every image anonymously from Docker Hub and
two of them failed on its rate limit in run 39059128678 (#990): the
fedora swarm drill on busybox:1.38 on all three nodes, the centos
compose drill while building tor-smtp_custom. The regular deploy jobs
pull through the GHCR mirror, but the mirror was tied to being a real
GitHub runner: Profile.image_mirror_enabled() returned
runs_on_github(), and the swarm routine generated mirrors.yml only when
cli.meta.runtime reported github, which is act inside make
swarm-zombie.
The mirror is now its own capability, INFINITO_GHCR_MIRROR (next to
INFINITO_GHCR_MIRROR_PREFIX), written as a run fact like
INFINITO_CACHE_STACK: true on a real GitHub runner,
false everywhere else, and a non-empty exported value wins for the run.
Profile.image_mirror_enabled() reads the declaration and keeps
runs_on_github() as the fallback, and 02_provision_inventory.sh gates
on the same value instead of the runtime name. make swarm-zombie
forwards it into act through ACT_ENV, since act ignores the host env.
call-test-workspace.yml declares INFINITO_GHCR_MIRROR=true and passes
INFINITO_GHCR_MIRROR_PREFIX into the workspace container. Nothing else
of the workspace changes: is_ci, runs_on_github, the cache stack and
the docker root stay local-like. The step that loads the rescue paths
now removes the .env it generated on the runner, because on debian
rows load.sh sourced it inside the container and the run inherited
INFINITO_RUNNING_ON_GITHUB=true.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every cold render of the applications SPOT walks the templates of all
269 roles, and those templates call lookup('config'), lookup('tls'),
lookup('domain') and others. Each of these re-enters
get_merged_applications while the render guard is set, and each one
rebuilt the unrendered view from scratch: a deepcopy of all role
defaults plus the override merge, about 8 ms, never cached. A local
cold render of the development inventory made 351 such calls, 2.82 s
of a 6.86 s render.
The outer render now keeps a copy of its unrendered view and its cache
key on the render guard, taken before the inventory volumes are
popped, so it is the same view a nested call used to build. A nested
call with the same key returns it; any other key still builds its own.
The guard drops it when the render ends.
Measured on the same inventory: the nested calls took 0.07 s instead
of 2.82 s and the cold render 3.60 s instead of 6.86 s. The rendered
SPOT is unchanged: the old and new dumps differ in 69 user passwords,
which the development inventory regenerates on every read (two runs of
the old code differ in 68), and in one absolute checkout path, the only
other difference.
In CI each cold render took 40 to 77 s and the heaviest swarm cell of
#950 missed its budget by a few minutes (#984).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
utils/yaml_bootstrap.py promises that load_block(text) either equals
yaml.safe_load(text) or raises BootstrapYamlError. A fuzzer comparing
both found two inputs that broke the promise:
- str.splitlines() also breaks lines at \x0b, \x0c and \x1c to \x1e,
which PyYAML's reader rejects as non-printable. "\x1e/: R" came back
as {"/": "R"}, and a non-printable character inside a comment line was
skipped with the comment.
- An indented " ---" on line 1 was taken as the document marker, while
PyYAML reads it as a plain scalar and rejects the mapping that follows.
load_block now refuses any character outside PyYAML's own printable
class before it splits lines, and takes "---" as the document marker
only at column 0. The line breaks splitlines() still honours (\r, \r\n,
NEL, LS, PS) are line breaks for PyYAML too, so splitting stays as is.
The eight inputs join STRUCTURES in the divergence test; against the
previous reader, four of them diverge and fail it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The autoformat step of the gate wrapped the long assertions that
660ad2ff16 committed unformatted. No behaviour change.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The frontend and its sso, logout, dashboard and prometheus consumers were
gated on 'svc-net-tor' not in group_names, written when a Tor node served
onion only. On a multi node that removed the clearnet filer vhost, so the
storage specs of every SeaweedFS consumer failed on a clearnet pick with
ERR_SSL_UNRECOGNIZED_NAME_ALERT on filer.<domain>.
lookup('network_served', <network>, *role_ids) answers whether the node's
resolved NETWORK_MODE serves a network and every named role is deployed.
The gates use it instead of restating the mode resolution and the provider
check, so the frontend now drops only on a tor-only node. is_explicit_truth
registers the lookup in GATED_LOOKUPS so the static closure still reads
those flags as dynamic, and the dynamic-flag and universal-logout checks ask
is_explicit_truth instead of matching "in group_names" themselves.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Some servers build every absolute URL from one site URL, so a second
network cannot be answered under its own origin. reachability.handover
keeps the siblings but turns each one outside the canonical network into a
302 to the same path on the canonical URL (basic and ws_generic vhosts).
The redirect target comes from the canonical domain, not the application
id, because lookup('tls', application_id) follows the network of the vhost
being rendered and would point the sibling at itself.
The flag also drops Onion-Location for the role (Tor Browser would bounce
between the two), skips those siblings in the CSP crawl, expects 302 from
the webserver health check and keeps Playwright on the canonical domain.
The key needs a nocheck reason like single_mode.
Mattermost opts in: it builds every OAuth callback from its one SiteURL, so
an onion login came back on the clearnet callback and failed with 400.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Since 778c18fb23 the carrier key holds the tokens file's mtime and size.
sys-token-store changed that stat on every inclusion (00_core.yml
touched the file) and on every persisted token, and nothing re-parked a
carrier afterwards. Every later task's fresh worker then rendered the
applications SPOT cold, 30 to 80 s each. On main fea6f31d9f (run
38812964308) seven heavy jobs logged 194 to 287 such tasks and ran out
of the 20400 s budget; the same jobs before 778c18fb23 logged 65 to 83,
all of them real waits.
- 00_core.yml touches the store with modification_time and access_time
preserved, so it only creates a missing file. In swarm the controller
sees the store over NFS with attribute caching, and the touch alone
made the second pass go cold without any write.
- write.yml re-parks the carrier when the persist or the controller
mirror changed the store. Both run once, so every host that ran the
include gets the same verdict.
- tasks/utils/setup/role.yml re-parks after every role on every host,
so hosts that did not run the writing role recover too. Where the
key still matches this serves the carried payload; measured 0.1 s per
role in a local compose deploy.
- The tokens file enters the key as the SHA-256 of its content,
memoised per inode, mtime and size, so a touch no longer changes the
key. A stat that still matches serves the memo (1.67 us against
1.41 us for the old stat signature), and a stale stat keeps key and
YAML cache on the same old content instead of pairing new content
with an old key.
- applications_carrier opens the tokens file before it computes the
key, so on the NFS-mounted controller the parked key reflects the
revalidated attributes rather than a cached stat that changes a
minute later.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On the Fedora, CentOS Stream and Manjaro workspace jobs install-lint
ran `puppeteer browsers install chrome-headless-shell --install-deps`
as root, and puppeteer rejected it with "apt-get does not seem to be
available". The installer only warned ("optional lint tool"), so
install-lint passed while mmdc had no browser.
--install-deps is now passed only where apt-get exists. Everywhere else
the libraries `ldd` reports missing are installed by soname: dnf and
yum through the RPM provide `<soname>()(64bit)`, pacman through the
first package `pacman -F usr/lib/<soname>` names (libgbm.so.1 is owned
by both mesa and mesa-amber, which conflict). browser_installed() also
requires every library to resolve, so the install-lint stamp stays
stale until the browser can actually load.
A failed npm install, a missing mmdc, a failed or timed-out browser
download and unresolved libraries now raise instead of warning.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The Manjaro workspace job failed test-unit-ruby with "cannot load such
file -- minitest". Debian vendors minitest in libruby, but Arch ships it
as ruby-minitest and Fedora and CentOS Stream as rubygem-minitest, none
of which ruby pulls in. ensure_ruby_present now probes
`require "minitest"` and installs the distro package when the probe
fails, then fails loudly if minitest is still missing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
No conflicts across 121 incoming commits. The one file both sides changed is
tasks/stages/01_constructor.yml: upstream parks an applications carrier for
the swarm bootstrap roles after the add_host tasks, and keys the carrier on
the tokens file as well. This branch's mesh bootstrap opens and drops its own
block-scoped carrier before that point, so both keep working as written.
Upstream also brings ruff 0.17.0, distro-derived env keys, a scoped act outer
teardown, the stalwart relay rule, the onion apex redirect and image and
version updates; ruff 0.17 reformats nothing on this branch.
One oauth2-proxy applies one cookie-secure flag to every host it answers, so a secure clearnet instance drops the CSRF cookie of its plaintext onion sibling and the login loops. proxy_instances now yields one instance per network with its own compose service, sso or sso_onion host port and config file; each config takes cookie_secure, redirect_url and cookie_domains from its own network, and every vhost routes /oauth2/ to the instance of its network. The 29 gated roles gain a sso_onion port, and Bluesky heals the bind mount of every instance.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A process that renders network-bound URLs once, such as oauth2-proxy or the dashboard, can serve only one network, so a multi-network role runs one per network. network_instances lists those networks with a label, the first one unlabelled so a single-network role keeps its names, and labelled and labelled_file suffix service and file names with it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The onion pick only bypassed the SSO issuer, so a clearnet CDN, asset or dashboard host an onion page loads went through Tor, which the lab network cannot exit. browser_proxy_bypass derives the list from the deployed applications' domains plus the issuer; it is empty on a clearnet pick and on a tor-only node. Specs that passed their own proxy to newContext dropped the bypass and now inherit use.proxy.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- browser_revision() reads revisions.js from the puppeteer-core that
node resolves for mmdc's puppeteer: puppeteer/node_modules first, then
the node_modules directories above it. When mermaid-cli pins another
puppeteer-core, npm nests puppeteer's own copy, and the sibling held a
different revision.
- A failed browser install removes only the directories of the pinned
revision before the retry. Removing the whole chrome-headless-shell
cache also deleted browsers that other puppeteer installs share.
Addresses the CodeRabbit review on #958.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Only the node that runs svc-net-tor had it in group_names, so every other swarm node resolved no node onion, served no onion siblings and scheduled no onion apex redirect. tor_serves_host also accepts a provider anywhere in a cluster of more than one svc-swarm-node; a single compose host keeps the old answer.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
install-lint provisioned chrome-headless-shell with `npx --yes
puppeteer browsers install`, which runs the latest puppeteer from the
registry. mmdc runs with the puppeteer npm installed next to it as a
peer dependency when mmdc itself was installed, and looks for exactly
the revision that puppeteer-core pins. Once puppeteer 25.14.0 was out,
npx installed 155.0.8059.79 while mmdc with puppeteer-core 25.13.0
wanted 155.0.8059.39, and lint-mermaid failed on every file. Because
browser_installed() accepted any chrome-headless-shell, a stack with
the wrong revision never repaired itself.
The browser is now installed through the puppeteer CLI next to mmdc,
which installs its puppeteer-core's revision, and only that revision,
read from puppeteer-core's revisions.js, counts as installed.
Closes#957
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The constructor parked no applications carrier between the merge block
and the token store, so every lookup in the svc-swarm-node, svc-storage,
svc-registry and svc-virt bootstrap rendered the SPOT cold, 25 to 30 s
each on a loaded host.
A carrier there was unsafe: in swarm, svc-storage-nfs-server mounts the
NFS export over the controller's DIR_VAR_LIB, so the tokens file the
render reads can appear in the middle of that section, and the cache key
did not notice. merged_applications_cache_key now includes the tokens
file's path, mtime and size, as the users cache key already did, so a
carrier rejects itself once the tokens change. A carrier parked after
the add_host tasks then serves the bootstrap roles. It is parked only in
swarm mode: in compose the section runs in about a second, so a carrier
there would add a render without saving one.
In a swarm drill of web-app-prometheus up to the known NFS probe-write
failure, "Resolve the NFS server backend flavor" went from 29.6 s to
0.07 s, "Probe-mount the NFS export" from 26.4 s to 0.4 s, "Mount
cluster-shared infinito state on controller" from 23.7 s to 0.5 s and
"Ensure Docker-In-Docker DNS works" from 30.0 s to 3.6 s; the run to
that point took 768 s instead of 1735 s.
Closes#925
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ruff 0.17.0 adds `datetime` -> `dt` to the default aliases of
flake8-import-conventions, and the project selects ICN, so lint-python
failed with ICN001 in seven files.
The files now import `datetime as dt`. ruff's unsafe fix also renamed
two local names in utils/github/playwright_summary.py into the alias:
the `_to_iso_z` parameter and the parsed value in
`_parse_iso_timestamp`. Both are now called `moment`; with the bare fix,
nine tests of test_playwright_summary.py failed with UnboundLocalError
and AttributeError.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
No conflicts. The 14 incoming commits fix the n8n gateway probe trigger under
n8n 2, let stalwart address bans expire, keep the Docker apt key when its
refresh is refused, leave eol roles out of image discovery, stop the release
backfill below the oldest buildable tag, and provision the mermaid browser
only in install-lint. None of them touches the mesh, swarm, NFS, network or
workflow files this branch changes.
