Stalwart, Mailu and LDAP provision every bot and account holder of the whole users catalogue, but the generator pinned only the users of the resolved roles. The bots of undeployed roles (no-reply, blackhole, bounce, newsletter, helpdesk, contact) fell back to strong_password, got a new secret on every deploy, rewrote the token store six times per run and left email.password consumers on a stale value. The generator now also pins every catalogue user holding an account or the bot role unless a role states its password outright, and reads policies independent of declaration order; existing pins are never overwritten.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An inventory group web-app-cst-<name> with applications.<id>.custom: true and
a pinned source {url, version, path} is pulled into roles/ before the play,
read raw (Jinja, Ansible tags and credential URLs refused), placed through a
staging directory under an exclusive lock, and owned by a marker that names
its own directory, so a pull or roles-clean never replaces or removes a
directory it did not place and a failed pull leaves roles/ as it was.
Directory inventories, host lists and non-UTF-8 files behave as Ansible
reads them; generated test deploys pass --skip-pull. make roles-pull,
roles-clean and roles list, INFINITO_EXTRA_MOUNTS for local sources, the
web-app-cst category, the reserved prefix lint and a central
repository_files() that also yields the gitignored custom roles let every
lint scan them like in-tree roles. Supersedes the uncommitted 045 External
Roles draft and reuses its fetch, sync and mount design.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The validator cut every !vault block with a regex that ran up to the next line at column 0, so the keys after a vaulted password vanished, a whole user among them. utils.cache.yaml has parsed the !vault tag itself since c4556fe72d, so the loader now hands the file to it unchanged and a vaulted value loads as its ciphertext. An inventory-only user that brings its own username no longer warns as unknown.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The Keycloak stage on echoserver ran ten minutes before sys-svc-dns stopped
it with "API.cloudflare.api_token must be set": the inventory still kept the
token under the flat name CLOUDFLARE_API_TOKEN, which nothing reads any
more. The validator that runs before every deploy now catches that class.
- api.py reports a host that selects DNS_PROVIDER: cloudflare without
API.cloudflare.api_token, with group_vars/all merged under the host_vars
- variables.py reports every top-level key of host_vars and group_vars that
core does not read: not declared in group_vars or a role's defaults or
vars, not tested with "is defined" or "| default(", not read by name in a
plugin, and not a SYSTEM_EMAIL_<KEY> the email lookup builds; a flat
credential name such as CLOUDFLARE_API_TOKEN names its API.<provider>.<key>
path; ansible_* keys are exempt
A key that is read in a way the scan cannot see takes
"# nocheck: inventory-variable", documented in suppression.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`MODE_CLEANUP` says what to tidy on the deployed system: prune unused Docker
resources, clear Mastodon caches, switch an LDAP user list from `present` to
`exact`. Eight roles read it that way. The dedicated runner read the same flag
and ran `make clean` in the control repository instead, which is
`git clean -fdX` over every ignored path there.
`inventories/production/` is one of those paths. The published host install
guide provisions into it and then deploys against `$INVENTORY/.password`, so
the deploy deleted the vault password it was about to use:
Removing inventories/production/
[ERROR] The vault password file .../inventories/production/.password was
not found
Reproduced in CI on two unrelated rows, arm64/zfs/debian and
amd64/btrfs/centos, so it is neither architecture nor filesystem specific.
The coupling was a name collision, not a decision. 7e5990aa16 generated the
MODE_* flags from `01_modes.yml` and removed the legacy skip flags, and the
repository clean - which had had a flag of its own - was attached to the
generated `MODE_CLEANUP` because one now existed by that name. No other deploy
path does this; `container`, `development` and `swarm` never call `make` at
all.
`make clean` keeps its own entry points, `make setup-clean`, `make clean-sudo`
and `infinito --git-clean`, where a caller asks for it deliberately.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
apply_tor_node_onion rewrote host_vars through utils.cache.yaml, whose
!vault constructor returns a plain string and, depending on import order,
wins over the VaultScalar constructor of utils.handler.vault. Provision
calls it after generating credentials, so with svc-net-tor in the include
every credential lost its tag and Ansible rendered the raw ciphertext:
redis got --requirepass '$ANSIBLE_VAULT...', compose cut env values to
";1.1;AES256" and the openldap ldapsm bind failed with result 49.
Write through the round-trip helpers like every other writer in the
package. test_preserves_existing_vault_credentials passes again.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Without a swarm manager the mesh planner only built full meshes, whose
handshake gate waits for every peer, so laptops behind NAT could not join.
An inventory may now name a hub in networks.hub: the planner builds every
declared mesh around it, the hub forwards between its spokes, a spoke marked
networks.roaming gets no endpoint in the hub's peer list and dials the hub
with PersistentKeepalive, and the hub's gate skips roaming spokes while a
roaming spoke gates only on the hub. The mesh CLI takes the hub's public
address per host (--endpoint) and a vault password per host
(--host-vault-password-file). Full meshes and the swarm manager hub plan
exactly as before; NETWORK_MESH_POOL keeps the operator's subnet.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ruamel folds every scalar and flow collection past 80 columns, so each
run of the provision, credentials or mesh CLI broke SSH keys and alias
lists in host_vars over several lines. round_trip_yaml() now sets an
unlimited width, and the credentials CLI and the services disabler use it
instead of building their own handler.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Declare `__all__` in every package `__init__.py` whose `PROJECT_ROOT`
is imported by its modules (163 packages), so the constant counts as
exported; list every public name where the package defines more.
- Empty the 15 package `__init__.py` files whose `PROJECT_ROOT` no module
imports.
- Declare the public API of `utils.github.variant.axes` in `__all__`,
re-exports included, instead of silencing them with `noqa: F401`; the
list matches the one `feature/store-i18n` already carries.
- Use `import unittest.mock as mock` in `test_applications.py`, as the
rest of the suite does.
- Spell `_NON_PRINTABLE` in `utils/yaml_bootstrap.py` as the equivalent
BMP-only positive class; it matches exactly the same 2111 code points
as the old negated class across U+0000..U+10FFFF.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
baudolo 7.1.x writes volume generations with `rsync -aHAX`, so the
producing host keeps `user.*` extended attributes (OpenCloud keeps its
storage metadata in `user.oc.*`). Every later copy dropped them again,
because none of the other rsync calls passed `-X` (nor `-A`/`-H`):
- NFS export snapshots (svc-bkp-nfs-2-local `script.sh`),
- the backup host's pull (svc-bkp-remote-2-local `pull_specific_host.py`),
- the device sync (svc-bkp-local-2-device `script.py`),
- every recover into a live target (`utils/recovery/base.py`, used by the
volume, NFS and device recover scripts),
- the `recover full` and `recover remote` tree pulls.
A disaster recovery through the documented chain therefore restored
OpenCloud without `user.oc.*`, silently: the copies succeeded and the
dirval stamp does not cover attributes.
All of these now run `rsync -aHAX` (`-aHAXP` for the device sync). A
target filesystem without xattr support makes rsync exit non-zero as soon
as a source file carries attributes, so the hop fails loudly instead of
dropping them; there is no per-device opt-out. The single-image pull in
`recover remote` keeps `-a`. The provider's ssh-wrapper needs no change:
its glob accepts any server flags and its character guard allows them.
Two tests write a `user.oc.id` attribute and assert it survives the NFS
snapshot and the recover; with the old `-a` both fail with
`OSError: [Errno 61] No data available`.
Closes#1021
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When the IPAM config of a compose project's network changes (#999 added
`ip_range`), compose recreates the network on the next `up`. With a
foreign endpoint attached, as every worktree stack is on the primary's
network, compose first stops the project's own containers, then fails
with `error while removing network: ... has active endpoints`, and a
repeated `up` or `up --force-recreate` does not recover. On 2026-10-19
this took the shared registry/package cache and coredns down for every
local session.
`make compose-network-recreate` (alias `i8conere`) runs the sequence that
recovered it: detach every running container of another project from the
network, `docker compose down --remove-orphans` without `-v` so the
caches keep their volumes, the normal `up`, then attach the detached
containers again, which draw addresses from the dynamic pool. If `up`
fails, the still-detached containers are listed instead of reattached.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The workspace jobs pulled every image anonymously from Docker Hub and
two of them failed on its rate limit in run 39059128678 (#990): the
fedora swarm drill on busybox:1.38 on all three nodes, the centos
compose drill while building tor-smtp_custom. The regular deploy jobs
pull through the GHCR mirror, but the mirror was tied to being a real
GitHub runner: Profile.image_mirror_enabled() returned
runs_on_github(), and the swarm routine generated mirrors.yml only when
cli.meta.runtime reported github, which is act inside make
swarm-zombie.
The mirror is now its own capability, INFINITO_GHCR_MIRROR (next to
INFINITO_GHCR_MIRROR_PREFIX), written as a run fact like
INFINITO_CACHE_STACK: true on a real GitHub runner,
false everywhere else, and a non-empty exported value wins for the run.
Profile.image_mirror_enabled() reads the declaration and keeps
runs_on_github() as the fallback, and 02_provision_inventory.sh gates
on the same value instead of the runtime name. make swarm-zombie
forwards it into act through ACT_ENV, since act ignores the host env.
call-test-workspace.yml declares INFINITO_GHCR_MIRROR=true and passes
INFINITO_GHCR_MIRROR_PREFIX into the workspace container. Nothing else
of the workspace changes: is_ci, runs_on_github, the cache stack and
the docker root stay local-like. The step that loads the rescue paths
now removes the .env it generated on the runner, because on debian
rows load.sh sourced it inside the container and the run inherited
INFINITO_RUNNING_ON_GITHUB=true.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The swarm output is byte-identical: the dev layout and the three-mesh example,
each with and without a controller, produced the same host_vars, wg.conf and
stdout before and after, with deterministic keys.
Under a svc-swarm-manager host the declarations stay the hub's own. Without
one, every host may declare the meshes, and hosts declaring different ones are
refused rather than merged, because a member would otherwise render a mesh
its peers do not know.
With no hub every joined host is a member, addressed from the same offsets a
spoke takes, and each host is primary on the first declared mesh it joined.
A topology may carry no hub. Its meshes then peer every member with every
other one, which is what a Compose deployment of several hosts needs: there
is no manager to route through.
No conflicts across 121 incoming commits. The one file both sides changed is
tasks/stages/01_constructor.yml: upstream parks an applications carrier for
the swarm bootstrap roles after the add_host tasks, and keys the carrier on
the tokens file as well. This branch's mesh bootstrap opens and drops its own
block-scoped carrier before that point, so both keep working as written.
Upstream also brings ruff 0.17.0, distro-derived env keys, a scoped act outer
teardown, the stalwart relay rule, the onion apex redirect and image and
version updates; ruff 0.17 reformats nothing on this branch.
ruff 0.17.0 adds `datetime` -> `dt` to the default aliases of
flake8-import-conventions, and the project selects ICN, so lint-python
failed with ICN001 in seven files.
The files now import `datetime as dt`. ruff's unsafe fix also renamed
two local names in utils/github/playwright_summary.py into the alias:
the `_to_iso_z` parameter and the parsed value in
`_parse_iso_timestamp`. Both are now called `moment`; with the bare fix,
nine tests of test_playwright_summary.py failed with UnboundLocalError
and AttributeError.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The CLI reads the declarations and memberships from the host_vars it writes
to. --contract-file and --application-id go with them: there is no contract
left to write, and the application block is fixed by the mesh reader.
Each host's state moves from a map keyed by mesh name to a list under
svc-net-wireguard.networks.interfaces. The inventory validator does not walk
lists, so a deployment's own mesh names need no generated contract, and
every reader goes through utils.networks.mesh.
The declarations are the hub's svc-net-wireguard.networks.meshes, and a host
joins every mesh its roles list in their own networks.meshes, so two hosts
of one role may sit in different meshes. The hub is the svc-swarm-manager
host. A deployment narrows the pool in its own group_vars, otherwise the
platform's NETWORK_MESH_POOL applies. A mesh a role names but nobody
declared is refused rather than dropped.
members_of, primary_meshes and plan_mesh take the Topology instead of raw
inventory groups. The spokes come out deduplicated, sorted and without the
hub, as before.
MeshSpec loses hub_group, spoke_groups and spoke_group_prefixes: membership
is no longer part of a declaration but resolved from the inventory into a
Topology of one hub and the spokes of every mesh.
--controller-mesh defaulted to 'swarm', a name only the development
declarations use; any other deployment left the controller off the mesh. It
now defaults to the first declared mesh, an undeclared name fails instead of
being ignored, and the primary of every host is resolved over all declared
meshes before any is written.
Every spoke routed the whole pool through every mesh it belonged to. For a
spoke of two meshes that is two routes for one prefix: wg-quick refuses the
second with 'RTNETLINK answers: File exists' and deletes the interface. The
three-mesh example hits it on the swarm lab, where the NFS server is a spoke
of beta and of gamma.
Only the primary mesh routes the pool now; the others route their own subnet,
which the interface address already covers. Each entry records whether it is
the host's primary.
A host's primary mesh is the first declared mesh it belongs to. It is the one
interface a spoke of several meshes may route the pool through, and the
address the other planes reach the host on.
The guarantees were asserted against the declared swarm and data pair,
so they proved two meshes work rather than that the planner works. A
deployment declares its own, and the review asks for any number of them with
the nodes arranged however the client wants.
The same invariants now run over generated topologies: one to eight meshes,
three arrangements each, hosts dealt out so some belong to several meshes and
some to none. 2247 subtests, four seconds, and the planner needed no change to
pass them.
The GHCR mirror covered role images only. The images the runner pulls on
its own went straight to Docker Hub: coredns from compose.yml, the cache
stack in compose/cache.override.yml and alpine:latest, the init image of
scripts/docker/cache/package-frontend/certs.sh. With the anonymous
githubactions quota spent, the guide step of Postmarks#0 (main run
37954203718) and Shopware#2 (#872) died on "toomanyrequests" while pulling
alpine:latest, after coredns had hit the same limit.
iter_runner_images (utils/docker/image/discovery.py) finds those images,
and the mirror sync and the fork-PR wait now cover them next to the role
images. On a real GitHub runner compose_file_args layers a generated
build/compose/runner-mirror.override.yml that points only the services of
the layered compose files at their mirror, and the certificate generator
gets the mirrored init image. Local runs and act are unchanged, and the
coredns literal stays in compose.yml for Dependabot. test-dns now waits for
images-mirror-missing like the deploy and workspace jobs, since it brings up
the stack too.
Verified with docker compose config on a simulated GitHub runner: the
service set is unchanged and only the runner images point at
ghcr.io/infinito-nexus/core/mirror/docker.io/..., and an existing role mirror
under that scheme pulls anonymously.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The arm existed to answer one question: how much of the meshed arm's
cost is the connection plugin and how much is the tunnel. Runs 37592061960
(docker), 37753506661 (SSH on the underlay) and 37621556682 (SSH over
WireGuard) answered it on web-app-gitea#0, debian/ext4:
pass 1 5964s / 8511s / 8832s
pass 2 5319s / 7472s / 7705s
Leaving ansible_connection: docker costs +4700s, the tunnel +554s - 89.5%
against 10.5%. The answer is recorded; the scaffolding does not need to ship.
Reverts d792519725, 1b28f87b37, 89b8a8c8d1, 0470491e1a, 529c5ace43,
dd3d10a9a2, c20dd0e5f5, 458d8956e1, 63ade1e03b, d281555945 and 4adfb6e3d2.
95cfc8b7b7 is already superseded by bbfd82f89, which sets the profiling limit
upstream.
keep_address left ansible_host alone, and the lab never sets it: the
docker connection uses inventory_hostname as the container name. Switching to
SSH without writing an address made ssh resolve 'web-app-gitea-swarm-mgr-01'
through DNS, and run 37617424998 died UNREACHABLE on all three nodes in 13
minutes.
switch_to_mesh now takes the addresses to dial. A member missing from the
mapping raises instead of falling back to its mesh address, because that
fallback would have produced the meshed arm under the other arm's name - a
wrong number rather than a failed run.
keep_address switches the connection, the user, the key and the ssh
args exactly as the mesh switch does, and leaves ansible_host where it was.
Against the other two transports that isolates the cost of leaving the local
connection from the cost of the tunnel.
The prose explaining why offsets are reused and how the hub keeps index
zero is recorded in requirement 037. A reader of plan.py needs the signature,
not the history.
kevinveenbirkenbachandClaude Opus 5.5
evangelostsak