Both creates: guards pointed at paths that could never match, so the
bootstrap and setup:install shells re-ran on every deploy.
The first guard went through docker_volume_path, which hardcodes
/var/lib/docker/volumes/<name>/_data and misses any relocated docker
data-root. The second was worse: path_join over the bare volume name
produced a relative path, resolved against the Ansible working
directory, which matched nowhere at all.
Both now derive from the mountpoint the daemon reports. The stack-host
gate moves to a single include, mirroring web-app-akaunting.
With its last caller converted, docker_volume_path is deleted. It could
not be repaired: a Jinja filter runs on the controller and cannot know a
remote host's data-root.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Since the swarm-NFS migration two chains copied the same bytes: svc-bkp-nfs-2-local
reads the export from the NFS server's own filesystem, while baudolo on the manager
copied the same files again THROUGH the NFS mount. That was a migration leftover -
svc-bkp-nfs-2-local arrived in adb8bcf69 (2026-07-10) and nobody taught the older role
to step back - and the role README already forbade it as a rule nobody enforced. In the
DR drill the duplicate costs three times over: the pulled tree, the device copy and the
restore root.
swarm_nfs_backed() in utils/storage/nfs.py is now the single answer to "does the swarm
NFS rewrite back this volume". plugins/filter/compose_volumes.py (which performs the
rewrite) and this role's backup_volume lookup (which feeds baudolo's
--volumes-no-backup-required) both call it, so the two cannot drift. On the current tree
that excludes 83 of 158 type: volume entries.
Four conditions keep the exclusion from ever dropping a volume nobody else captures:
* a manager-pinned role's volumes are not on the export (web-app-seaweedfs holds
Nextcloud's object store, deliberately node-local per its meta/services.yml)
* a role forcing another mode via compose_mode_force is not on the export either.
That override is per role (roles/sys-svc-compose/defaults/main.yml:3), so it is
resolved per role inside the predicate from roles/<id>/vars/main.yml, not read once
from ambient scope. web-app-bigbluebutton pins compose statically;
web-app-matrix computes it, and a value the predicate cannot resolve counts as
forcing - unknown mode keeps the volume backed up
* an entry opting out with nfs: false stays node-local
* the exclusion applies at all only when the svc-bkp-nfs-2-local inventory group is
populated. That role enters a play by group membership alone
(tasks/groups/svc-bkp-roles.yml:19-22) and svc-storage-nfs-server ships a variant
with nfs_backup: {enabled: false}, so without the gate an export whose repository
nobody deployed would lose 83 volumes from both capture paths
Every condition fails toward the old double capture: wasted space, never lost data.
Compose mode is untouched - the exclusion cannot fire without swarm and an nfs backend.
DEPLOYMENT_MODE and storage are templated through the lookup's templar, because
group_vars/all/18_swarm.yml:2 defines the mode as a Jinja expression; comparing the raw
source string would silently disable the whole exclusion.
Twelve new tests pin the contract, including the two failure modes that would be
invisible in production: an untemplated mode, and an exclusion without an export
repository. Both were verified by mutation - removing either guard makes its test fail.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two places decided independently what the pre-created <entity> network is called
inside the compose project. render.py emits the key; create.yml asserted it was
always `default`. They diverged in July and nobody noticed, because compose
5.3.1 resolved networks by name and never read the claim.
Compose 5.4.0, published 2026-08-03T14:39:12Z, resolves them by label. For
web-app-seaweedfs - the one deployed role that provides its own shared_net and
therefore renders an unnamed `default` - the mislabelled `seaweedfs` network is
adopted as the project default, `seaweedfs_default` is never created, and the
containers fail to start against it. That failed web-app-nextcloud 0 and 3 in
run 30830377785.
_shared_network_key() is now the single derivation. render_compose_networks
takes its branch from it, so the rendering decision IS the key decision;
shared_network_compose_key() exports the same value, the shared_network_key
lookup hands it to Ansible, and create.yml stamps it: `seaweedfs` for a
self-provider, `default` for everyone else. Both true, both from the source that
renders the file.
Verified on one host with only the compose binary swapped:
5.4.0 network=default -> FAIL, network <project>_default not found
5.4.0 network=<entity> -> OK
5.3.1 network=default -> OK
5.3.1 network=<entity> -> OK
so this is not a fix that trades one version for the other.
test_key_matches_the_key_the_renderer_emits renders the block and asserts the
reported key appears in it, for both role shapes and both modes. The drift that
sat unnoticed for a month now breaks a test.
The move of the attachment computation into utils/networks/attachments.py rides
along because render.py would otherwise be broken between two commits: it is a
pure relocation, re-exported so no importer changes, and it brings the file from
352 to 243 lines.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
compose_networks, container_networks and compose_external_networks each rebuilt
the same Ansible-side context before calling their renderer: resolve the
variables, settle the deployment mode, load applications, build the service
registry, and close over the config and database lookups. Around forty lines,
three times, each with its own copy of _resolve_var. A change to how the
registry or the closures are obtained could reach one lookup and miss another.
build_context() returns that context once. honour_mode_force carries the only
real difference between the callers: compose_external_networks never consulted
compose_mode_force, and still does not.
The lookups keep only what is theirs - term validation, their own arguments,
their renderer call - and drop from 94/90/87 lines to 59/54/53.
No assertion changed in the tests; only the mock targets moved with the seam,
from the plugin modules to utils.networks.lookup_context. Twelve legacy comments
in those test files lost their grandfathering when the files entered the lint's
diff scope: two became docstrings naming the call contract, the rest were
narration and are gone.
Behaviour-preserving: the same 13 lookup tests pass before and after.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The pre-pull task set `async` unconditionally while gating only `poll`. With
async disabled, `poll` became `omit` and ansible fell back to its built-in 15s
poll while `async` still routed the task through async_wrapper: ansible
collected the job itself and then ran `async_status mode=cleanup`, deleting the
job file that tasks/utils/async/reap.yml expects to read. The reap reported a
successful `ollama pull` as a failed async job, failing web-app-nextcloud 2 in
run 30781723102.
Both keys are now switched off together, matching all six peer call sites. On
the resulting synchronous path the reap's `async_reap_tolerate` list no longer
applies, so the tolerated "up to date" answer is carved out in `failed_when`
instead. It keys on `failed` rather than `rc`, so a module-level failure that
carries no rc is not swallowed.
async_failures._which() now honours loop_control.loop_var: the fired result
records the chosen name under `ansible_loop_var`, so a batch failure names
`smollm2:135m` instead of the bare job id.
A second lint case asserts that async and poll carry `omit` together - the rule
this call site was the only one of seven to break.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
container_healthcheck resolves application_id from the play vars
instead of taking it as a positional term. Vars handed over by
include_role arrive unrendered, so under sys-stk-full the lookup got
the literal '{{ sys_stk_full_application_id }}', resolved no app and
refused to render the dashboard compose file.
Template such values before use. container_hostname had the same
defect: it is a Jinja string in every role's vars and was baked
verbatim into the curl Host header. Raise when a value survives
templating unrendered, since ansible-core returns untrusted templates
as-is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Seven include templates under sys-svc-container decided how a container was
probed, and they read their inputs from template-local `{% set %}` scope.
That scope leaks: a value set for one service silently reached every service
below it, which is how taiga's front and gateway ended up probing a port
declared for a different service, and how bluesky's frontend probed a PDS
endpoint.
Probes now live in utils/docker/healthcheck.py, one class per flavor behind a
PROBES registry, and each service declares its own:
healthcheck:
flavor: curl
path: health/ready
start_period: 20m
`port_key` picks another entry under ports.internal for the services that
expose their probe elsewhere (keycloak on management, nextcloud talk on
signaling). Without a flavor the entry must carry an explicit test argv, so a
missing declaration fails loudly instead of rendering a probe against nothing.
Rendering HEAD's templates against the new output confirms 49 of 56 call
sites are byte-identical; the remaining seven are covered by an Ansible run.
Three probes change on purpose: erpnext's backend gains a check it never had,
bluesky's frontend drops the leaked PDS path, and msmtp_curl moves from a
hardcoded localhost:80 to the service's own port on 127.0.0.1 like every
other flavor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The repo's async convention is fire and forget: ASYNC_POLL is 0 whenever
ASYNC_ENABLED is true, and the swarm test lane forces it true through
extra vars. A task dispatched that way returns a stub, so a changed_when
or failed_when reading its own register evaluates against nothing and can
never report changed and never fail. Several tasks carried a
"not ASYNC_ENABLED and" prefix in front of those conditions, added to
avoid touching undefined rc and stdout, which turned the guard off
entirely rather than adapting it.
That is how the nextcloud plugin configuration stayed broken unseen: the
batch reported ok after 0.8s while its delegated commands kept running,
the next task read the half written configuration, and no failure in any
of them could surface.
Every async in the repo was decided individually rather than swept. Where
nothing consumes the result, the async stays and a one line rationale now
records what does not read it. Where a later task, a handler or a browser
does consume it, either the async goes or the result is collected.
Collecting is what the new reaper does. It waits for a whole batch, then
reports every broken job at once instead of aborting on the first, so one
run shows whether a single item or the whole set is broken. Deciding what
counts as broken lives in a filter plugin, because the distinction is
subtle enough to deserve tests: a module reports failure in msg, a command
in rc and stdout, an idempotent API answers "already exists" and means
success, and a job that cannot be read at all comes back marked finished
with no outcome, which would otherwise read as green.
Timing cost is roughly one to two minutes on a deploy measured in hours.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The consumer count came from roles_with_service, which builds dashboard card
targets and therefore requires a single canonical domain. web-app-matrix
declares domains.canonical as a mapping (synapse, element) and was dropped;
in a deployment where it was the only consumer the count reached zero, the
server started with -volume.max=7, the unnamed default collection took all
seven slots on first write, and every S3 PUT for the matrix collection failed
with "No writable volumes and no free volumes left". Synapse answered 500 on
media upload and no avatar ever reached the bucket. web-app-openwebui carries
the same mapping shape and the same latent failure.
Consumers are now enumerated by the binding itself: a role is a consumer when
its merged config binds the engine and marks it shared, whatever the role is
called and however many domains it serves. The S3 identity file and the volume
budget read that one list, so the identities granted write access and the
slots budgeted for their collections cannot drift apart.
Selection reads the merged applications config rather than meta/services.yml,
because an operator may bind a store through the applications inventory
variable alone; that path is covered by a test so a later optimisation cannot
quietly reintroduce a naming or declaration assumption.
f956de889 made the slot ceiling generous, which removed the accidental brake the old autosizing had provided: while the ceiling was derived from free disk space, growth stopped long before the disk filled. With a fixed ceiling the only thing between growth and ENOSPC is the free-space guard, and its stock 1% is 300 MB on a 30 GB data root - less than the 1024 MB unit SeaweedFS allocates in, so a single growing volume can outrun it.
The floor is now expressed as whole allocation units rather than a percentage, which is meaningless without knowing the disk size. Two units: one for the growth in flight, one for concurrent writers. The size limit is stated alongside it so both come from one constant instead of the floor deriving from a value the image injects - verified that our flag wins, since -master.volumeSizeLimitMB=512 makes the master report 512 MB.
-volume.minFreeSpace replaces the deprecated -volume.minFreeSpacePercent, per the image's own help text. The rendered command was run against chrislusf/seaweedfs:4.39: master, volume server and filer all start.
Left open deliberately: web-app-seaweedfs declares min_storage: 304MB, which cannot hold with a 2 GiB floor. That value feeds the CI storage filter in scripts/meta/resolve/apps.sh, where 60GB is the budget, so nothing is dropped today - but the declaration understates what the role now needs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
a3fcd3e7f stopped the entrypoint from autosizing the ceiling out of free disk space, but replaced it with a flat 64 that I adopted without deriving. Each bucket is a SeaweedFS collection and the master grows one in batches of seven - 'create 7 volume' in the artifact, copy_1 = 7 in the image's own master scaffold - so 64 covers nine collections. Twenty-five roles declare the service, and the tenth consumer would have hit 'No writable volumes and no free volumes left' again, just later and without the disk-space link.
The ceiling now comes from volume_slots(): one grow batch per bucket plus one for the default collection, with the consumer count read from roles_with_service. Note what that count is: the consumer gate gets whether seaweedfs is in group_names, not whether the consumer itself is installed, so the value is derived from the repository rather than from the host. It grows with the codebase, which is what the ceiling has to keep up with.
The collection count is a required parameter with no default. A silent fallback would be worse here than a loud failure: without the flag the stock entrypoint injects -volume.max=0 and the autosizing trap is back.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role passed no -volume.max, so the stock entrypoint injects -volume.max=0, which means 'derive the ceiling from free disk space'. That derivation charges every non-full volume its full volumeSizeLimit: the master's default grow batch creates seven 1 GB volumes on the first write, reserving ~7.0 GB, and the manager's docker data root had 6.8 GB free. unclaimedSpaces goes non-positive, maxVolumeCount collapses to volumeCount, and the node reports zero free slots.
Those seven volumes belong to the default collection, so they can never serve collection 'nextcloud'. The master says it plainly - 'topo failed to pick 1 from 0 node candidates', 'create 7 volume, created 0' - and every S3 PUT returns 500 while seven empty volumes sit on disk. The volume server never left the topology: RestartCount 0, one unbroken log stream across the window.
That 500 is what starts the web-app-nextcloud 0 chain: occ maintenance:install fails eleven times, leaves a half-installed instance, and slot 1 then holds nextcloud-init-sync.lock for four minutes while every other replica dies on a blocking flock. Reproduced independently in variant 5 with collection 'peertube'; both compose jobs stayed green.
-volume.max makes isArgPassed true, so the entrypoint stops injecting the autosizing value. Slots become a fixed budget rather than a function of disk space; volumes stay sparse and unpreallocated, and -volume.minFreeSpacePercent still guards a genuinely full disk.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
_any_bridge_partner_deployed matched a bridge against deployed role ids by equality or '-<bridge>' suffix. That reaches a partner whose entity carries the name (matrix -> web-app-matrix) but never one registered under a service key: 'sso' is provided by web-app-keycloak and 'ldap' by svc-db-openldap, neither of which ends in the bridge name. Those five declarations therefore evaluated to false in every run, so their addon flags were false everywhere and skipUnlessAddonEnabled skipped the specs in every variant - silently, because a gate that never opens looks exactly like a variant that does not apply.
The lookup now resolves a bridge through build_service_registry_from_applications first and keeps the entity-suffix rule as the fallback, so both spellings work. Existing bridge tests are unaffected: their applications stub yields an empty registry and takes the old path.
The new lint asserts that every bridges: value names a service some meta/services.yml declares. It fails on a typo or an invented name while staying agnostic about which run deploys the service, which is the lookup's question at runtime. Verified both ways: no findings against the real service set, thirty against an empty one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A module that dies on an unhandled exception carries no msg: the AnsiballZ wrapper returns only failed and exception, whose event holds the sole account of what went wrong. The aggregation defaulted to the literal string package install failed and discarded it, so an AUR fetch that threw in run 30713666732 reported nothing at all and took a full investigation to place. Walk msg, exception.event.msg, exception, module_stderr, stderr instead, and tag every result with the module that produced it. This applies to apt, dnf, pacman and every other module the plugin drives, not only to the AUR.
The aur module fetches the RPC record and the snapshot tarball through bare open_url calls with no error handling, so one blip kills a deploy; distribution repositories need none of this because apt, dnf and pacman retry inside their own transport. Give the calls that reach an endpoint of their own a single shared policy. Only the AUR path is declared today; copr, ppa and build carry it against the day they are.
Split the result along its responsibilities so no file passes 250 lines: calls.py holds the vocabulary, sources.py one builder per acquisition path, plan.py only the dispatcher, outcome.py the reporting and facts.py the fact shapes. The action plugin is left with executing modules. Tests follow the code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`ruff`'s B905 wants every `zip()` to say whether it tolerates operands of
different length. Here it never sees one: both operands are the `packed`
form of an address from the same network, so both are four bytes for IPv4
and sixteen for IPv6. `strict=True` states that, and turns a silently
truncated - and therefore too permissive - regex into an error should the
invariant ever break.
The three other files carry `ruff format` reflows the same run produced.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Subnet arithmetic lived in Jinja expressions - octet slicing for a JGroups bind
matcher, a regex_replace for a gateway address. Neither is reachable from a
test, and neither says anything useful when it meets a shape it cannot handle.
utils/networks/address.py carries the computation without an Ansible dependency,
so tests/unit reaches it directly.
subnet_address_regex works for any prefix length, aligned or not. A prefix mask
leaves a contiguous run of low bits free, so per octet the addresses span exactly
network_address to broadcast_address, and the cross product of those four ranges
is the network. That makes the octet decomposition exact without a case analysis:
/24 yields 192\.168\.24\.\d+ and /23 yields 192\.168\.(24|25)\.\d+ from the same
three lines.
subnet_gateway returns the address Docker assigns as the gateway, and raises for
a network too small to hold one.
Both lookups follow compose_networks: no positional terms, application_id from
context, networks.local.subnet resolved internally, ValueError turned into an
AnsibleError naming the role and the reason. The lookup form rather than a filter
is deliberate - the caller states its intent and the plugin owns where the value
comes from, instead of repeating the config key at every call site.
Verified exhaustively rather than by sampling: over all 65536 addresses of a /16,
for eight subnets, regex match equals network membership - about 524000
comparisons. Boundaries checked for every prefix length /8 to /32 including the
addresses immediately outside, gateways checked against ipaddress for /8 to /31,
and a cross check that each network's gateway is matched by its own regex.
The regex uses \d+ for a fully free octet, so it is exact over valid IPv4
addresses rather than over arbitrary strings; JGroups only ever matches addresses
of local interfaces. The test module docstring records that boundary.
No consumer at this commit; the call sites follow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
package_install read task_vars['ansible_facts'], which under delegate_to still describes the INVENTORY host. svc-storage-nfs-server delegates its controller-mount step to localhost, so a fedora node resolved the RedHat mapping and then ran `dnf install` on the Debian-family runner. Debian-only runs never showed it because node and runner shared os_family and apt happened to exist on both; fedora, centos and arch all hit it.
The delegation target is now asked directly with a minimal setup call, so the facts always describe the host the module runs on, and a failing gather names that host instead of silently falling back. _fact() normalises the two shapes, since task_vars carries bare keys while the setup module returns them ansible_-prefixed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
makepkg refuses to run as root, so the AUR plan marks its kewlfft.aur.aur
call with become_user=aur_builder. The action plugin honoured that by
mutating _play_context.become_user, which ansible-core never reads when
it escalates: the target user comes from the become plugin bound to the
connection, and _play_context mirrors that plugin rather than driving it.
The build therefore ran as root and makepkg aborted with rc 10.
Writing the plugin option instead flips only the AUR call to
-u aur_builder and restores the previous user afterwards, so the
bootstrap calls in the same plan keep running as root. Without a bound
become plugin the action now fails loud instead of silently building as
root.
The old test mocked _play_context and _execute_module, then asserted the
mocked attributes carried what the setter had just written, so it passed
against the broken code. It now builds a real sudo become plugin and
asserts on build_become_command output.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The template carried a three-way conditional across three {% set %} lines,
which was already at the edge of readability. Move the decision into
lookup('smtp_host') as plain control flow and pin all six cases in tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
categories.yml moves from roles/ to the repository-root meta/, joining packages.yml. utils/roles/categories.py owns its location plus the tree-walking helpers that were stranded in entity/name.py, so no caller spells the path or duplicates the lookup.
meta/distros.yml becomes the SPOT for every distribution: os_family, Galaxy platform name, dev-environment base image, and the pkgmgr/pkgmgr-virgin/environment reference templates. It replaces seven scattered copies (DISTRO_FAMILY, VALID_DISTROS, _DISTROS, _CANONICAL_PLATFORMS, two shell arrays, a Makefile literal). INFINITO_DISTROS is no longer a static default but generated by utils/env/handlers/infinito/distros.py; the shell consumers read it through the env layer.
Drops dead CLI surface found along the way: --categories-file and --roles-dir on the provision CLI plus three unused parameters of generate_dynamic_inventory, and the positional roles_file of the invokable categories command.
tests/lint/repository/test_distros_spot.py pins the copies that cannot call the SPOT (workflow literals, docs table) and forbids composing a pkgmgr reference outside it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`ansible.builtin.package` has no module implementation. It is a documentation-only stub whose logic lives in Ansible's own action plugin, so `_execute_module` shipped that stub to the target and every deploy died at the first package install with "module (ansible.builtin.package) is missing interpreter line".
Resolve the generic marker to the host's `ansible_facts.pkg_mgr` before execution.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A role states which package it needs; the registry owns every distribution
mapping and every acquisition path. Declarations live in the repository-root
meta/packages.yml when more than one consumer installs them, and in a role's
own meta/packages.yml otherwise.
schema.py owns the declaration shape, registry.py finds and indexes the
declarations, plan.py turns a resolved spec into ordered module calls, and the
package_install action plugin only executes them. Keeping the plan pure makes
every acquisition path unit-testable without an Ansible connection:
distribution repositories, the AUR with its unprivileged build user, COPR, PPA,
an EPEL bootstrap package, the pacman multilib section and source builds.
The data is guarded by lints: a package id and a concrete distribution package
name each belong to exactly one declaration, a role installs only its own or
shared ids, every id resolves for all five default distributions with an
empty mapping justified by a nocheck, declarations read in ascending order,
every plan builds and names only whitelisted modules, and an external test
queries the real package indexes of all five distributions.
This commit adds the mechanism and the shared collection; no consumer uses it
yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both iteration guides and the nfs_flavor lookup docstring still described
the flavor as RUNTIME conditional, selecting ganesha only for dev and act
and kernel everywhere else. Since the per-variant switching was dropped,
meta/services.yml pins ganesha for every runtime and kernel is a
per-inventory override.
State that, name meta/services.yml as the single point of truth, and
record which os families ship ganesha.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Five helpers rescanned the whole roles/ tree on every call while their result is a pure function of the (static) tree, so callers that invoke them per-role/per-app/per-service turned the scan into O(N^2) filesystem ops. compute_application_gid was the measured hotspot: 254 rescans in one applications render (~25s of a 41s cold render). Memoize each tree-derived result with functools.cache keyed by the resolved roles_dir (the repo's existing idiom), returning tuples so the shared cache cannot be mutated; dependency_resolver uses an instance cache instead since functools.cache on a method would pin instances via self. Cold applications/domains render drops 41s -> ~8.5s (5.6x); behaviour is unchanged (GID/registry/placement/path values identical).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The ruff toolchain was unpinned, so host venvs, the lint container and CI
installed different releases over time. That drift is why lint-python only
failed on CI: locally ruff 0.15.x preview-gated ISC004/FURB192 and (falsely)
fired S310 on urllib.request.Request, while CI's 0.16.0 has ISC004/FURB192
stable and drops the S310-on-Request false positive.
Pin ruff==0.16.0 in pyproject.toml's dev extras (Dependabot keeps it current)
and have utils.install.lint.ruff read that pin and reinstall whenever the
found version drifts, so every environment resolves the same ruff. Then fix
what 0.16.0 flags: parenthesise the implicit string concatenations (ISC004),
use min() over sorted()[0] (FURB192), drop the now-unused S310 noqa on the
Request() builders (RUF100), and delete the narration comments that the
comment lint surfaces in the touched files.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.
Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add lookup('asset', app, pkg, path) and lookup('asset_host'): both resolve
the asset origin through one shared resolve_host, returning the internal
CDN domain when web-svc-cdn is deployed with flavor internal and
cdn.jsdelivr.net otherwise, so template URLs and CSP origins can never
diverge. Versions come from the role's committed files/package-lock.json.
Roles declare frontend dependencies in files/package.json; web-svc-cdn
aggregates every role lockfile, runs one npm ci per role in a node
container and mirrors the packages under /npm/<pkg>@<version>/ following
jsdelivr's path scheme, so internal and external URLs differ only by host.
Convert web-app-dashboard (keycloak-js 24.0.5 UMD) and web-app-littlejs
(bootstrap, fontawesome) to the asset lookup, drop the vendored keycloak
blob, and add a repository lint forbidding hardcoded external CDN hosts
in templates and CSP metas; pre-existing external CDN entries are checked
out with nocheck markers and a migration TODO.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Transient CDN/mirror TLS resets are a recurring hard-fail class (docker.com, PyPI). Harden all three egress paths at their SPOT: the curl filter now emits --retry-all-errors (retries=0 opts a call out), so every ansible curl retries; a new lint requires --retry-all-errors on file-downloading curl and --tries on wget in verbatim files/*.sh (which cannot use the filter), fixing the moodle/alias downloads it flagged; and the update role configures native retries for every package manager - apt Acquire::Retries, dnf retries=, pacman XferCommand over retrying curl - covering all five CI distros (ubuntu shares apt, centos shares dnf).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The compose and swarm render paths launder dollars differently; the launched argv arrived with $pid collapsed to an empty string and $! doubled, and the sidecar crash-looped on a sh syntax error, so opentalk's wait for the embedded object store exhausted its retries. The bucket bootstrap now polls /status from a backgrounded subshell and exec makes the server PID 1, which needs no pid bookkeeping and receives docker stop's TERM directly; the poll loop dies with the container and readiness stays gated by the consumer's bounded wait.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A force_bridge app (matrix's MDAD flavor) deploys as a node-local compose stack and publishes its local port on its host, while openresty runs as an overlay task whose 127.0.0.1 is its own loopback; the readiness probe died on 502. The proxy_pass lookup now resolves networks.local.force_bridge (trust-tagged for the 2.19 templar, hard-failing on an unrendered template instead of silently falling back to loopback) and keys the decision on the real DEPLOYMENT_MODE, since such apps rewrite the effective mode to compose via compose_mode_force. The upstream becomes host.docker.internal:<local_port> rendered as a literal directive, because the request-time resolver cannot see /etc/hosts entries, and the swarm openresty service maps host.docker.internal to the host gateway like prometheus already does. Non-force_bridge renders stay byte-identical.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
categories.yml now declares modes defaults per category (deepest match wins, parents apply as fallback): update, drv and svc-net disable compose and swarm (host-hardware and host-networking categories), svc-registry requires both enabled (CI bootstrap infrastructure). role_modes_defaults resolves a role's policy; the new category-modes lint requires every deploy-bearing service entry to declare exactly those modes.<mode>.enabled values explicitly, exempt via '# nocheck: category-modes' on the service key. update, the four drv roles and the three wireguard roles declare their values explicitly; the dict-valued modes meta key is excluded from the category-tree walkers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A native-metrics app on a node-local force_bridge network cannot be joined or resolved by a swarm-wide prometheus: the stack deploy fails with the network scope error and the scrape job would flap as permanently down. native_metrics_apps now drops those apps at the source (DEPLOYMENT_MODE swarm + networks.local.force_bridge), covering all four call sites (network precreate, compose service networks, external-networks block, scrape include) without per-template guards. DEPLOYMENT_MODE is templated without a default so a missing value fails loudly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
roles/categories.yml now declares the ordered 'stages' and a per-category 'stage' (plus run_after and bootstrap). utils/roles/stage.py resolves a role to its stage and exposes stage_groups(); the new stage_groups lookup feeds the 01_constructor / 02_server / 02_workstation / 03_destructor group loops in place of their hardcoded lists. A static invariant test proves the lookup reproduces the exact previous loops, so the rendered plays (and deploy behavior) are unchanged.
cli.meta.roles.order.run orders its main phase by stage as a run_after tiebreak. plugins/filter/invokable_paths tolerates the new sibling 'stages' key, and the invokable-roles wiring test recognizes stage_groups-driven wiring.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.
Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Resolve every 3+-file same-prefix group flagged by the new prefix lint
by moving the files into a subfolder named after the shared prefix and
dropping the prefix from the file names, updating all references:
- utils/env/handlers/infinito_*.py -> infinito/ (dir/, package_cache/
nested a second level); registry imports and the four handler-dir
globs switched to iter_project_files
- utils/install/lint/ansible_*.py -> ansible/ (aliased imports)
- roles/sys-svc-compose/files/swarm_*.{py,sh} -> files/swarm/
- web-app-erpnext/web-app-zammad files/scripts/apply_*.{py,rb} -> apply/
- web-app-odoo files/sql/{select,update}-* and sql/oidc/{grant,insert,
select,update}-* -> matching subfolders
- web-app-keycloak tasks/update/05_ldap*.yml -> 05_ldap/ (internal
includes made role_path-absolute)
- plugins/filter/get_*.py -> plugins/filter/get/ (filter names are the
filters() dict keys, so Jinja usage is unaffected; ansible discovers
the one-level-deep subdir)
Ansible lookup dirs, addon dirs and .github/workflows keep their flat
layout (the filename is the public id / GitHub only discovers flat).
Unit tests mirror the new source layout.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The export subdir was created run_once on the NFS server only; a worker
that had already readdir'd the export kept a negative dentry and swarm
rejected every replicas>1 task with 'readdirent .../_data: no such file
or directory' for the whole converge window. Materialize the bind-device
dir through each node's own mount, guarded by a per-node mountpoint
assert so an unmounted node fails the deploy loudly instead of receiving
a local mkdir that splits the brain and trips the nfs-client
has-local-data assert on the next full deploy.
compose_volumes popped the volume 'nfs' metadata before nfs_prep could
read it, so declared uid/gid/mode never took effect and every subdir
fell back to root 0755; keep the mapping under the compose-spec
extension key x-infinito-nfs (ignored by docker, verified against stack
config and compose config). KIX declares uid/gid 110010100 mode 0770
through it, fixing the backend_api_token EACCES crash on its shared
volume.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Flatten a single list term in the service lookup so query(service, [ids]) does one applications load instead of one per id. Precompute the required service ids in sys-service-loader and gate the preload by set membership, collapsing about 30 applications loads into one.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
svc-runner's CLI test runs a full nested reinstall deploy and needs
75+ minutes, so the uniform 3600s timeout killed it deterministically
in both modes. A new role_tests lookup reads roles/<id>/meta/tests.yml
(cli.timeout, scaled through the timeout lookup) with the existing
default as fallback, keeping the short hang-leash for every other CLI
test. svc-runner gets 4h and pins the nested deploy to variant 0; the
second matrix round only redeployed cdn+dashboard and added no
assertion value.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The custom flag moves from 33 hardcoded template kwargs into the merged
service config: 'custom: true' names the shared <entity>_custom build,
a string names a second distinct '<string>_custom' image inside one app
(nextcloud whiteboard). The image lookup reads the declaration as a
fallback, so config consumers (e.g. backup image matching) resolve the
exact deployed reference from one declaration point. nextcloud cron now
runs the nextcloud_custom build like the app service instead of the
upstream image.
A new services lint couples the declaration to reality in both
directions: custom requires a role Dockerfile, and a role Dockerfile
requires a custom declaration. The eight legacy roles deploying local
builds under upstream tags or ad-hoc image= overrides are ratcheted via
'# nocheck: dockerfile-custom'.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New 'image' lookup is the SPOT for resolving <registry>/<image>:<version>
(swarm registry prefix in swarm mode, bare in compose; custom= accepts a
string base for a second distinct custom image inside one app).
container_image becomes a thin wrapper that forwards terms+kwargs and adds
the compose image: "<ref>" line, so all existing template call sites keep
their output byte-identical. The lookup-usage integration test now counts
lookup_loader.get() delegation as production usage.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The OpenTalk controller v0.34.1 verifies its bucket via S3 ListBuckets
membership, but SeaweedFS hides every bucket from an unconfigured
identity (confirmed identical on 4.38 and 4.39), and the embedded sidecar
started with no s3.config, so the controller's signed request got an
empty bucket list and crash-looped on 'bucket missing' although the
bucket existed. The sidecar now runs s3.configure to register the
consumer's access key with Read/Write/List/Tagging on its bucket, the
same grant the central provider gets from its mounted s3.json.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two wrapper behaviours move onto single sources of truth:
Exec timeout: SYS_TIMEOUT_CONTAINER_EXEC (07_schedule.yml, default 7d so a
hard kill never beats a production migration; the dev/CI inventory
tightens it to 60min) reaches every task host via the play environment as
CONTAINER_EXEC_TIMEOUT. The wrapper prefixes only the exec subcommand with
coreutils timeout --kill-after=15. Ansible task timeouts are SIGALRM-based
and slept through the last wedged docker socket for 4.5 hours; the OS
level kill still fires there and returns 124, which until/failed_when and
the rescue diagnostics react to. Unset or 0 keeps todays behaviour.
CA container paths: the bind-mount targets for the injected root CA and
wrapper script lived as string literals in container.py, compose_ca.py
and the filter plugin. They now live once in the CA_TRUST group_vars SPOT
(inject_cert_container, inject_wrapper_container) and reach container.py
as env vars (play environment plus CA_TRUST_SYSTEMD_ENV) and compose_ca.py
as CLI arguments built by the compose_ca_inject_cmd lookup. A caller that
provides the host CA env but not the container paths is half-migrated and
fails loudly instead of silently dropping CA injection; a host without any
CA env keeps the soft fallback. All tests read the paths from the SPOT,
and a regression test asserts the runtime scripts carry no literal copy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
storage.nfs shrinks to the deployment-specific server address: export_base moves into svc-storage-nfs-server's services.yml (the server dictates where exports root) and the mount protocol version into svc-storage-nfs-client's services.yml (the client owns how it mounts). Both are read through utils.storage.nfs loaders by the nfs lookups, the env handler and 00_topology, eliminating the unrendered-variables class for these values; the group_vars assert shrinks accordingly and the swarm test extras stop restating role defaults.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lookup plugins receive ansible variables unrendered, so routing the state subdir through a group_vars jinja value handed state_path the literal '{{ lookup(...) }}' string and the controller mount asked the server for /srv/nfs/{{ lookup('nfs_state_subdir') }} (Connection refused, nextcloud swarm roundtrip). Both nfs lookups now import STATE_SUBDIR directly; the intermediate lookup, the group_vars indirection and the unit-test fixture var that masked the gap are gone.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A wedged DinD dockerd hung a single diagnostics docker exec until the 360-minute job timeout cancelled the job together with its artifact uploads (job 86013153099 sat 5h in service logs). Route every diagnostics exec through a timeout wrapper, tail-limit the log dumps, bound showmount, cap the collect step at 30 minutes, and dump the pending services' log tails once on the final converge failure. The NFS state path moves into a single source of truth: STATE_SUBDIR in utils/storage/nfs.py feeds the nfs_state_subdir lookup (deploy side) and the derived INFINITO_SWARM_NFS_STATE_PATH env var (harness side); the diagnostics script requires both path vars instead of defaulting, and the swarm inventory drops its redundant export_base restatement.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The controller crash-looped on a missing S3 bucket and the recorder on an unprovisioned OIDC client while the swarm converge gate blocked before any post-flush task could create either. The embedded seaweedfs sidecar now bootstraps its consumer bucket itself at startup (new seaweedfs_sidecar_script filter: entrypoint-wrapped server, bounded readiness poll, idempotent weed shell bucket create, TERM/INT forwarded to the server), and the Keycloak clients are provisioned before the handler flush. Post-flush bucket tasks stay as an idempotent backstop; the obsolete restart tasks are dropped. Live-validated against chrislusf/seaweedfs:4.38: bucket created, stop behavior identical to the stock entrypoint.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rename the meta/services.yml key across all roles, the roles_with_default_placement lookup (now roles_with_placement), the get_role_default_placement / iter_roles_with_default_placement helpers, and every consumer (swarm closure, compose_volumes filter, nfs_prep, container deploy template, constructor stage, extend_inventory) plus their tests and docs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
compose_networks/container_networks read DEPLOYMENT_MODE directly, so a node-local compose-in-swarm role (compose_mode_force=compose, e.g. the matrix ansible flavor) rendered a swarm overlay a standalone compose container cannot attach to ("network matrix not found"). Honor compose_mode_force like the other mode-aware lookups.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A curl against an accepted-but-silent peer hangs forever: task retries
never fire because the first attempt never returns, and the CI job dies
at the 6h runner cut (proven by the mailu admin-API hang that took the
suitecrm/peertube swarm legs down). The new `curl` filter renders the
hardened base command ("{{ 30 | curl }}" -> curl -s --connect-timeout 5
--max-time 30); scaling composes with lookup('timeout').
test_curl_via_filter enforces it: bare curl invocations in role tasks,
templates and shipped scripts fail the lint with a pointer to the
filter. Plain .sh files, which cannot render Jinja, pass by pinning
--max-time inline; Docker healthchecks are exempt (their timeout field
caps them); `# nocheck: curl-timeout` opts out (catalogued in
suppression.md).
Convert all 34 existing invocations: mailu (replacing the short-lived
MAILU_CURL var), elasticsearch, checkmk, yourls (300s for the
schema-creating installer POST), the telegram alert script, and the
plain scripts of jellyfin, svc-runner, apt_repo, moodle (600s tarball)
and the mattermost provisioner. Mailu's user/token tasks additionally
carry task-level timeouts so a wedged container exec cannot stall the
play.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
