The mount pipeline was split across two layers: compose_volumes and
container_volumes lived as filter modules that ansible's filter loader
rejected on every start ("has no attribute 'FilterModule'", two warnings
per run), while the same-named lookups wrapped them thinly. The filter
bodies now live inside their lookups (the tor_socks precedent), the
filter files are gone, and the render tests moved next to the lookup
tests.
As lookups they see the templating context, so the application_id term
every call site passed back to its own role is gone too: both lookups
read application_id from context and fail loudly when it is missing.
107 templates and nfs_prep.yml drop the term; the calls collapse to
lookup('compose_volumes') and lookup('container_volumes', service_name).
The container_volumes call is now canonical - four leading spaces,
service_name in the service position, | indent(4) - enforced by a new
lint (test_container_volumes_call_shape). Normalizing the 85 literal
call sites surfaced that erpnext keyed its meta/volumes.yml mounts by
container name instead of compose service key, the only role to do so;
its meta now uses the service keys. wordpress's column-0 call, which
rendered its first line at top level, is indented like every other.
Template logic that only restated meta/volumes.yml is deleted: mailu fed
compose_volumes an extra_volumes dict whose eight names were looked up
from the very meta file compose_volumes reads itself; zammad's init and
railsserver extra mounts were the volumes.yml entries minus the two
services, which the meta now lists (the when-gated OIDC binds included).
Conditionality that meta cannot express stays in code, but in role-local
lookup plugins instead of template loops: matrix_bridge_mounts renders
the per-bridge registration mounts, nextcloud_extra_volumes the
feature-gated whiteboard/recording volumes (type: volume entries are
declared unconditionally by compose_volumes, so gating must happen
before it).
Parity is proven, not assumed: every mechanically rewritten template is
byte-identical to HEAD after the transformation regexes (95/95), and the
semantic roles render identically through the real functions against
HEAD's meta - zammad across six sso/tls polarities, all seven erpnext
services, mailu's top level. make test-lint passes complete.
Also: suitecrm's compose template pins container_extra_hosts with
host_alias (the SOCKS proxy URL host.docker.internal never resolved in
compose+tor), and the five disabled mautrix bridge addons use the
empty-default config lookup so templating the applications dict stops
emitting 125 errors per batch in every CI job.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Output of 'make autoformat': ruff collapses expressions that were wrapped across lines without need, shfmt and the yaml formatters drop surplus blank lines. No semantic change - the whole diff is 10 insertions against 39 deletions, and every hunk joins lines that already said the same thing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.
Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both creates: guards pointed at paths that could never match, so the
bootstrap and setup:install shells re-ran on every deploy.
The first guard went through docker_volume_path, which hardcodes
/var/lib/docker/volumes/<name>/_data and misses any relocated docker
data-root. The second was worse: path_join over the bare volume name
produced a relative path, resolved against the Ansible working
directory, which matched nowhere at all.
Both now derive from the mountpoint the daemon reports. The stack-host
gate moves to a single include, mirroring web-app-akaunting.
With its last caller converted, docker_volume_path is deleted. It could
not be repaired: a Jinja filter runs on the controller and cannot know a
remote host's data-root.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Since the swarm-NFS migration two chains copied the same bytes: svc-bkp-nfs-2-local
reads the export from the NFS server's own filesystem, while baudolo on the manager
copied the same files again THROUGH the NFS mount. That was a migration leftover -
svc-bkp-nfs-2-local arrived in adb8bcf69 (2026-07-10) and nobody taught the older role
to step back - and the role README already forbade it as a rule nobody enforced. In the
DR drill the duplicate costs three times over: the pulled tree, the device copy and the
restore root.
swarm_nfs_backed() in utils/storage/nfs.py is now the single answer to "does the swarm
NFS rewrite back this volume". plugins/filter/compose_volumes.py (which performs the
rewrite) and this role's backup_volume lookup (which feeds baudolo's
--volumes-no-backup-required) both call it, so the two cannot drift. On the current tree
that excludes 83 of 158 type: volume entries.
Four conditions keep the exclusion from ever dropping a volume nobody else captures:
* a manager-pinned role's volumes are not on the export (web-app-seaweedfs holds
Nextcloud's object store, deliberately node-local per its meta/services.yml)
* a role forcing another mode via compose_mode_force is not on the export either.
That override is per role (roles/sys-svc-compose/defaults/main.yml:3), so it is
resolved per role inside the predicate from roles/<id>/vars/main.yml, not read once
from ambient scope. web-app-bigbluebutton pins compose statically;
web-app-matrix computes it, and a value the predicate cannot resolve counts as
forcing - unknown mode keeps the volume backed up
* an entry opting out with nfs: false stays node-local
* the exclusion applies at all only when the svc-bkp-nfs-2-local inventory group is
populated. That role enters a play by group membership alone
(tasks/groups/svc-bkp-roles.yml:19-22) and svc-storage-nfs-server ships a variant
with nfs_backup: {enabled: false}, so without the gate an export whose repository
nobody deployed would lose 83 volumes from both capture paths
Every condition fails toward the old double capture: wasted space, never lost data.
Compose mode is untouched - the exclusion cannot fire without swarm and an nfs backend.
DEPLOYMENT_MODE and storage are templated through the lookup's templar, because
group_vars/all/18_swarm.yml:2 defines the mode as a Jinja expression; comparing the raw
source string would silently disable the whole exclusion.
Twelve new tests pin the contract, including the two failure modes that would be
invisible in production: an untemplated mode, and an exclusion without an export
repository. Both were verified by mutation - removing either guard makes its test fail.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The pre-pull task set `async` unconditionally while gating only `poll`. With
async disabled, `poll` became `omit` and ansible fell back to its built-in 15s
poll while `async` still routed the task through async_wrapper: ansible
collected the job itself and then ran `async_status mode=cleanup`, deleting the
job file that tasks/utils/async/reap.yml expects to read. The reap reported a
successful `ollama pull` as a failed async job, failing web-app-nextcloud 2 in
run 30781723102.
Both keys are now switched off together, matching all six peer call sites. On
the resulting synchronous path the reap's `async_reap_tolerate` list no longer
applies, so the tolerated "up to date" answer is carved out in `failed_when`
instead. It keys on `failed` rather than `rc`, so a module-level failure that
carries no rc is not swallowed.
async_failures._which() now honours loop_control.loop_var: the fired result
records the chosen name under `ansible_loop_var`, so a batch failure names
`smollm2:135m` instead of the bare job id.
A second lint case asserts that async and poll carry `omit` together - the rule
this call site was the only one of seven to break.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The repo's async convention is fire and forget: ASYNC_POLL is 0 whenever
ASYNC_ENABLED is true, and the swarm test lane forces it true through
extra vars. A task dispatched that way returns a stub, so a changed_when
or failed_when reading its own register evaluates against nothing and can
never report changed and never fail. Several tasks carried a
"not ASYNC_ENABLED and" prefix in front of those conditions, added to
avoid touching undefined rc and stdout, which turned the guard off
entirely rather than adapting it.
That is how the nextcloud plugin configuration stayed broken unseen: the
batch reported ok after 0.8s while its delegated commands kept running,
the next task read the half written configuration, and no failure in any
of them could surface.
Every async in the repo was decided individually rather than swept. Where
nothing consumes the result, the async stays and a one line rationale now
records what does not read it. Where a later task, a handler or a browser
does consume it, either the async goes or the result is collected.
Collecting is what the new reaper does. It waits for a whole batch, then
reports every broken job at once instead of aborting on the first, so one
run shows whether a single item or the whole set is broken. Deciding what
counts as broken lives in a filter plugin, because the distinction is
subtle enough to deserve tests: a module reports failure in msg, a command
in rc and stdout, an idempotent API answers "already exists" and means
success, and a job that cannot be read at all comes back marked finished
with no outcome, which would otherwise read as green.
Timing cost is roughly one to two minutes on a deploy measured in hours.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three conflicts, each resolved by combining both sides rather than picking one.
compose.yml passes both env vars through to the container: swarm-nfs adds
INFINITO_SWARM_MGR_IP, this branch adds INFINITO_TOR_SOCKS_PORT, and neither
substitutes for the other.
daemon.json.j2 takes the data-root block from swarm-nfs and keeps HEAD's dns
key. The incoming side writes networks.internet.dns directly; lookup(
'container_dns') already returns that value as its clearnet entry and prepends
the docker0 bridge on a Tor node in compose mode, so the incoming form is a
strict subset and dropping it loses nothing.
The WordPress CA-trust addon spec gains skipUnlessServiceEnabled("sso") from
swarm-nfs alongside HEAD's gotoOnion and resolveTimeout, and asserts on the
Plugin Name the mu-plugin header declares - the mustuse screen renders that,
not the filename - with resolveTimeout applied to the literal the incoming
side left bare.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
f956de889 made the slot ceiling generous, which removed the accidental brake the old autosizing had provided: while the ceiling was derived from free disk space, growth stopped long before the disk filled. With a fixed ceiling the only thing between growth and ENOSPC is the free-space guard, and its stock 1% is 300 MB on a 30 GB data root - less than the 1024 MB unit SeaweedFS allocates in, so a single growing volume can outrun it.
The floor is now expressed as whole allocation units rather than a percentage, which is meaningless without knowing the disk size. Two units: one for the growth in flight, one for concurrent writers. The size limit is stated alongside it so both come from one constant instead of the floor deriving from a value the image injects - verified that our flag wins, since -master.volumeSizeLimitMB=512 makes the master report 512 MB.
-volume.minFreeSpace replaces the deprecated -volume.minFreeSpacePercent, per the image's own help text. The rendered command was run against chrislusf/seaweedfs:4.39: master, volume server and filer all start.
Left open deliberately: web-app-seaweedfs declares min_storage: 304MB, which cannot hold with a 2 GiB floor. That value feeds the CI storage filter in scripts/meta/resolve/apps.sh, where 60GB is the budget, so nothing is dropped today - but the declaration understates what the role now needs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
a3fcd3e7f stopped the entrypoint from autosizing the ceiling out of free disk space, but replaced it with a flat 64 that I adopted without deriving. Each bucket is a SeaweedFS collection and the master grows one in batches of seven - 'create 7 volume' in the artifact, copy_1 = 7 in the image's own master scaffold - so 64 covers nine collections. Twenty-five roles declare the service, and the tenth consumer would have hit 'No writable volumes and no free volumes left' again, just later and without the disk-space link.
The ceiling now comes from volume_slots(): one grow batch per bucket plus one for the default collection, with the consumer count read from roles_with_service. Note what that count is: the consumer gate gets whether seaweedfs is in group_names, not whether the consumer itself is installed, so the value is derived from the repository rather than from the host. It grows with the codebase, which is what the ceiling has to keep up with.
The collection count is a required parameter with no default. A silent fallback would be worse here than a loud failure: without the flag the stock entrypoint injects -volume.max=0 and the autosizing trap is back.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role passed no -volume.max, so the stock entrypoint injects -volume.max=0, which means 'derive the ceiling from free disk space'. That derivation charges every non-full volume its full volumeSizeLimit: the master's default grow batch creates seven 1 GB volumes on the first write, reserving ~7.0 GB, and the manager's docker data root had 6.8 GB free. unclaimedSpaces goes non-positive, maxVolumeCount collapses to volumeCount, and the node reports zero free slots.
Those seven volumes belong to the default collection, so they can never serve collection 'nextcloud'. The master says it plainly - 'topo failed to pick 1 from 0 node candidates', 'create 7 volume, created 0' - and every S3 PUT returns 500 while seven empty volumes sit on disk. The volume server never left the topology: RestartCount 0, one unbroken log stream across the window.
That 500 is what starts the web-app-nextcloud 0 chain: occ maintenance:install fails eleven times, leaves a half-installed instance, and slot 1 then holds nextcloud-init-sync.lock for four minutes while every other replica dies on a blocking flock. Reproduced independently in variant 5 with collection 'peertube'; both compose jobs stayed green.
-volume.max makes isArgPassed true, so the entrypoint stops injecting the autosizing value. Slots become a fixed budget rather than a function of disk space; volumes stay sparse and unpreallocated, and -volume.minFreeSpacePercent still guards a genuinely full disk.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
categories.yml moves from roles/ to the repository-root meta/, joining packages.yml. utils/roles/categories.py owns its location plus the tree-walking helpers that were stranded in entity/name.py, so no caller spells the path or duplicates the lookup.
meta/distros.yml becomes the SPOT for every distribution: os_family, Galaxy platform name, dev-environment base image, and the pkgmgr/pkgmgr-virgin/environment reference templates. It replaces seven scattered copies (DISTRO_FAMILY, VALID_DISTROS, _DISTROS, _CANONICAL_PLATFORMS, two shell arrays, a Makefile literal). INFINITO_DISTROS is no longer a static default but generated by utils/env/handlers/infinito/distros.py; the shell consumers read it through the env layer.
Drops dead CLI surface found along the way: --categories-file and --roles-dir on the provision CLI plus three unused parameters of generate_dynamic_inventory, and the positional roles_file of the invokable categories command.
tests/lint/repository/test_distros_spot.py pins the copies that cannot call the SPOT (workflow literals, docs table) and forbids composing a pkgmgr reference outside it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
categories.yml moves from roles/ to the repository-root meta/, joining packages.yml. utils/roles/categories.py owns its location plus the tree-walking helpers that were stranded in entity/name.py, so no caller spells the path or duplicates the lookup.
meta/distros.yml becomes the SPOT for every distribution: os_family, Galaxy platform name, dev-environment base image, and the pkgmgr/pkgmgr-virgin/environment reference templates. It replaces seven scattered copies (DISTRO_FAMILY, VALID_DISTROS, _DISTROS, _CANONICAL_PLATFORMS, two shell arrays, a Makefile literal). INFINITO_DISTROS is no longer a static default but generated by utils/env/handlers/infinito/distros.py; the shell consumers read it through the env layer.
Drops dead CLI surface found along the way: --categories-file and --roles-dir on the provision CLI plus three unused parameters of generate_dynamic_inventory, and the positional roles_file of the invokable categories command.
tests/lint/repository/test_distros_spot.py pins the copies that cannot call the SPOT (workflow literals, docs table) and forbids composing a pkgmgr reference outside it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Five helpers rescanned the whole roles/ tree on every call while their result is a pure function of the (static) tree, so callers that invoke them per-role/per-app/per-service turned the scan into O(N^2) filesystem ops. compute_application_gid was the measured hotspot: 254 rescans in one applications render (~25s of a 41s cold render). Memoize each tree-derived result with functools.cache keyed by the resolved roles_dir (the repo's existing idiom), returning tuples so the shared cache cannot be mutated; dependency_resolver uses an instance cache instead since functools.cache on a method would pin instances via self. Cold applications/domains render drops 41s -> ~8.5s (5.6x); behaviour is unchanged (GID/registry/placement/path values identical).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Transient CDN/mirror TLS resets are a recurring hard-fail class (docker.com, PyPI). Harden all three egress paths at their SPOT: the curl filter now emits --retry-all-errors (retries=0 opts a call out), so every ansible curl retries; a new lint requires --retry-all-errors on file-downloading curl and --tries on wget in verbatim files/*.sh (which cannot use the filter), fixing the moodle/alias downloads it flagged; and the update role configures native retries for every package manager - apt Acquire::Retries, dnf retries=, pacman XferCommand over retrying curl - covering all five CI distros (ubuntu shares apt, centos shares dnf).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The server.yml split hoisted csp to the top level; the filter kept
reading server.csp.whitelist and silently returned no mirrorable asset
hosts, leaving the mirror vhost whitelist and body rewrite empty.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The compose and swarm render paths launder dollars differently; the launched argv arrived with $pid collapsed to an empty string and $! doubled, and the sidecar crash-looped on a sh syntax error, so opentalk's wait for the embedded object store exhausted its retries. The bucket bootstrap now polls /status from a backgrounded subshell and exec makes the server PID 1, which needs no pid bookkeeping and receives docker stop's TERM directly; the poll loop dies with the container and readiness stays gated by the consumer's bounded wait.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
categories.yml now declares modes defaults per category (deepest match wins, parents apply as fallback): update, drv and svc-net disable compose and swarm (host-hardware and host-networking categories), svc-registry requires both enabled (CI bootstrap infrastructure). role_modes_defaults resolves a role's policy; the new category-modes lint requires every deploy-bearing service entry to declare exactly those modes.<mode>.enabled values explicitly, exempt via '# nocheck: category-modes' on the service key. update, the four drv roles and the three wireguard roles declare their values explicitly; the dict-valued modes meta key is excluded from the category-tree walkers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
roles/categories.yml now declares the ordered 'stages' and a per-category 'stage' (plus run_after and bootstrap). utils/roles/stage.py resolves a role to its stage and exposes stage_groups(); the new stage_groups lookup feeds the 01_constructor / 02_server / 02_workstation / 03_destructor group loops in place of their hardcoded lists. A static invariant test proves the lookup reproduces the exact previous loops, so the rendered plays (and deploy behavior) are unchanged.
cli.meta.roles.order.run orders its main phase by stage as a run_after tiebreak. plugins/filter/invokable_paths tolerates the new sibling 'stages' key, and the invokable-roles wiring test recognizes stage_groups-driven wiring.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.
Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Resolve every 3+-file same-prefix group flagged by the new prefix lint
by moving the files into a subfolder named after the shared prefix and
dropping the prefix from the file names, updating all references:
- utils/env/handlers/infinito_*.py -> infinito/ (dir/, package_cache/
nested a second level); registry imports and the four handler-dir
globs switched to iter_project_files
- utils/install/lint/ansible_*.py -> ansible/ (aliased imports)
- roles/sys-svc-compose/files/swarm_*.{py,sh} -> files/swarm/
- web-app-erpnext/web-app-zammad files/scripts/apply_*.{py,rb} -> apply/
- web-app-odoo files/sql/{select,update}-* and sql/oidc/{grant,insert,
select,update}-* -> matching subfolders
- web-app-keycloak tasks/update/05_ldap*.yml -> 05_ldap/ (internal
includes made role_path-absolute)
- plugins/filter/get_*.py -> plugins/filter/get/ (filter names are the
filters() dict keys, so Jinja usage is unaffected; ansible discovers
the one-level-deep subdir)
Ansible lookup dirs, addon dirs and .github/workflows keep their flat
layout (the filename is the public id / GitHub only discovers flat).
Unit tests mirror the new source layout.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The export subdir was created run_once on the NFS server only; a worker
that had already readdir'd the export kept a negative dentry and swarm
rejected every replicas>1 task with 'readdirent .../_data: no such file
or directory' for the whole converge window. Materialize the bind-device
dir through each node's own mount, guarded by a per-node mountpoint
assert so an unmounted node fails the deploy loudly instead of receiving
a local mkdir that splits the brain and trips the nfs-client
has-local-data assert on the next full deploy.
compose_volumes popped the volume 'nfs' metadata before nfs_prep could
read it, so declared uid/gid/mode never took effect and every subdir
fell back to root 0755; keep the mapping under the compose-spec
extension key x-infinito-nfs (ignored by docker, verified against stack
config and compose config). KIX declares uid/gid 110010100 mode 0770
through it, fixing the backend_api_token EACCES crash on its shared
volume.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The OpenTalk controller v0.34.1 verifies its bucket via S3 ListBuckets
membership, but SeaweedFS hides every bucket from an unconfigured
identity (confirmed identical on 4.38 and 4.39), and the embedded sidecar
started with no s3.config, so the controller's signed request got an
empty bucket list and crash-looped on 'bucket missing' although the
bucket existed. The sidecar now runs s3.configure to register the
consumer's access key with Read/Write/List/Tagging on its bucket, the
same grant the central provider gets from its mounted s3.json.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New web-svc-mirror role: an nginx vhost that proxies and caches exactly
the external asset hosts aggregated from all deployed applications' CSP
whitelists (external_asset_origins filter) under first-party paths
(/<origin-host>/...), refusing every other upstream. svc-net-tor
consumes the service, so any tor deployment automatically serves the
mirror on its onion.
For apps with services.tor.enabled the shared body filter rewrites
external asset URLs in HTML to the onion mirror (onion sessions only;
clearnet untouched) and the CSP builder allows the mirror origin in the
asset directives. The dashboard's keycloak-js vendoring is reverted:
onion sessions now load the adapter through the mirror, clearnet loads
jsdelivr directly as before.
Verified: variant=0 deploy green (failed=0), dashboard 14/14 over onion
with keycloak-js served through the mirror, mirror spec green, health
check accepts the 404 vhost root.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two wrapper behaviours move onto single sources of truth:
Exec timeout: SYS_TIMEOUT_CONTAINER_EXEC (07_schedule.yml, default 7d so a
hard kill never beats a production migration; the dev/CI inventory
tightens it to 60min) reaches every task host via the play environment as
CONTAINER_EXEC_TIMEOUT. The wrapper prefixes only the exec subcommand with
coreutils timeout --kill-after=15. Ansible task timeouts are SIGALRM-based
and slept through the last wedged docker socket for 4.5 hours; the OS
level kill still fires there and returns 124, which until/failed_when and
the rescue diagnostics react to. Unset or 0 keeps todays behaviour.
CA container paths: the bind-mount targets for the injected root CA and
wrapper script lived as string literals in container.py, compose_ca.py
and the filter plugin. They now live once in the CA_TRUST group_vars SPOT
(inject_cert_container, inject_wrapper_container) and reach container.py
as env vars (play environment plus CA_TRUST_SYSTEMD_ENV) and compose_ca.py
as CLI arguments built by the compose_ca_inject_cmd lookup. A caller that
provides the host CA env but not the container paths is half-migrated and
fails loudly instead of silently dropping CA injection; a host without any
CA env keeps the soft fallback. All tests read the paths from the SPOT,
and a regression test asserts the runtime scripts carry no literal copy.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The controller crash-looped on a missing S3 bucket and the recorder on an unprovisioned OIDC client while the swarm converge gate blocked before any post-flush task could create either. The embedded seaweedfs sidecar now bootstraps its consumer bucket itself at startup (new seaweedfs_sidecar_script filter: entrypoint-wrapped server, bounded readiness poll, idempotent weed shell bucket create, TERM/INT forwarded to the server), and the Keycloak clients are provisioned before the handler flush. Post-flush bucket tasks stay as an idempotent backstop; the obsolete restart tasks are dropped. Live-validated against chrislusf/seaweedfs:4.38: bucket created, stop behavior identical to the stock entrypoint.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rename the meta/services.yml key across all roles, the roles_with_default_placement lookup (now roles_with_placement), the get_role_default_placement / iter_roles_with_default_placement helpers, and every consumer (swarm closure, compose_volumes filter, nfs_prep, container deploy template, constructor stage, extend_inventory) plus their tests and docs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New to_onion_url filter swaps a clearnet host suffix for the tor_node and forces http (no-op without tor_node or off-primary hosts). body_filter.lua uses it in onionize() so that .onion requests receive the onion CDN origin for injected head/body snippets (e.g. universal-logout.js), which the Tor browser cannot reach on clearnet. Covered by 7 unit tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A curl against an accepted-but-silent peer hangs forever: task retries
never fire because the first attempt never returns, and the CI job dies
at the 6h runner cut (proven by the mailu admin-API hang that took the
suitecrm/peertube swarm legs down). The new `curl` filter renders the
hardened base command ("{{ 30 | curl }}" -> curl -s --connect-timeout 5
--max-time 30); scaling composes with lookup('timeout').
test_curl_via_filter enforces it: bare curl invocations in role tasks,
templates and shipped scripts fail the lint with a pointer to the
filter. Plain .sh files, which cannot render Jinja, pass by pinning
--max-time inline; Docker healthchecks are exempt (their timeout field
caps them); `# nocheck: curl-timeout` opts out (catalogued in
suppression.md).
Convert all 34 existing invocations: mailu (replacing the short-lived
MAILU_CURL var), elasticsearch, checkmk, yourls (300s for the
schema-creating installer POST), the telegram alert script, and the
plain scripts of jellyfin, svc-runner, apt_repo, moodle (600s tarball)
and the mattermost provisioner. Mailu's user/token tasks additionally
carry task-level timeouts so a wedged container exec cannot stall the
play.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Deploy GitLab from the official CNG CE images as six services: webservice (puma) and sidekiq scale to the topology default, workhorse is the sole HTTP entry, gitaly and a one-shot migrations job are pinned singletons, shell runs gitlab-sshd. The Omnibus container, its Dockerfile and the GITLAB_OMNIBUS_* config groups are removed. All rails/workhorse/shell/gitaly configs and shared secrets are pre-rendered by Ansible and mounted read-only; eleven credentials land in meta/schema.yml including an rsa_pem_2048 OIDC signing key (new value_generator algorithm). Postgres, redis and S3 object storage ride the existing lookups; OIDC, SMTP, the public SSH port and the root password are preserved. compose_volumes gains a per-volume nfs:false opt-out so the gitaly repositories volume stays node-local in swarm (upstream forbids git repositories on NFS); nfs_prep skips volumes without driver_opts accordingly. The Nextcloud gitlab integration execs the CNG rails runner. ruby_single_line dies with its only consumer; ruby_dq keeps test coverage. Ports (8181/8080/2222/8075/3808/9122) and service names resolve from single SPOT declarations in meta/services.yml via vars constants; the version pin sits on the webservice image pair so the updater can bump it. Live-validated in compose up to all six services healthy and migrations completed; fresh-install-only semantics for existing Omnibus data are documented.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Over Tor a page served on an .onion still advertised a clearnet-only CSP and oauth2-proxy config, so onion origins were blocked (dashboard could not frame auth.<onion>; oauth2-proxy rejected the redirect back to the onion app). build_csp_header now mirrors the clearnet primary to the node onion per token; the keycloak oauth2-proxy config derives whitelist_domains and cookie_domains from the app resolved domains. Exclusive (onion-only) apps get onion tokens only, primary apps get both, non-tor apps stay clearnet.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The swarm branch passed values through raw, so a value with interior
whitespace (MAIL_FROM_NAME='Infinito.Nexus by ...') landed unquoted in
the rendered .env and broke the env parser: the snipe-it service
crashlooped at 0/3 until the converge wait failed in every variant.
Quote (and escape backslash/double-quote) only when the value contains
space/tab/newline; everything else stays raw, so the swarm invariants
hold: no compose $$-doubling, raw $ and double quotes preserved,
and the playwright --env-file path (docker CLI keeps quotes literal)
is unaffected because no playwright env value can contain whitespace.
Compose quoting is untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
resource_filter now falls back service_name -> entity_name -> hard
default, so entity-level limits keep applying to per-service compose
entries. Compose service names drop the entity prefix (kix, jitsi,
zammad, bluesky); the docker container_name keeps it via compose_only.
A new lint forbids '<entity>-'-prefixed service names.
Service names and internal ports are declared once: vars/main.yml
holds <ROLE>_<SERVICE>_SERVICE constants (container names derived via
entity_name), meta/services.yml declares ports.internal consumed
through config lookups. sso-proxy upstreams stack-qualify the origin
host in swarm because short aliases are ambiguous on shared overlays.
Stale proxy.service_key entries fixed for erpnext/shopware/nextcloud.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One entrypoint wrapper imports the project CA into the JVM truststore for penpot and jenkins: env-driven (CA_TRUST_CERT from sys-svc-compose-ca, set in both modes), fail-open without a cert (production), fail-loud with one; writable cacerts are patched in place, read-only layouts fall back to /tmp + JAVA_TOOL_OPTIONS. Replaces penpot's inline compose command and jenkins' role-local entrypoint, which silently skipped the import when cacerts were not writable. ca_cert_container becomes a plain constant since its only Jinja consumer moved into the wrapper.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ca_cert_host derives the host cert path from SOFTWARE_DOMAIN; ca_cert_container carries the fixed in-container bind-mount target. CA_ROOT.cert_host and the test fixtures consume the filter; the standalone runtime scripts (compose_ca.py, container.py) cannot import it and are pinned to the SPOT by the unit test instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
'login source already exists' was matched inline in both the LDAP and the OIDC setup; the filter is the single marker source and normalizes case.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The two handler tasks carried the identical four-marker stderr chain inline and matched case-sensitively; the filter is the single marker source and normalizes case.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The five docker network connect/disconnect tolerances were inline stderr substring chains, inconsistent between sites (the handler variants matched case-sensitively without a default). The docker_network_noop filter is the single marker source per operation; disconnect additionally tolerates absent networks/containers, which is the proven swarm failure ('No such container: postgres' when the central DB is a swarm service). Drop the now-orphaned DISCOURSE_ERROR_NOT_CONNECTED.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In the swarm-DinD test the ansible controller (ansible_connection: docker) cannot read the STACK_HOST-rendered xwiki.properties, so compose_volumes' content hash falls back to the source path and the config name is constant. xwiki deploys twice (bootstrap with the OIDC block off, final with it on after the extension installs), so the config content changes but the name does not, and the final docker stack deploy fails to update the immutable config with 'only updates to Labels are allowed'.
Honor an explicit name: on config/secret entries in compose_volumes (volumes already did), and give xwiki's properties config a name that rotates with the OIDC phase, so the final deploy creates a new config instead of updating in place. Backward-compatible: no other role sets name: on a config. make test green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
