Pins of every role applied at every round index, so a role the round did not deploy could pull a partner into a heavier variant (web-app-discourse#1 measured 35.46 GB against the 32 GB reservation cap). resolve_round now starts every role on its clamped index and recomputes the include and the pins in force until the assignment settles, raising VariantPinError on a cycle. The planner resolves each round once (39.1 s to 16.0 s for all rounds), affected judges every (role, index) with the round the planner deploys for it, and the resource budget lint measures the pin-aware rounds. The stale web-app-matomo pin above the primary-only docs entry is dropped, and test_variant_pin_fixed_point.py guards every planned round against unsettled pins and pins above a disabled entry. docs/contributing/artefact/files/role/variants.md explains the settle loop with diagrams.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fork CI run 41147980556 pgAdmin#0 failed in sys-ctl-hlth-runtime with
"stalwart_webmail: PHP 8.4.26 is below the security floor 8.4.27".
The deploy ran the fork's GHCR mirror of roundcube/roundcubemail:1.7.5-
apache, copied on 10-19 with PHP 8.4.26, while upstream re-pushed the
same tag on 10-27 with PHP 8.4.27. Branch CI runs the sync with
--only-missing, which skipped the tag because it existed ("destination
exists, skipping") without comparing digests, and the full refresh only
runs on the fork's default branch, which still pins 1.7.4.
- utils/docker/registry.py: manifest_digest() reads the
Docker-Content-Digest of a reference with a HEAD request, which does
not count against Docker Hub's pull allowance.
- GHCRProvider.is_stale() reports a mirrored tag as stale only when both
registries answer with a digest and the two differ.
- Under --only-missing an existing tag is re-copied when it is stale; a
failed re-copy is reported on stderr with the image and keeps the
existing copy, so one refresh cannot block every deploy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI run 40979252521 Weblate#1 (swarm) never converged: every task after
the first died in collectstatic with "PermissionError: [Errno 13]
Permission denied: '/app/cache/static/weblate_fonts'". The tmpfs at
/app/cache/static sits inside the shared weblate_cache volume; the first
task's runc creates the mountpoint as root 0755, and runc gives a tmpfs
on an existing mountpoint that directory's mode. 99a7d28067 declared
mode 1777, which compose honours, but docker stack deploy keeps only the
tmpfs size (docker/cli 29.9.2 convert/volume.go), so swarm still got a
root 0755 tmpfs.
- utils/storage/nfs.py: nested_tmpfs_mountpoints() derives every tmpfs
mountpoint whose innermost enclosing volume of the same service is
NFS-backed, with its declared mode; a nested tmpfs without a mode, or
one mountpoint declared with two modes, raises.
- plugins/lookup/nfs_tmpfs_mountpoints.py feeds it the rendered compose
volumes, so compose_volumes stays the source of which volumes live on
NFS.
- sys-svc-compose nfs_prep.yml creates those mountpoints on the NFS
server with the declared mode before every stack deploy, which also
repairs a mountpoint an earlier deploy left at 0755.
- web-app-weblate: the cache volume declares nfs uid 1000, gid 0,
mode 0770; a pre-created mountpoint makes the volume non-empty, so
Docker no longer copies the image's /app/cache root in.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cursor#0 (CI run 40979252521, debian host) installed the pinned cursor_3.27.6 deb on pass 1; its postinst registers Cursor's apt repo, so pass 2's dist-upgrade moved cursor to 3.27.22, and the plan's ansible.builtin.apt deb=<3.27.6> then failed with 'A later version is already installed': the apt module skips only an equal version and otherwise fails or, with allow_downgrade, downgrades. package_install now asks the target before such a call: dpkg-deb --field reads Package and Version from the checksum-verified deb, dpkg-query reads the fully installed version, and dpkg --compare-versions decides; at or above the pin the call is skipped with changed false, so the deploy-time upgrade may move vendor packages forward and the pin never pulls them back. Any query that cannot be read falls through to the apt install, which decides loudly. The dnf path already skips an older local rpm, so both families now behave alike.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main CI run 40976399687, socialhome#0 (swarm, tor, centos): the CLI tests
of svc-net-tor (onion:25 "socks CONNECT failed with reply 0x01") and
web-app-stalwart (probe_mail relay: ConnectionRefusedError on port 25)
failed. dockerd on the manager had aborted the whole ingress program of
the mail stack:
Failed to add ingress: failed to program ingress ports:
failed to bind host port 0.0.0.0:993/tcp: address already in use
/proc/net/tcp named the squatter: 192.168.244.10:993 -> :2049, the
node's own NFS client. The mount carries no noresvport, so the kernel
drew a source port from the 665-1023 reserved window and landed on
imaps. None of 25/465/993/995/4190/8095 got a DNAT rule or a
docker-proxy listener while every other published port did, and
stalwart still reported healthy because its healthcheck runs inside the
container. 993 and 995 are the only published ports inside that window,
so this is luck of the draw on any NFS-backed swarm host that runs the
mail role.
mount_opts now emits noresvport. insecure on the kernel-flavor export is
the other half: kernel nfsd defaults to secure and answers a mount from
an unprivileged source port with EACCES. It reaches the kernel flavor
only; ganesha already accepts unprivileged clients.
What it costs: the next deploy of every existing NFS client sees a
pending opts change and remounts the shared state; a host-side job
holding that path at that instant fails loudly. A plain remount may keep
a running mount on its old reserved port until the next real mount.
Port of f46c4d0bab from feature/store-i18n, applied identically so the
later store-i18n -> main merge stays conflict-free.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The svc-cache-package CLI probe GETs every declared route path through the frontend and fails on 502/503/504. registry.ollama.ai declared no paths, so the probe asked for /, which upstream now answers with 503; Nexus relays an upstream 5xx as its own 503, so package#0 (CI run 40979252521) failed with 'registry.ollama.ai/ on port 443 answered 503'. The host now declares /v2/, the registry API root every ollama pull goes through (manifests and blobs); upstream answers it with 404, which Nexus relays as 404 and the probe accepts. On a TLS host the frontend still proxies the whole host through location /, because upstreams.conf.j2 reads routes only for plain-HTTP listeners: the rendered upstreams.conf is byte-identical before and after.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Parallel compose slots, swarm lab clusters and GUI lanes each kept their own pip, npm and pacman downloads, and a failed swarm run left its cluster (about 25 GB of node docker data plus the rescue bundle in the act container) on the root filesystem until someone ran make swarm-down. INFINITO_CACHE_SHARED_HOST_PATH (default <DIR_CACHE>/shared) is now mounted read-write at the same path into every compose slot and every swarm lab node, with PIP_CACHE_DIR and npm_config_cache below it; empty, the renders stay byte-identical to before. On a host whose os-release ID is exactly arch running an arch stack, INFINITO_CACHE_HOST_PACMAN_PATH mounts the host pacman cache read-only and scripts/docker/cache/pacman.sh makes it the first CacheDir with <shared>/pacman/ second, so new downloads land in the shared folder and the host cache stays untouched; on Manjaro and every other host pacman uses the shared folder alone.
make swarm-zombie without keep=true now copies the act job's rescue diagnostics and Playwright artefacts to the host and then releases the lab nodes, the lab network, the NFS export volume and the act outer container, on success and on failure; when the copy fails the nodes still go, the act container stays and the run exits naming make swarm-down. make roundtrip passes keep to every swarm step, make clean-cache removes the shared folder, and the disc-space cleanup skips pacman cache directories it cannot write. apt and dnf keep their per-container caches behind the package-cache proxy, and no docker data root is shared between daemons. Requirement 065.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rsync 3.5.0 (Debian security update 3.5.0+ds1-0+deb13u1) refuses a
destination symlink owned by neither root nor the calling user.
svc-opt-ssd-hdd turns a volume's `_data` into a symlink to /mnt/hdd or
/mnt/ssd. The restore drill empties the volume and runs `compose create`;
docker repopulates it and lchowns the `_data` symlink to the image
directory's owner (e.g. www-data). The restore then failed with
"refusing to follow a symlink owned by an untrusted user" and
"change_dir#1 .../_data/ failed: Too many levels of symbolic links",
the drill left its subjects down, and the dependent checks timed out
(CI run 40979252521, ssd-hdd#0, on feature/store-i18n; recover.py is
identical on main).
VolumeRecovery now resolves the mountpoint before handing it to rsync:
os.path.realpath locally, `ssh <host> realpath -e -- <mountpoint>` for
--docker-host, so rsync writes into the real directory and never meets
the symlink. `--insecure-links` was not used because older rsync builds
on other distributions lack it. Two new unit tests (local symlinked
mountpoint, remote resolution); both fail against the previous code.
Root cause and fix by the i18n session (feature/store-i18n triage of
ssd-hdd#0), ported to main.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A logo whose solid pixels share one colour (alpha >= 128, every channel within 24) is detected once per file and tinted to the text colour of each slot: frame_fill on frame slots, fill elsewhere, plus a prefers-color-scheme: dark variant in the SVG of page slots and the page text colour for the favicon. Edge pixels below half opacity are ignored so antialiasing does not count as a second colour. Multi-colour logos render byte-identical to before.
make design-logos (alias i8delo) renders every role with a design entry from one logo and base colour into an HTML page with day and night tiles, without a deploy. Slots render in four threads and roles in one process per CPU: the full preview went from 287.6s to 33.2s with identical output (diff -rq over 1229 PNGs). A GPU does not pay off for these PIL resizes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Addresses three CodeRabbit findings on PR #1265.
1. LAM hardening invisible to the probe (thread 4413764929): the runtime
check probes PHP with `php -r`, the CLI SAPI. web-app-lam mounted the
hardening ini only into /etc/php/8.4/apache2/conf.d, which the Debian
CLI does not scan, so the probe read unhardened CLI settings and the
check failed for LAM. The php_hardening entry now also mounts the ini
into /etc/php/8.4/cli/conf.d. No other role mounts it into a Debian
per-SAPI directory; the rest use the shared scan dir of the official
image or of Alpine. The TODO entry now names the remaining gap: the
probe reads only the CLI copy.
2. Explanatory comment above HEALTH_RUNTIME_FLOORS (thread 4413764949):
removed. The README already states the end-of-life rule and now also
defines a floor as the lowest patched release of each line.
3. `apk --no-cache add` missed (thread 4415018662): PACKAGE_INSTALL_PATTERN
required `add` right after `apk`, so `RUN apk --no-cache add php7`
passed as an image without PHP. The apk branch now accepts options
before `add`, like the apt branch. New tests cover the option form and
guard that `apk` in a path or a comment is no package install.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
make autoformat output for three lines the feature/updater merge brought
in over 88 characters; matches feature/updater-lockdown-probe 88d138ba1c.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in feature/updater accb22da75, which carries main 5995766f58.
Conflict resolutions:
- web-app-fider: main's tenant bootstrap (01_tenant.yml) runs first on
the stack host, OIDC setup moves to 02_setup_oidc.yml, design to
03_design.yml; flat includes so main's gate test reads them.
- sys-front-tls test_nginx_ssl_header: the tls fake maps the domain term
store-i18n passes back to its app.
- group_vars 04_webserver.yml, sys-svc-proxy CSP header: store-i18n's
webserver_resource lookup plus #1256's WEBSERVER_SECURITY_HEADERS and
the security_headers.lua include after the CSP block.
- web-app-nextcloud: updater's decoupled Talk stack (Janus, signaling,
NATS, recording builds) replaces aio-talk; store-i18n's per-network
Talk backends move from the wrapper script into signaling.conf.j2 urls
(NEXTCLOUD_HPB_SIGNALING_BACKEND_URLS from network_urls, canonical
first); IS_COMPOSE_MODE/IS_SWARM_MODE booleans kept; the integration
OpenProject digest stays delegated to STACK_HOST.
- users lookup docs, token-store write_many, cache README, meta info
lint: both sides combined.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
repository_files() listed every path `git ls-files` returns, including tracked files the worktree has already deleted. Every content scan that reads the listing then reported "[Errno 2] No such file or directory" as a finding: in a worktree that removed roles/web-app-nextcloud/filter_plugins/nextcloud_janus.py and its test but had not committed the removal yet, test_no_rsync_backup_flag failed on exactly those two paths and turned the whole lint target red. No scan can read such a file until the deletion is committed.
The listing now subtracts `git ls-files --deleted`. A clean checkout, as in CI, has no deleted entries, so the listing there is unchanged. The rsync lint is green again on the same worktree. The unit tests stub the extra git call and cover a file deleted in the worktree.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
services.dashboard.enabled already says whether a role takes part in the dashboard: roles_with_service lists only roles with enabled and shared as tile-reachability targets, the CSP filter adds the dashboard to frame-ancestors only when it is set, and docker_cards uses it as the card's iframe flag. A separate display key duplicated that switch and left a role with enabled false still rendering a card. docker_cards now skips the card when services.dashboard.enabled is false (a role without a dashboard binding keeps its card, and a value that is not a boolean fails loudly), so display, its validator exception and its roles_with_service filter go again; base.md documents enabled as the per-host switch.
An inventory hides a tile with applications.<role>.services.dashboard.enabled: false, which also drops the dashboard framing of that role. Unit tests cover false, true, a missing binding, a rendered string and a non-boolean; forcing the helper to always show the card fails four of them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The dashboard read `display` from the role's meta/info.yml, which no inventory can override, so a host could not hide a tile. docker_cards now skips a card when `services.dashboard.display` in the merged applications is false, and absent still means shown; a non-bool value fails loudly. roles_with_service drops the same role from the dashboard tile-reachability targets so the Playwright spec does not wait for a hidden tile, and the inventory validator accepts `services.dashboard.display` for every role that declares `services.dashboard`, since no role default carries it. `display` leaves the meta/info.yml allowlist, web-app-jellyfin drops its redundant `display: true`, and base.md and layout.md document the new switch.
Real lookups with web-app-gitea set to display false render the Dashboard and Jellyfin cards and only jellyfin as a tile target, and all three without the override; the nine tests for the new behaviour fail against the old code.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
trusted_proxies held the host's public addresses, but OpenResty reaches Nextcloud through the published port, which Docker forwards from the gateway of one of the container's networks (192.168.201.1, the central MariaDB network, on production). Nextcloud therefore ignored X-Forwarded-For, attributed every request to that gateway and brute-force throttled it. The TRUSTED_PROXIES env var was joined with commas while the image's reverse-proxy.config.php splits it on spaces, which produced the single malformed entry the admin overview rejects, and the occ value reached config:system:set as a Python repr instead of JSON, so config.php got null.
Add the proxy_hops lookup next to resolve_upstream in utils/networks/proxy.py: under compose it returns the gateway of every network the application joins, under swarm the subnets of the overlays it shares with svc-prx-openresty, both derived from the service registry and role_subnets. NEXTCLOUD_TRUSTED_PROXIES now uses it, env.j2 joins the list with spaces and system.yml passes it through to_json.
Rendered with the real roles and group_vars for the galaxyserver layout: compose trusts 192.168.41.1, 192.168.201.1 and 192.168.202.1, swarm 192.168.107.0/24; unit and integration tests cover both modes, and two mutations of proxy_hops fail them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A token store write re-rendered the whole applications SPOT: merged_applications_cache_key carried the token store digest, so the applications_carrier park in sys-token-store/tasks/write_many.yml missed its key and rendered all 269 applications again (stalwart compose roundtrip 18.3 s and 10.2 s per pass, swarm pass 1 12.2 s and 12.6 s).
Rendered from the real roles tree under an empty, a realistic and an exhaustive token store (298 users x 269 applications), the applications tree differs only under <app>.users.<key>.tokens (28783 leaves in 46 applications, 0 elsewhere); the 69 password leaves that also differ change between two renders of the same store. No code reads a token through the applications tree: every reader goes through lookup('users'), lookup('config', <app>, 'users.<key>...') (served from get_merged_users), lookup('email') or the sys_token_store_users fact, and the users carrier keys on the store.
The token digest therefore moves from merged_applications_cache_key into merged_users_cache_key, whose value stays the same. A store write keeps the applications carrier and the write_many park re-renders the users alone; the task is renamed to match. applications.md now says that user records inside the payload can predate every store write and that application config must not derive a value from a token.
Real ansible-playbook harness on 2.20.6 with the real roles, group_vars and write_many.yml: the re-park after a first mint and after a rotation drops from 4.0 s to 0.55 s on one host (applications renders 1 -> 0), and every token probe (users, config users path, email password, stalwart_users) equals the old code in compose and swarm mode. Dropping the digest from the users key as well leaves those probes stale, which the harness and two unit tests catch.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ansible forks a fresh worker per task, so every lookup('users') in a forked worker re-rendered the whole user catalogue (298 users, 0.3 to 0.5 s locally and about 1 s on the runner, 1 s locally once a users password is vaulted): _MERGED_USERS_CACHE lives in the process and had no carrier fact.
applications_carrier now also parks lookup('users', carrier=True) in _INFINITO_USERS_RENDERED at the same park points, and clear drops both facts. get_merged_users serves the fact to templated reads when its key matches: the applications carrier key (roles dir, applications, users, DOMAIN_PRIMARY, SYSTEM_EMAIL_DOMAIN, group_names, token store digest) plus inventory_hostname, so a carrier a run_once park broadcasts never answers another host with its become password. A park whose users carrier still matches leaves it in place instead of returning 119 KB again, so parks cost what they cost before.
Real ansible-playbook harness on 2.20.6 and 2.21.5 with the real roles tree: users renders per users-reading task after a park 1 -> 0, task 0.56 -> 0.08 s (vaulted 1.21 -> 0.17 s); a rotated token, a rotated inventory password, a cleared carrier and another host's carrier all re-render; 32 of 32 probe digests equal HEAD's. Removing the carrier check fails three unit tests. The stalwart swarm roundtrip pass 1 and its DR drill ran green with it on all nodes, with no 401 from JMAP, msmtp or LDAP.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Three CodeRabbit findings on PR #1265:
- script.py listed the running containers with `container ps` and
`container inspect` without a timeout, so a hung engine kept the daily
unit active forever and blocked the next timer run. Both calls now use
EXEC_TIMEOUT_SECONDS; main turns a timeout or engine error into a
one-line failure and exit 1 instead of a traceback.
- runtime.py's PHP_PACKAGE_PATTERN requires a digit after "php", so an
image installing an unversioned interpreter package (`php-cli`) and
setting no PHP_VERSION was reported as PHP-free. The interpreter
packages php, php-cli, php-fpm, php-cgi, php-phpdbg and
libapache2-mod-php are now recorded as PhpRuntime.packages and reported
with an unknown patch level; extension and library packages
(php-mysql, php-pear, php-common, phpmyadmin) and paths do not count.
- php_finding returned early for a patched PHP_VERSION, so an end-of-life
package line installed next to it (e.g. php7) was never reported. The
version only returns a finding now; otherwise the lines are judged too.
Every finding has unit tests that fail on the previous code.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The two tests that only answer on a GPU host, the logout catalogue translation check and the complete-catalogs integration test, were gated on accelerated() alone: a contributor with an NVIDIA runtime had no way to keep them out of a local gate while the GPU translation run is paused, so every gate on such a host stayed red on them.
INFINITO_TEST_GPU is a new env key. Its handler derives the default from INFINITO_GPU_COUNT: true where the compose services reserve a GPU, false for 0 or none. custom.env is applied after every handler, so `INFINITO_TEST_GPU=false` there switches the tests off on a GPU host; an exported value wins as well. tests/utils/gpu.py reads the flag for both tests, fails on any value other than true or false, and falls back to the same GPU detection only where no .env is loaded, such as a direct pytest run.
The handler joins gpu_count in utils/env/handlers/infinito/gpu/ (count.py, only_tests.py), because two handlers sharing the gpu prefix are a folder by the prefix-grouping lint. The new module avoids a test* name so unittest discovery never collects it. customize.md lists the key among the common overrides.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two CI failures on PR #1265 (run 40834164415):
- Chunk 0 / SuiteCRM#2 (swarm): the post-load flush included
sys-ctl-hlth-runtime on every play host, the NFS server too, which never
gets sys-svc-container and therefore has no `container` CLI. The unit
died with FileNotFoundError: 'container'. The role entry now runs only
where a container runtime exists: in compose mode, or on swarm nodes
('svc-swarm-node' in group_names), the same role-entry host scoping that
sys-ctl-hlth-csp and sys-ctl-hlth-webserver use with IS_STACK_HOST.
Workers stay covered because PHP replicas can run there.
- Workspace (ubuntu): tests/external/repository/test_urls_reachable.py
probed the literal "https://hub.docker.com/v2" from
utils/docker/image/runtime.py:20 and got HTTP 404. The constant is gone;
the endpoint is written as one f-string at the call site, the way
utils/update/docker.py:104 and utils/update/source.py do, which the URL
test skips as a template. With the scheme now visible in the literal,
ruff reported the noqa: S310 on urllib.request.Request as unused.
Both test files that imported unittest twice (CodeQL) now use one
`from unittest import ...` form.
The run_once queue leak that spreads the stack host's post-load queue to
every host is fixed separately on the updater line (70ed8a2e98); when it
meets main, sys-ctl-hlth-runtime moves into sys-svc-container's queue.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two store-i18n-only code paths predate updater's custom roles
(web-app-cst-*, d92375922d), which the merge c9e7f950d9 brought in:
- utils/i18n/libretranslate.py deploys the translation tools lane with
cli.administration.deploy.dedicated but without --skip-pull. The lane
bind-mounts the checkout and its generated inventory declares no custom
role, so `make i18n-translate` would have run pull_custom_roles and
removed every pulled web-app-cst-* from the host's roles/. It now
passes --skip-pull, as every other generated deploy does.
- utils/i18n/extract.py role_messages globbed roles/*/meta/main.yml,
including the gitignored custom roles, so a checkout with a pulled
custom role failed test_core_catalogs_hold_exactly_the_current_sources
and its remedy would have written the custom role's description into
every tracked core catalog, against updater's rule that core never
tracks custom roles. It now skips is_custom(role), like the README
generators.
test_a_pulled_custom_role_stays_out_of_the_core_catalogs pins the second
point; without the skip it fails with the custom role in the list.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
updater's TestSwarmDeployCommands (d92375922d) patched
utils.tests.swarm.matrix.run_step, which store-i18n's matrix no longer
imports: every step goes through the module's _run(cmd, *, env, label),
which wraps the watchdog. After the merge (c9e7f950d9) both tests failed
with "does not have the attribute 'run_step'".
They now patch matrix._run, as test_run_reporting and
test_extend_inventory already do, and keep their assertions: the round
deploy (cli.administration.deploy.swarm) and the backup-host deploy
(cli.administration.deploy.dedicated) both pass --skip-pull, so the
custom roles placed before the round survive. The backup-host test hands
_deploy_backup_host a temporary inventory directory, because store-i18n
reads and may rewrite host_vars there (drop_backup_network_mode) before
it deploys.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
68 conflicts, resolved per hunk; resolutions agreed with the Updater
session were applied as agreed:
- svc-db-postgres (superuser password script and core) and
sys-svc-rdbms realign_dedicated_db: updater's argv/stdin redesign with
expand_argument_vars: false, nested into store-i18n's block structure.
- svc-db-redis 01_init: store-i18n's ACL password on stdin.
- sys-svc-proxy forwarded.conf.j2: store-i18n's handover-aware file,
without the forwarded_base include (Host and X-Forwarded-Proto would
be sent twice); forwarded_base serves the inline consumers, e.g.
web-app-seaweedfs nginx.conf.j2.
- basic.conf.j2 and web-app-openproject proxy.conf.j2: issuer redirect,
then handover, else updater's location/sso_acl.conf.j2; the
server-level webserver_extra_configuration hook stays.
- web-app-keycloak: updater's admin-REST settings step and renumbered
update steps; update/01_client.yml removed; store-i18n's design step
and LDAP block kept; a73435cc9d arrived through rename detection.
- web-app-nextcloud OpenProject provisioning: updater's rewrite, all
credentials on the Ruby runner's stdin.
- web-app-semaphore: store-i18n's canonical web root (a1d7835863).
- Role READMEs: no generated Cosmos/Architecture/Quick Setup/Credits
sections; desk-* paths resolved to the dsk-* renames.
Integration fixes for what merged cleanly but did not fit together:
- web-app-matrix ansible vars: drop updater's second
matrix_client_element_configuration_extension_json, which silently
overrode store-i18n's design-aware value.
- url.origin instead of a stripped url.base in 19 places store-i18n
added since 44aaa68924, same lookup targets (the double-slash lint).
- Package batch (18ea75049e, cb4d340de9): installs the deploy-start
batch covers are removed (caffeine, cursor, blender, npm in dsk-gnt-*,
with shared references), the rest carry batch: false and a nocheck
with the verified reason (PyPI fallback, before the batch, vendor
repository, configured flavor only, addon gate); dev-npm references
the root npm id.
- sys-token-store write_many.yml: store-i18n's change-gated block and
IS_SWARM_MODE.
- web-app-openproject: IS_COMPOSE_MODE in 02_migrate/run.yml; the
seeded-admin lock (ccbf8f168d) ported into 04_admin/provision.yml and
covered by the admin-state digest.
- Bucket policy (3b4ac40df5 and follow-ups) moved to web-svc-seaweedfs,
where store-i18n's object store lives; the S3 admin secret travels on
stdin instead of a task-level environment.
- act workflow.sh: no Exception-labelled comment.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The apt module loads its whole cache before it finds that every name is installed, about 2.4 s per call; an update deploy spent ~28 s on twelve such no-op calls from roles outside the batch. package_install now asks dpkg-query, pacman --query or rpm --query under LC_ALL=C first and returns unchanged only when every plain name is proven to be in the requested state; anything it cannot read in full runs the module as before.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ruff format reflows the package batch, registry, plan and invocation code and tests and the Stalwart and Nextcloud tests, ansible-lint rewraps long Jinja lines in folded scalars of sys-service, Nextcloud and SeaweedFS, mbake restores the Makefile's final newline, and the Nextcloud OAuth test merges its nested with statements (SIM117).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Answering "which deployed containers run a vulnerable PHP?" after
CVE-2026-109618/109619 took a manual inventory of registry metadata, and
nothing noticed a hardening ini that an image never loads.
- New role sys-ctl-hlth-runtime, routed through sys-svc-compose next to
sys-ctl-hlth-container. A daily timer runs `php` in every running
container and fails, alerting through sys-ctl-alm-compose, when PHP is
below the floor of its line or on a line older than every floor (7.4,
8.1). Every service that mounts php_hardening in meta/volumes.yml fails
while expose_php or session.upload_progress.enabled is on; other PHP
containers of such a stack are listed as info. Compose and swarm
containers are matched by their project/stack and service labels. The
check is not run at deploy time, so a finding never fails a deploy.
- The floors (8.2.35, 8.3.36, 8.4.27, 8.5.12) live once, in
HEALTH_RUNTIME_FLOORS of the role's vars/main.yml.
- `make security-runtimes` lists the PHP of every pinned role image
without pulling: Docker Hub tags through the Hub API, which no pull rate
limit meters, other registries through the OCI config blob (new
registry.config_blob, which the healthcheck probe now reuses).
PHP_VERSION, distro PHP packages and a PHP_BASE build argument count. It
exits 1 on an image below its floor or one the registry did not answer.
Today it flags nfrastack/fusiondirectory:1.5 (line 7.4) and
shopware/docker-base:8.3 (8.3.25).
The role TODO.md keeps the follow-ups: further runtimes, the split web
SAPI of web-app-lam, a scheduled CI run of the report and the CVE scan.
Refs #1127
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An inventory group web-app-cst-<name> with applications.<id>.custom: true and
a pinned source {url, version, path} is pulled into roles/ before the play,
read raw (Jinja, Ansible tags and credential URLs refused), placed through a
staging directory under an exclusive lock, and owned by a marker that names
its own directory, so a pull or roles-clean never replaces or removes a
directory it did not place and a failed pull leaves roles/ as it was.
Directory inventories, host lists and non-UTF-8 files behave as Ansible
reads them; generated test deploys pass --skip-pull. make roles-pull,
roles-clean and roles list, INFINITO_EXTRA_MOUNTS for local sources, the
web-app-cst category, the reserved prefix lint and a central
repository_files() that also yields the gitignored custom roles let every
lint scan them like in-tree roles. Supersedes the uncommitted 045 External
Roles draft and reuses its fetch, sync and mount design.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A constructor task after the update role and the swarm auto-placement installs,
in one package-manager call, every package id of every role the host's groups
may run: meta dependencies, service edges and every role invocation readable
from tasks and handlers, conditions ignored. Ids that must stay with their role
(a conditional install, a repository added first, a need before update) carry
batch: false in meta/packages.yml and keep their task; roles reference shared
root ids with shared: true. 42 per-role install tasks go, and a lint keeps the
split: no batched id in a role task, none in a role run ahead of the batch.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The carrier key switched from the tokens file's path and hash to the
sys_token_store_users fingerprint once the token store loaded, so the
stage-wide park re-rendered the applications SPOT (about 11 s) even when the
tokens were identical. Both sources now sign as a digest over exactly the
tokens the users hydration reads; the store helpers move to users/store.py
to keep the package under the file length cap.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
distros.sh stops a distro at its budget with `timeout -k 60`: SIGTERM at
the deadline, SIGKILL 60 s later. dedicated.sh's cleanup trap then ran
the in-container capture under `timeout 1500` (266 s in Baserow#0 of run
40625679270) and the host capture under `timeout 900`, so the SIGKILL
always landed mid-capture, the snapshot was never copied out and the
rescue-diagnostics artifact was lost. The inner `timeout` also sat in its
own process group and outlived the kill, holding the step open.
- distros.sh keeps the grace in one place, KILL_GRACE_SECONDS, and
exports INFINITO_CI_DISTRO_KILL_EPOCH next to the deadline epoch.
- scripts/tests/deploy/utils/rescue_window.sh decides per trap step:
past the deadline, the step gets half of the time left until the
SIGKILL (at least 1 s) and an end epoch; before it, the nominal
timeout and no end epoch. Inside the trap `$?` is 0 on a budget kill,
so the deadline epoch is the only reliable signal.
- dedicated.sh runs both captures through `timeout --foreground` with
that window, keeping them in the trap's process group, and passes the
end epoch as RESCUE_DEADLINE.
- utils/diagnostics/container.py caps every command at RESCUE_DEADLINE,
writes `[skipped: past RESCUE_DEADLINE]` for what no longer fits and
hands the deadline to nested captures. Without the variable nothing
changes, so the swarm and workspace callers behave as before.
-k 60, the distro budget and every timeout-minutes are unchanged. Drill
tests replay the trap with the real helper and container.py: a budget
kill ends with rc 124 and the snapshot copied out, the old shape ends
with rc 137 and nothing copied, an early failure keeps the full capture.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The NFS backup rsynced the whole export and honoured only one exclude, the
shared backup root. Volumes declared `backup: false` were copied anyway and
then pulled a second time by remote-2-local; in the swarm DR drill of
Magento#0 (run 40625679270) docs_sites alone (a full repo clone plus its
prepared and work trees) took about 15.5 GiB per copy and filled the
runner. The volume backup already skipped these through the backup_volume
lookup; only the NFS path ignored the declaration.
- utils/storage/nfs.py: volume_dir() names the state directory a volume
binds to (pinned name or key), shared with compose_volumes;
backup_disabled_dirs() returns every NFS-backed `backup: false` volume
dir that no backed-up volume also binds to.
- The role-local lookup backup_nfs_excludes feeds them into
BKP_NFS_2_LOCAL_EXCLUDES next to the backup root; script.sh takes any
number of excludes. Compose installs still pass only the backup root.
- recover.py protects every live state directory the snapshot does not
hold, so `rsync --delete` keeps the excluded volumes (model weights,
the docs tree, caches) and volumes created after the snapshot instead
of wiping them.
- The swarm DR drill restores BACKUP_NFS_VOLUME, the first NFS volume
not declared `backup: false`; PRIMARY_NFS_VOLUME stays for the drain
tests. BookWyrm now drills bookwyrm_media; decidim, fusiondirectory,
meta, coturn, svc-registry-cache and svc-registry-docker have nothing
backed up on NFS and skip the drill.
Measured: compose_volumes renders identically before and after for 291
roles across compose/swarm x local/nfs (1940 of 1940); the 15 excluded
directories are exactly the backup-disabled device dirs; the systemd unit
renders shell-safe on ansible-core 2.20 and 2.21 and stays byte-identical
in compose and swarm-local. A new unit test runs the real script.sh over
two generations and the real recover.py back into the export, and fails
without the restore protection.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Job and run titles read `🐳🧅…𓏴𓏛·Nextcloud#2`; the `·` between the
hieroglyph block and the title only costs width. RoleDisplayName.encode
now joins the two directly (`𓏴𓏛Nextcloud#2`). Spaces inside a README
heading still become `·`, which keeps a display name one token in the
space-separated whitelist and priority inputs.
decode and decode_list still map the dotted form of older runs back to
the role id, so report_failures, the run tooling and selection tokens
pasted from an earlier run keep working. Old and new names cannot
collide: a glyph block holds only hieroglyphs and a title starts with a
letter.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The chunk jobs read `🧱 Chunk N` and every priority row carried its own
trailing ⭐, so a priority sweep repeated the star down the whole job
list while the chunk that held those rows looked like any other.
- The four chunk jobs are named `0️⃣ Batch` to `3️⃣ Batch`, with ` ⭐`
when the batch holds priority rows, and their discover job reads
`🔍 Batch N ⭐` through the existing `marker` input.
- call-validation counts those batches once per run
(scripts/github/resolve/priority_batches.sh, which runs
`cli.meta.ci.matrix --priority-chunks`) and exposes the count as the
`starred` output. chunks.plan cuts the priority chunks from the
priority rows alone, so offset and the regular line cannot move them;
the orchestrator hands validation the same sweep and chunk size it
hands the chunk jobs.
- The row label drops its own star. report_failures and the job-name
parsers still accept a starred title from an older run.
- The glossary names the unit `batch` (`#️⃣`) instead of `chunk` (`📦`),
`ci plan` prints the batch column as the keycap the job name leads
with, and the name lint accepts a leading keycap emoji.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_backup_stop_decision_declared looked a volume mount's `service` up
as a meta/services.yml key. A role whose compose service is renamed via
`proxy.service_key` slipped through: web-svc-xmpp mounts `ejabberd`, its
entry is `xmpp`, and the lint never asked it to decide.
- compose_service_names() in utils/roles/applications/mounts.py is the
shared mapping from a meta key to the compose services it renders as
(the key plus a literal proxy.service_key); the lint resolves each
backed-up mount to its owning keys through it, and
test_image_volumes_mounted uses it instead of its own copy.
- web-svc-xmpp stops before the copy (`/opt/ejabberd/database` is a
Mnesia store spread over several files), which is what the missing
marker already did. Its upload volume is no longer backed up: the
ejabberd config enables no mod_http_upload, so nothing writes there.
- web-app-pretix deploys pretix_custom:3.13, the reference
lookup('backup_image', 'stop_required', false) resolves, instead of
overriding the tag with the source version. The override made the
deployed image miss --images-no-stop-required, so baudolo stopped
Pretix despite `stop_required: false`. The source version moves into
the build args, which keeps a bump rebuilding through the build
digest, and into an OCI version label on the image.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every role's base services carry `docs.enabled: "{{ 'web-app-docs' in
group_names }}"`, and the static closure counts any "in group_names" flag
as possibly true (is_explicit_truth). A round therefore pulled web-app-docs
in through any member whose active variant did not pin docs off, and a
round hands every role its index or clamps it to variant 0, where 84 roles
declare docs on. The README replay and everything it drags in (dashboard,
matomo, keycloak, prometheus with its MCP partners) landed in 365 of 469
rounds instead of one per role, which ran CI run 40625679270 out of
resources: Listmonk#0 memory, OpenProject#0 I/O in the DR drill,
Nextcloud#0/#2 budget (4310-4539 s in the replay each, 60 s over),
BookWyrm#0 drill disk, Matrix#1/#3 pids in the replay; none of those
primaries declares docs in the failing variant.
The guide host now stays on only where the primary app's own variant
declares it (declares_guide, the predicate instructions.py already used);
every other role has it switched off (guide_off), in the planner's round
overrides and in the swarm include set. The host_vars bake drops a docs
edge whenever the round does not deploy web-app-docs. Measured for
listmonk, openproject, translate and mariadb: web-app-docs is deployed in
exactly each primary's declared guide round, swarm and planner agree in
all 12 rounds, and rounds shed up to ten apps (mariadb#1 19 -> 9).
test_guide_rides_the_primary_only pins that on real role data (it fails
with the rule disabled); test_variant_pin_required leaves the guide host
out of its clone comparison and no longer counts the docs key as a
pairing edge, because the replay no longer follows a partner's index.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
translate#1 (CI run 40625679270) hit the disk floor and the watchdog
returned rc 75, but its proc.terminate() reached only the step's python
wrapper, which dies on SIGTERM without passing it on. ansible-playbook was
re-parented to PID 1 (rescue processes.txt) and kept deploying and testing
for another eleven minutes, ate the rescue headroom, and was SIGKILLed by
the teardown, which surfaced as a misleading rc 137.
The step now runs in its own process group; an abort signals the group
(SIGTERM, 30 s, SIGKILL). SIGINT and SIGTERM sent to the watchdog itself
are forwarded to that group while the step runs, so the outer `timeout`
in 00_one.sh (no --foreground) still stops the step.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A variant that pairs a clearnet-only partner with the onion (Matrix
variant 2 with web-svc-coturn) could draw the tor-only network mode, and
the deploy's domain map then aborted with "web-svc-coturn: allows
['clearnet'] but the node runs in 'tor' mode". disabled_services() now
also emits every deployed partner with domains and a Tor bond whose
networks the node mode does not serve, mirroring utils/cache/domains.py,
so the inventory provision drops it the same way it drops Tor on
clearnet rows. Measured across every role: 8 tor-only rows change, all of
which failed at that point before; clearnet and multi rows are untouched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
16 conflicts, resolved per hunk:
- sys-stk-front-proxy: tasks/main.yml keeps the network-sibling loop;
the single stack_host_contents write of http.conf plus vhost and the
front_proxy_vhost_changed gate move into tasks/vhost.yml, with the
http.conf decided per sibling domain so an onion vhost gets no HTTPS
redirect.
- sys-front-tls: 01_http.yml is gone as the updater intends; the
cert-renew include in main.yml gates on the vhost domain's TLS.
- sys-svc-compose: the updater's resolve_host_cid without the remote
echo and the prepared pending flag in up.yml, written with
IS_SWARM_MODE and the repeated-when block grouping.
- web-app-nextcloud: the updater's post-upgrade gate and pending-only
plugin configuration, which already carry the pinned-plugin handling.
- web-app-xwiki: one XWIKI_CORE_VERSION for the readiness probe and
style.css.j2; OIDC/LDAP follow xwiki_extensions_ready at bootstrap,
the S3 blob store stays off there as TestBlobstoreRenderOrder demands.
- roles_with_service: onion swap and dashboard card key combined, the
canonical domain is the canonical URL's host in both branches.
- sys-ctl-hlth-csp: --clearnet-timeout and --concurrency together;
run_checker forwards its options to build_docker_cmd, keeping the
script under the 500-line limit.
- web-app-matrix: Element branding and web_client_location side by side.
- web-app-stalwart/README.md: the generated Cosmos section is dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every app's compose pass ran three remote tasks before compose up: create the directories, compose ps for running containers, and stat the pending marker (53 apps x ~2.7 s in a production run). compose_instance_prepare does all three in one call from 02_stack_host_prep.yml, sharing the directory creation with directory_batch through utils/directory_batch.py; compose ps runs only in compose mode. up.yml evaluates the registered result, named compose_instance_before_files because it predates 04_files: the running check is skipped whenever the compose, env or override file changed in this run (those changes notify compose-up themselves).
Replay harness old vs new: rendered files and the compose-up, marker and swarm handlers identical in every scenario; the only difference is the skipped running check when only the override changed, where compose-up still runs. About 85 s per full production run.
Merge note: store-i18n's a4c7fda974 hoists the removed marker stat into a block; resolve up.yml with this side.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The dashboard card of every app opened its primary domain, so Matrix led to the Synapse status page and Stalwart to its admin UI. A role can now declare services.dashboard.card: <key of meta/domains.yml canonical>; docker_cards builds the card URL with lookup('canonical_url', app, key), the Apps menu reuses it, and roles_with_service('dashboard') hands the same domain to the tile e2e spec. Matrix sets card: element, Stalwart card: webmail. Without card nothing changes, and the prometheus/matomo target lists are untouched. A lint requires the key to resolve in the role's own domains.canonical.
The Matrix ansible flavor also sets Synapse's web_client_location to Element through matrix_synapse_configuration_extension_yaml, as the compose flavor already does, so matrix.<domain>/ redirects there. Merge note: feature/iam-profile-rbac defines the same extension variable inside the OIDC block; the next merge must combine both into one block (agreed with that session).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
write_catalog writes the rendered catalogue to .<name>.po.tmp and moves it
over the target. When the write fails, the staging file stayed behind: the
translation run that hit a full disk left eleven empty .docs.po.tmp files
untracked in locale/. Any failure between the write and the move now
deletes the staging file and re-raises, so the target keeps its old bytes
and nothing is left in the tree.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
