A custom role with a local source (`source.url: /opt/src/custom`) is copied with shutil.copytree, which skipped only .git and __pycache__. A dev checkout that Claude Code has worked in carries an empty `.claude/.cc-writes` (mode 0700, ignored by the global gitignore), unreadable for the runner uid, so `infinito administration deploy dedicated` aborted before the play with "Custom roles refused: ... [Errno 13] Permission denied". A git source never carries that directory, because it is staged through `git archive`.
The copy now skips `.claude` as well. The ignore list has one consumer, the local branch of stage(); git sources are unchanged. A new test creates `.claude/.cc-writes` at mode 000: before the fix it fails in shutil with the reported Permission denied, after it all 102 custom-role tests pass. Reported by the GUI lane on efd72474a6.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Fresh compose deploy (compose-deploy-req061-4.log) spent 764.9 s in the plugin phase. Two parts of it did no work:
Enable stage after a fresh install: occ app:install runs enableApp and prints "<app> enabled" (core/Command/App/Install.php, stable34), so the app:enable that 03_enable.yml ran next only answered "<app> already enabled". The include, the exec, the classify, the skip notice and the queue cost 90.9 s for the 26 plugins installed fresh. nextcloud_install_status now reports enabled when occ exited 0 and printed an "<app> enabled" line; 02_install.yml then queues the plugin for the configure stage directly and runs 03_enable.yml only for an install that left the plugin off ("already installed" from a pinned archive or the recognize background job, or a future --keep-disabled). Queue order is unchanged.
Configure stage: 49 pending config:app:set calls ran as 49 occ processes in one sh -s and took 62.8 s, about 1.28 s of occ boot each. files/php/utils/occ_batch.php boots the console the way Nextcloud's console.php does (lib/base.php, OC\Console\Application::loadCommands, setAutoExit(false)) and runs Application::run once per call, so every call goes through the same command object, app loading, ConsoleEvent and exit code as occ, with one boot. The filter nextcloud_occ_config_set_batch builds the argv per entry; a unit test proves it equals shlex.split of the former shell line for typed, quoted, multi-line, empty and non-ASCII values. The calls travel as base64 JSON assigned in front of the script on stdin, so no value reaches a host or container argv; the first failing call stops the run and names appid/configkey on stderr, the task still reports changed on "set to".
Expected saving per fresh deploy: about 74 s (enable chain, net of the skipped include and the new set_fact, 0.64 s per plugin) plus about 60 s (configure exec down to one boot), about 134 s of 764.9 s. A converged mode=update deploy installs nothing and writes about 3 values, so it saves about 2.5 s; only a live deploy can confirm both numbers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
make autoformat rewraps the firewall-backend entry in test_userland_proxy.py, the _direct_media call in test_talk_direct_media.py and the _git listing context in test_files.py, which ran past 88 characters. The text is ruff's own output, the same store-i18n commits, so the next merge between the lines resolves cleanly.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On a clearnet deploy with self-signed TLS, test-talk-recording.js failed three times: the call recording stayed at state 4 (starting audio) because the recording server could not notify Nextcloud. Its log showed CERTIFICATE_VERIFY_FAILED on POST /ocs/v2.php/apps/spreed/api/v1/recording/backend, although compose-ca sets REQUESTS_CA_BUNDLE to the mounted bundle. nextcloud-talk-recording's BackendNotifier sends through `session.send(prepared, verify=True)`, and Session.send() skips the environment merge that only Session.request() does, so requests falls back to certifi's bundle, which does not know the injected root CA. A probe in the recording container's network namespace verified Nextcloud with the mounted bundle (200) and failed without it.
When the CA is injected, the recording image now points certifi's cacert.pem at the mounted CA bundle (the system CAs plus the injected root), in the same build step as the Firefox certificate policy. Verification stays on. Without an injected CA, as with real certificates, the build arg is empty and the image is unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Conflict in roles/web-app-gitea/tasks/01_manager_ops.yml: main (c59e2838d6, c2577560f8) moved the readiness probes inside the Gitea container on its hosting node, because a host port lands on the swarm ingress mesh; this line (381c3cd5bc) had reduced them to one wait for /api/v1/version = 200 and dropped the explicit `gitea migrate`, which raced Gitea's own migration on galaxyserver. The resolution keeps main's in-container probe as that single wait with `curl -fS` (200 required), after the resolve and with this line's 600 s budget scaled by lookup('timeout'), and drops main's app.ini wait, its migrate task and its post-admin readiness wait.
main's new tests/unit/python/roles/sys-front-tls/templates/test_nginx_ssl_header.py renders nginx_ssl_header.j2 without `domain`, which ssl_credentials.j2 needs on this line since 47160d755e; its render() now passes the vhost's domain.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
00_core.yml included 01_setup_oidc.yml only when services.sso.enabled
was true, but that file carried the whole tenant bootstrap: wait for
HTTP, count tenants, POST /_api/tenants, create the admin user and
activate the tenant. Only its last task configured the Keycloak
provider. With SSO off no tenant was ever created.
Fider's RequireTenant middleware then answers every tenant route,
/api/v1/tags included, with 307 to /signup, and /signup is open to any
visitor, who can claim the fresh installation and become its
administrator. The MCP read fider_list_tags got that 307 in the swarm
SSO-off matrix row.
The tenant bootstrap moves unchanged into tasks/01_tenant.yml, which
00_core.yml includes on the stack host regardless of SSO. The OIDC file
becomes tasks/02_setup_oidc.yml and keeps only the provider upsert,
still gated on services.sso.enabled. The README states which bootstrap
steps run without SSO.
A unit test evaluates the include and task guards for every SSO,
stack-host and deploy-mode state: a fresh stack host posts exactly one
tenant with and without SSO, an existing tenant is never posted again,
the Keycloak provider is written only with SSO, and a non-stack host
runs none of it. The compose matrix row with SSO off stayed green
because nothing there read a tenant route.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nextcloud/aio-talk bundled signaling, Janus, NATS and an onboard TURN server behind a start script that rewrote their configs, and AIO publishes only build stamps, so the updaters could not raise it and the role had to guard Janus against config drift. The backend now runs as talk (Janus built from a meetecho/janus-gateway tag, files/Dockerfiles/Janus), talk_signaling (upstream strukturag/nextcloud-spreed-signaling image), talk_nats (official nats image) and talk_recording (built from a nextcloud-talk-recording release tag, files/Dockerfiles/Recording), each semver-pinned, single-replica, healthchecked and resource-limited. The Dockerfiles take every value as a build arg from compose, and Janus and the recording server get their config path through `command:`.
The role renders janus.jcfg, the signaling server.conf and the recording server.conf; they match what AIO stamp 20260409_094910 generated apart from the service addresses and the session keys, which are now derived from the signaling secret. Secrets reach the services only through those files. Janus may join the host network in compose mode, which keeps direct media available there, and an assert refuses host mode in swarm. talk and talk_recording keep their keys, so one deploy replaces the AIO containers in place.
The onboard TURN server is gone with its ports, vars and credential, so Talk uses web-svc-coturn only, and the deploy now stops when Talk is enabled without it instead of handing out no STUN or TURN server. The recording image installs the injected root CA as a Firefox policy when the CA is mounted, so recordings work against self-signed TLS, and the recording scratch volume stays off NFS. The Janus drift guard and the janus_config_drift filter are removed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
services.talk.direct_media was an expression in meta/services.yml that read ansible_facts['all_ipv4_addresses'] and nested config lookups. lookup('config') renders it where the applications are merged without host facts; the render fails there, the lookup hands back the raw expression and `| bool` coerces it to False with a deprecation warning. Direct media therefore never switched on: both the first requirement-061 deploy and CI run 40877408319 logged the coercion, and a render in the development container returned the raw string without facts and true with them.
meta/services.yml keeps only the inventory switch, now `direct_media: false`, so every Talk stream goes through web-svc-coturn unless an operator opts a host in. That is the behaviour the bug produced everywhere, so no host changes on deploy, and it keeps Janus off the public network by default. vars/main.yml decides in task context from the switch, Talk, standalone TURN, the bridged network, compose mode and NEXTCLOUD_HPB_HOST_ADDRESSES, which reads the facts the way web-svc-coturn does. The flag tests render that chain against the merged applications: off by default, on for an opted-in compose host with coturn and an owned address, off for host networking, a foreign or loopback address, swarm, onboard TURN and missing facts. Administration.md documents the opt-in.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With the userland proxy on, Docker starts one docker-proxy process for every published port. A published range pays that per port: Janus's direct-media range showed 74 proxies for 64 UDP ports plus the other published ports in the development stack, at about 1.7 MB unique memory (PSS) each, which projects to roughly 1.7 GB for the default 1000-port range and the same again for OpenTalk's.
daemon.json now sets "userland-proxy": false, so iptables/nftables forwards every published port, which the CIS Docker Benchmark (2.16) also asks for. Traffic from outside already took that path, so client addresses are unchanged. Host-local traffic to ports published on 127.0.0.1 now runs through route_localnet and the loopback DNAT Docker installs. From Docker 28.0 on, Docker keeps hosts on the same L2 segment off those loopback ports; every install path of this role ships 29.x.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
repository_files() listed every path `git ls-files` returns, including tracked files the worktree has already deleted. Every content scan that reads the listing then reported "[Errno 2] No such file or directory" as a finding: in a worktree that removed roles/web-app-nextcloud/filter_plugins/nextcloud_janus.py and its test but had not committed the removal yet, test_no_rsync_backup_flag failed on exactly those two paths and turned the whole lint target red. No scan can read such a file until the deletion is committed.
The listing now subtracts `git ls-files --deleted`. A clean checkout, as in CI, has no deleted entries, so the listing there is unchanged. The rsync lint is green again on the same worktree. The unit tests stub the extra git call and cover a file deleted in the worktree.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The Validation step "Assert the base branch is fully merged into the PR
head" ran main_merged.sh, which fetches the live tip of main when the job
starts. PR runs wait in the queue for hours while image-update PRs land
on main several times an hour, so nearly every queued run failed
Validation on start and had to be re-pushed after another main merge
(PR #1265: behind_by=0 at push, behind_by=9 while its run still queued).
main_merged.sh takes a new INFINITO_MAIN_COMMIT override: when set, it
fetches that full commit SHA from the remote and requires HEAD to
contain it instead of the live tip; a set but malformed or empty value
exits 2. The Validation step passes github.event.pull_request.base.sha,
the main tip at event time, which the step's pull_request-only guard
always provides. The run tests the merge against that base, so the gate
still guarantees the tested tree contains it.
The local pre-push hook (make test-main-merged) leaves the override
unset and keeps checking the live tip: a push happens right now, so
lagging the current main is the real defect there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
services.dashboard.enabled already says whether a role takes part in the dashboard: roles_with_service lists only roles with enabled and shared as tile-reachability targets, the CSP filter adds the dashboard to frame-ancestors only when it is set, and docker_cards uses it as the card's iframe flag. A separate display key duplicated that switch and left a role with enabled false still rendering a card. docker_cards now skips the card when services.dashboard.enabled is false (a role without a dashboard binding keeps its card, and a value that is not a boolean fails loudly), so display, its validator exception and its roles_with_service filter go again; base.md documents enabled as the per-host switch.
An inventory hides a tile with applications.<role>.services.dashboard.enabled: false, which also drops the dashboard framing of that role. Unit tests cover false, true, a missing binding, a rendered string and a non-boolean; forcing the helper to always show the card fails four of them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The dashboard read `display` from the role's meta/info.yml, which no inventory can override, so a host could not hide a tile. docker_cards now skips a card when `services.dashboard.display` in the merged applications is false, and absent still means shown; a non-bool value fails loudly. roles_with_service drops the same role from the dashboard tile-reachability targets so the Playwright spec does not wait for a hidden tile, and the inventory validator accepts `services.dashboard.display` for every role that declares `services.dashboard`, since no role default carries it. `display` leaves the meta/info.yml allowlist, web-app-jellyfin drops its redundant `display: true`, and base.md and layout.md document the new switch.
Real lookups with web-app-gitea set to display false render the Dashboard and Jellyfin cards and only jellyfin as a tile target, and all three without the override; the nine tests for the new behaviour fail against the old code.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The pinned onlyoffice archive was unpacked with tar over custom_apps/onlyoffice on every deploy, so files that a newer release no longer ships stayed behind. On cloud.veen.world that left the 52 files of onlyoffice 9.15.1 (NC32 line) inside the 10.2.1 tree, and the integrity check reported them as EXTRA_FILE; the 52 equal exactly the 9.15.1 files absent from 10.2.1. replace_pinned_app.sh now unpacks the release beside the app, keeps the app when diff -rq finds it identical, and otherwise swaps the whole directory, so the next deploy heals the drifted tree and later deploys report ok. A broken archive fails before the live app is touched.
The real install task, run through ansible-playbook against a reproduced 9.15.1 plus 10.2.1 tree, changed it once to a tree identical to a clean 10.2.1 and stayed unchanged on the rerun; eleven unit cases cover identical, drifted, missing and broken inputs under sh and dash.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The admin overview reported that no server identifier is configured. Nextcloud 34 accepts 1 to 511, treats values below 1 as unset and asks for one NC_serverid per server when config.php is shared, which it is between nextcloud, cron, the task processing worker and the swarm replicas. A single serverid in config.php would let them mint identical Snowflake IDs in the same millisecond. The nextcloud service now carries NC_serverid 1 in compose and its task slot in swarm, so the check passes and replicas never share an id; the other services keep the hostname fallback.
A unit test renders the service environment in both modes and fails when the compose id leaves the accepted range.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Nextcloud 34 flags "Task Processing worker status: never run" because only occ taskprocessing:worker records core/ai.taskprocessing_worker_last_iteration, and the role ran background jobs through the cron container alone; since 32.0.7 the worker command is taskprocessing:worker, so background-job:worker would never clear the check. Add a taskprocessing service that reuses the cron image, mounts, healthcheck, restart policy and networks and runs occ taskprocessing:worker -t 60 in an in-container restart loop that backs off while Nextcloud is not yet installed, so swarm neither replaces the task every minute nor marks the stack fatal on a fresh install. It scales with the swarm topology, since task claims use FOR UPDATE SKIP LOCKED.
Rendered compose and swarm stacks only gain the taskprocessing service and validate against the compose schema; the new unit tests fail when the worker mount is dropped or background-job:worker is used instead.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
trusted_proxies held the host's public addresses, but OpenResty reaches Nextcloud through the published port, which Docker forwards from the gateway of one of the container's networks (192.168.201.1, the central MariaDB network, on production). Nextcloud therefore ignored X-Forwarded-For, attributed every request to that gateway and brute-force throttled it. The TRUSTED_PROXIES env var was joined with commas while the image's reverse-proxy.config.php splits it on spaces, which produced the single malformed entry the admin overview rejects, and the occ value reached config:system:set as a Python repr instead of JSON, so config.php got null.
Add the proxy_hops lookup next to resolve_upstream in utils/networks/proxy.py: under compose it returns the gateway of every network the application joins, under swarm the subnets of the overlays it shares with svc-prx-openresty, both derived from the service registry and role_subnets. NEXTCLOUD_TRUSTED_PROXIES now uses it, env.j2 joins the list with spaces and system.yml passes it through to_json.
Rendered with the real roles and group_vars for the galaxyserver layout: compose trusts 192.168.41.1, 192.168.201.1 and 192.168.202.1, swarm 192.168.107.0/24; unit and integration tests cover both modes, and two mutations of proxy_hops fail them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A token store write re-rendered the whole applications SPOT: merged_applications_cache_key carried the token store digest, so the applications_carrier park in sys-token-store/tasks/write_many.yml missed its key and rendered all 269 applications again (stalwart compose roundtrip 18.3 s and 10.2 s per pass, swarm pass 1 12.2 s and 12.6 s).
Rendered from the real roles tree under an empty, a realistic and an exhaustive token store (298 users x 269 applications), the applications tree differs only under <app>.users.<key>.tokens (28783 leaves in 46 applications, 0 elsewhere); the 69 password leaves that also differ change between two renders of the same store. No code reads a token through the applications tree: every reader goes through lookup('users'), lookup('config', <app>, 'users.<key>...') (served from get_merged_users), lookup('email') or the sys_token_store_users fact, and the users carrier keys on the store.
The token digest therefore moves from merged_applications_cache_key into merged_users_cache_key, whose value stays the same. A store write keeps the applications carrier and the write_many park re-renders the users alone; the task is renamed to match. applications.md now says that user records inside the payload can predate every store write and that application config must not derive a value from a token.
Real ansible-playbook harness on 2.20.6 with the real roles, group_vars and write_many.yml: the re-park after a first mint and after a rotation drops from 4.0 s to 0.55 s on one host (applications renders 1 -> 0), and every token probe (users, config users path, email password, stalwart_users) equals the old code in compose and swarm mode. Dropping the digest from the users key as well leaves those probes stale, which the harness and two unit tests catch.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ansible forks a fresh worker per task, so every lookup('users') in a forked worker re-rendered the whole user catalogue (298 users, 0.3 to 0.5 s locally and about 1 s on the runner, 1 s locally once a users password is vaulted): _MERGED_USERS_CACHE lives in the process and had no carrier fact.
applications_carrier now also parks lookup('users', carrier=True) in _INFINITO_USERS_RENDERED at the same park points, and clear drops both facts. get_merged_users serves the fact to templated reads when its key matches: the applications carrier key (roles dir, applications, users, DOMAIN_PRIMARY, SYSTEM_EMAIL_DOMAIN, group_names, token store digest) plus inventory_hostname, so a carrier a run_once park broadcasts never answers another host with its become password. A park whose users carrier still matches leaves it in place instead of returning 119 KB again, so parks cost what they cost before.
Real ansible-playbook harness on 2.20.6 and 2.21.5 with the real roles tree: users renders per users-reading task after a park 1 -> 0, task 0.56 -> 0.08 s (vaulted 1.21 -> 0.17 s); a rotated token, a rotated inventory password, a cleared carrier and another host's carrier all re-render; 32 of 32 probe digests equal HEAD's. Removing the carrier check fails three unit tests. The stalwart swarm roundtrip pass 1 and its DR drill ran green with it on all nodes, with no 401 from JMAP, msmtp or LDAP.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A Dockerfile change notifies compose-up and compose-build in one flush. The compose pull handler (pull.py) builds every buildable service with `compose build --pull`, and the Build compose handler right after it built again with plain `compose build`. BuildKit keys a --pull build and a plain build differently, so the second build missed the first build's cache and ran every RUN layer again; in the stalwart compose roundtrip that was 233 s for tor, 129 s for openldap and 62 s for simpleicons, about 420 s per fresh compose deploy. It also overwrote pull.py's image, built on the freshly resolved base, with one from the plain-build cache: in pass 2 the --pull build of openldap ran its RUN layer again (34.5 s) while the plain build right after it was fully CACHED.
Both build handlers now run `compose build --pull` and fall back to a plain `compose build` when that fails, as pull.py does, so the second build of a flush is a cache hit and a lone-service rebuild keys like the flush builds. Without a reachable registry the fallback builds from the local base exactly as before.
A scratch compose project in the stack's daemon showed the cache behaviour: --pull after --pull CACHED, plain after --pull ran the RUN layer again. The new handler test fails on the old plain `compose build`.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
test_sso.py, test_sso_oidc_plugin.py and test_sso_proxy_consumers.py evict utils.cache.applications, utils.cache and the roles packages from sys.modules to import their plugin against fresh modules, and never put the originals back. A later test module in the same worker then held two copies of utils.cache.applications: plugins.lookup.applications, imported earlier, kept calling the old one, while test_email patched and reset the new one, so its carrier tests counted 0 renders and failed depending only on which modules xdist placed in one worker.
_load_module now runs inside mock.patch.dict(sys.modules): the plugin keeps the fresh modules it was built against, and sys.modules returns to its prior state afterwards. `pytest test_sso.py test_email.py` failed test_worker_reads_the_provider_view_from_the_provider_carrier and test_worker_renders_when_the_provider_carrier_key_differs before; the three SSO modules followed by test_email.py now pass 66/66. Same patch-id as feature/store-i18n ec0fde6561.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every JMAP call templated STALWART_ADMIN_USER and STALWART_ADMIN_PASSWORD (lookup('users')) and STALWART_JMAP_URL (config lookup) again. Ansible forks a fresh worker per task, so get_merged_users re-rendered the whole user catalogue in every call: about 0.4 to 0.6 s of controller time per call, 37 to 39 calls per deploy pass.
jmap_call.yml now sets the host fact stalwart_jmap_client (url, user_password) on the first call of a run, masked like the call itself, and every call reads it. STALWART_CONTAINER_ADDRESS stays in the command, so swarm keeps resolving the running container per call; 11_oidc_restart.yml replaces that container mid-run. The include interface is unchanged: stalwart_jmap_body in, stalwart_jmap out.
A scratch harness with a fake container measured the shell task at 1.01 to 1.10 s before and 0.46 to 0.49 s after, with argv and stdin sha256 identical in compose, swarm, vaulted-password and two-host runs. The admin password only rotates between runs, so a per-run fact stays current.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The nested carrier in tasks/utils/load/app.yml inherited run_once from the apply of sys-stk-full's front-proxy include, so ansible-core wrote the stack host's carrier, keyed with its group_names, to every play host. Workers and the NFS server then missed their own key and re-rendered the whole applications SPOT on the next carrier and on every task that read it. run_once: false keeps each host's carrier.
The email lookup's onion check renders the provider host's group view, a key no carrier on a worker holds, so every lookup('email') there rendered the whole SPOT: 8-11 s per task, 17 tasks per swarm pass in the stalwart roundtrip. Hand the provider host's carrier, read untemplated via hostvars.raw_get, to that view; get_merged_applications still serves it only when its key matches and renders as before otherwise.
Each bug hid the other: fixing either alone only moves the misses. Together they remove about 142 s per swarm pass. With real ansible-playbook and HostVars the worker email task drops from 3.97 s to 0.77 s on ansible-core 2.20.6 and from 4.22 s to 0.70 s on 2.21.5 with the same result; the new tests fail with the reuse disabled.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The update started one kcadm JVM per step: the client and both realms' mail settings as async jobs with their reaps, then each declared client in turn, about 27-31 s in a Nextcloud deploy. realm_settings.py reads the client scopes, clients and both realms once over the admin REST API, creates what is missing and writes a client or realm only when it drifted; secrets reach it on stdin through the new rest_script_stdin parameter. The remaining update steps are renumbered and every reference follows.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`make autoformat` rewrites roles/web-app-seaweedfs/vars/main.yml and
tests/unit/python/roles/web-app-stalwart/test_store_guard.py on every
checkout of main, so every quality gate on every branch leaves them as
unstaged changes, and the gate lints a tree the commit does not carry.
The YAML change folds one line of the `>-` scalar
SEAWEEDFS_S3_ADMIN_CURL; the parsed value is identical. The Python
change is ruff's line wrapping only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stalwart v1.0 refuses an existing account with primaryKeyViolation; the batched provisioning from 1fbd4758e3 only knew alreadyExists, so every update deploy skipped the password sync and the listed-id fallback hid the refusal (msmtp then failed with 535). An account now counts as existing when the refusal's objectId is the listed account of that name, any other refusal fails the assert, and the update patches credentials/<k>/secret instead of replacing the whole credentials list, which would have dropped every app password, API key and second factor. sys-token-store gains write_many.yml, so all bot tokens land in one store write and one applications re-render; write.yml delegates a one-entry batch to it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Stalwart, Mailu and LDAP provision every bot and account holder of the whole users catalogue, but the generator pinned only the users of the resolved roles. The bots of undeployed roles (no-reply, blackhole, bounce, newsletter, helpdesk, contact) fell back to strong_password, got a new secret on every deploy, rewrote the token store six times per run and left email.password consumers on a stale value. The generator now also pins every catalogue user holding an account or the bot role unless a role states its password outright, and reads policies independent of declaration order; existing pins are never overwritten.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The apt module loads its whole cache before it finds that every name is installed, about 2.4 s per call; an update deploy spent ~28 s on twelve such no-op calls from roles outside the batch. package_install now asks dpkg-query, pacman --query or rpm --query under LC_ALL=C first and returns unchanged only when every plain name is proven to be in the requested state; anything it cannot read in full runs the module as before.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ruff format reflows the package batch, registry, plan and invocation code and tests and the Stalwart and Nextcloud tests, ansible-lint rewraps long Jinja lines in folded scalars of sys-service, Nextcloud and SeaweedFS, mbake restores the Makefile's final newline, and the Nextcloud OAuth test merges its nested with statements (SIM117).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
select-admin-state.sql joined select-ldap-auth-source-exists.sql under the shared 'select' prefix, which the prefix-grouping lint rejects. Both now live in files/sql/select/ and their three readers point there.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The admin step ran its Rails runner on every deploy. It now digests its sources, its inputs and the administrator and settings state OpenProject holds, compares that with the admin-provision.sha256 stamp in the instance directory, and runs only on a mismatch or when services.openproject.admin.force is set.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The dashboard sent script-src 'unsafe-inline' from two sources. Its own
meta/csp.yml granted it to script-src-attr, and build_csp_header adds it
to script-src-elem and script-src-attr for every app with
services.logout.enabled. Both reach script-src through the family union,
and the token in script-src-elem also closes the hash gate, so the
hashes that sys-front-inj-javascript, -dashboard, -logout and -matomo
register for their inline snippets were discarded.
build_csp_header now skips the logout grant on a directive whose
meta/csp.yml sets unsafe-inline: false. Only web-app-dashboard sets it:
on script-src and script-src-elem. Its injected snippets now run by hash
and the logout one-liner is hashed like the others. A render of
build_csp_header for every role, variant, logout on/off and tor on/off
(1712 headers) differs from origin/main only in the 8 web-app-dashboard
headers; the 86 other roles that declare a logout service render
byte-identical policies.
script-src-attr keeps 'unsafe-inline': port-ui 2.2.2 still renders
inline event handlers (menu onclick entries, openDynamicPopup,
closeAllModals, the card image onerror). It goes once the port-ui
release without inline handlers is pinned and header.yml.j2 moves from
onclick to action.
The dashboard Playwright spec asserts that script-src and script-src-elem
carry no 'unsafe-inline' on the landing page, next to its existing check
for CSP violations.
Refs #1122
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
db:migrate took about 24 s on every deploy. After a successful run the
deploy stamps the web container's image id as the comment of
public.schema_migrations; a deploy whose image matches the stamp skips the
migration, a new image, a failed run or a recreated database migrates. The
stamp is read unmasked because masking short config values would corrupt a
hex image id, and written through a DO/format() block because sqlfluff cannot
parse COMMENT ON ... IS with a placeholder.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
OpenBao keeps no record of who read which secret without an audit device,
and 2.7 refuses to enable one over the API, so the server config declares a
file device writing to stdout, which cannot block and lands in the container
log with secret values only as salted HMACs. Every deploy fails unless the
device is active, a probe secret read through the AppRole session shows up
in the log with path, operation, display_name and time, and the probe value
does not appear in plaintext. openbao becomes a regular shared provider so
consumers such as the landing page role can declare services.openbao.
Requirement 047; taken over from the feature/openbao-audit worktree.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An inventory group web-app-cst-<name> with applications.<id>.custom: true and
a pinned source {url, version, path} is pulled into roles/ before the play,
read raw (Jinja, Ansible tags and credential URLs refused), placed through a
staging directory under an exclusive lock, and owned by a marker that names
its own directory, so a pull or roles-clean never replaces or removes a
directory it did not place and a failed pull leaves roles/ as it was.
Directory inventories, host lists and non-UTF-8 files behave as Ansible
reads them; generated test deploys pass --skip-pull. make roles-pull,
roles-clean and roles list, INFINITO_EXTRA_MOUNTS for local sources, the
web-app-cst category, the reserved prefix lint and a central
repository_files() that also yields the gitignored custom roles let every
lint scan them like in-tree roles. Supersedes the uncommitted 045 External
Roles draft and reuses its fetch, sync and mount design.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A constructor task after the update role and the swarm auto-placement installs,
in one package-manager call, every package id of every role the host's groups
may run: meta dependencies, service edges and every role invocation readable
from tasks and handlers, conditions ignored. Ids that must stay with their role
(a conditional install, a repository added first, a need before update) carry
batch: false in meta/packages.yml and keep their task; roles reference shared
root ids with shared: true. 42 per-role install tasks go, and a lint keeps the
split: no batched id in a role task, none in a role run ahead of the batch.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The onion port probe gave every SOCKS connect 60s, half of Tor's
SocksTimeout (120s by default; torrc does not set it), although its own
docstring argued that abandoning an attempt before SocksTimeout is
destructive. Every attempt authenticates with fresh credentials, so Tor
builds a whole new rendezvous for it and keeps working on it until
SocksTimeout. The probe walked away first and called the port dead. The
other raw onion connects in the repository, the svc-db-* CLI tests and
test-e2e-playwright's wait_onion_reachable.sh, wait 120s per attempt.
Run 40128751852 shows how often an attempt needs more than that. Of 178
svc-net-tor CLI runs in its 89 compose onion jobs, three lost their
first attempt to the 60s timeout and connected on the second within
seconds (Shopware#1, Hugo#1, n8n#1; the CLI test took 77-79s against
the usual 1-16s), and Hugo#1's other run took 40s on a single attempt. When
the stall outlasts the 70s until the second attempt, the probe fails
while the HTTP probe reaches the same onion port seconds earlier:
OpenBao#1 (run 39166850273, job 118050491920, the original failure of
#1048) and Socialhome#1 (run 40088473634, job 121019758461). In the
Socialhome job tor.log holds no line for either attempt, because Tor
never reached its own verdict.
ONION_PORT_TIMEOUT is now 130: SocksTimeout plus room for Tor's
once-per-second expiry sweep, so an attempt ends with Tor's own answer
(reply 0x06, and a "Giving up" notice in tor.log, which the rescue
artifact captures) instead of the probe's. The second attempt now starts
at 140s and ends at 270s, so the probe rides out every stall the old
budget did, and any stall shorter than about 260s instead of about 120s.
A green run still costs one connect per port, and a refused or unmapped
port still fails at once. A unit test pins the window above Tor's
SocksTimeout and fails as soon as torrc sets its own.
That doubles what a dead port costs, and meta/tests.yml caps the whole
CLI test. The largest compose onion deploy in run 40128751852 probes 19
domains and 5 ports; with everything dead that is 19 x 350 + 5 x 260 +
10 = 7960s, beyond the 7200s that 069df39027 set for the same reason.
9000 keeps the full failure output for up to 22 domains.
Closes#1214
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Listing, reading, uploading and activating the Sieve forward scripts each run
as one request for every forwarding account, with one method call per account
addressed through its accountId by the recovery admin, at most
STALWART_JMAP_MAX_METHOD_CALLS per request, instead of one request per account
under act-as. The account lookup joins the limits read, a missing account
stops before the limits are raised, and only changed scripts are uploaded.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The carrier key switched from the tokens file's path and hash to the
sys_token_store_users fingerprint once the token store loaded, so the
stage-wide park re-rendered the applications SPOT (about 11 s) even when the
tokens were identical. Both sources now sign as a digest over exactly the
tokens the users hydration reads; the store helpers move to users/store.py
to keep the package under the file length cap.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
#1115 raised HSTS to one year but set no X-Content-Type-Options,
Referrer-Policy, Permissions-Policy or HSTS includeSubDomains/preload,
because none of them is safe globally: nosniff lets ORB block no-cors
responses typed text/html, includeSubDomains on the apex covers hosts
the platform does not serve, and a server-level add_header is dropped
in every location that sets its own.
WEBSERVER_SECURITY_HEADERS (group_vars/all/04_webserver.yml) now holds
the platform values, all opt-in: nosniff false, no Referrer-Policy, no
Permissions-Policy, HSTS max-age one year without includeSubDomains or
preload. A role overrides any key under security_headers in its
meta/server.yml, an inventory under
applications.<role>.server.security_headers.
The new security_headers filter merges both, rejects unknown keys,
wrong types, unknown Referrer-Policy tokens, non-printable header values
and preload without includeSubDomains or below a one-year max-age, and
composes the HSTS value.
- nosniff, Referrer-Policy and Permissions-Policy are set in the Lua
header filter of content_security_policy.conf.j2, next to CSP, so a
location's own add_header cannot drop them. Referrer-Policy is only
set when the upstream sends none; Permissions-Policy only on HTML
documents.
- HSTS stays the single server-level add_header in nginx_ssl_header.j2,
now with the per-app value. A second emitter in Lua would duplicate
the header. It stays inside the TLS branch, which onion vhosts never
enter, so an onion vhost gets no HSTS even when its app opts in.
No role opts in yet, so rendered vhosts are unchanged.
Closes#1123
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
forwarded.conf.j2 includes headers/forwarded_base.conf.j2 by its path from
the repository root, the way Ansible searches templates, so the test's loader
searches the root next to the headers directory.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The guard now includes store_guard/migrate.yml for a store from before 1.0
and re-reads its layout, so the walk feeds that include's result into the
final assert: a migrated store passes, one left before 1.0 still stops.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The file was named after the application, so an app served on several domains (SeaweedFS filer/master, Stalwart mail/webmail) overwrote it once per domain on every deploy. The last domain won its dashboard iframe host and Matomo site id for all others, and every deploy reloaded OpenResty for the flip-flop. The file is now <domain>.lua, the naming store-i18n already uses, and its path comes from lookup('nginx', 'files.lua', domain) for writer, reader and the Matomo reinjection. The reinjection stats the live domain's file instead of globbing leftovers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
