Every JMAP call templated STALWART_ADMIN_USER and STALWART_ADMIN_PASSWORD (lookup('users')) and STALWART_JMAP_URL (config lookup) again. Ansible forks a fresh worker per task, so get_merged_users re-rendered the whole user catalogue in every call: about 0.4 to 0.6 s of controller time per call, 37 to 39 calls per deploy pass.
jmap_call.yml now sets the host fact stalwart_jmap_client (url, user_password) on the first call of a run, masked like the call itself, and every call reads it. STALWART_CONTAINER_ADDRESS stays in the command, so swarm keeps resolving the running container per call; 11_oidc_restart.yml replaces that container mid-run. The include interface is unchanged: stalwart_jmap_body in, stalwart_jmap out.
A scratch harness with a fake container measured the shell task at 1.01 to 1.10 s before and 0.46 to 0.49 s after, with argv and stdin sha256 identical in compose, swarm, vaulted-password and two-host runs. The admin password only rotates between runs, so a per-run fact stays current.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The nested carrier in tasks/utils/load/app.yml inherited run_once from the apply of sys-stk-full's front-proxy include, so ansible-core wrote the stack host's carrier, keyed with its group_names, to every play host. Workers and the NFS server then missed their own key and re-rendered the whole applications SPOT on the next carrier and on every task that read it. run_once: false keeps each host's carrier.
The email lookup's onion check renders the provider host's group view, a key no carrier on a worker holds, so every lookup('email') there rendered the whole SPOT: 8-11 s per task, 17 tasks per swarm pass in the stalwart roundtrip. Hand the provider host's carrier, read untemplated via hostvars.raw_get, to that view; get_merged_applications still serves it only when its key matches and renders as before otherwise.
Each bug hid the other: fixing either alone only moves the misses. Together they remove about 142 s per swarm pass. With real ansible-playbook and HostVars the worker email task drops from 3.97 s to 0.77 s on ansible-core 2.20.6 and from 4.22 s to 0.70 s on 2.21.5 with the same result; the new tests fail with the reuse disabled.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The update started one kcadm JVM per step: the client and both realms' mail settings as async jobs with their reaps, then each declared client in turn, about 27-31 s in a Nextcloud deploy. realm_settings.py reads the client scopes, clients and both realms once over the admin REST API, creates what is missing and writes a client or realm only when it drifted; secrets reach it on stdin through the new rest_script_stdin parameter. The remaining update steps are renumbered and every reference follows.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`make autoformat` rewrites roles/web-app-seaweedfs/vars/main.yml and
tests/unit/python/roles/web-app-stalwart/test_store_guard.py on every
checkout of main, so every quality gate on every branch leaves them as
unstaged changes, and the gate lints a tree the commit does not carry.
The YAML change folds one line of the `>-` scalar
SEAWEEDFS_S3_ADMIN_CURL; the parsed value is identical. The Python
change is ruff's line wrapping only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stalwart v1.0 refuses an existing account with primaryKeyViolation; the batched provisioning from 1fbd4758e3 only knew alreadyExists, so every update deploy skipped the password sync and the listed-id fallback hid the refusal (msmtp then failed with 535). An account now counts as existing when the refusal's objectId is the listed account of that name, any other refusal fails the assert, and the update patches credentials/<k>/secret instead of replacing the whole credentials list, which would have dropped every app password, API key and second factor. sys-token-store gains write_many.yml, so all bot tokens land in one store write and one applications re-render; write.yml delegates a one-entry batch to it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Stalwart, Mailu and LDAP provision every bot and account holder of the whole users catalogue, but the generator pinned only the users of the resolved roles. The bots of undeployed roles (no-reply, blackhole, bounce, newsletter, helpdesk, contact) fell back to strong_password, got a new secret on every deploy, rewrote the token store six times per run and left email.password consumers on a stale value. The generator now also pins every catalogue user holding an account or the bot role unless a role states its password outright, and reads policies independent of declaration order; existing pins are never overwritten.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The apt module loads its whole cache before it finds that every name is installed, about 2.4 s per call; an update deploy spent ~28 s on twelve such no-op calls from roles outside the batch. package_install now asks dpkg-query, pacman --query or rpm --query under LC_ALL=C first and returns unchanged only when every plain name is proven to be in the requested state; anything it cannot read in full runs the module as before.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ruff format reflows the package batch, registry, plan and invocation code and tests and the Stalwart and Nextcloud tests, ansible-lint rewraps long Jinja lines in folded scalars of sys-service, Nextcloud and SeaweedFS, mbake restores the Makefile's final newline, and the Nextcloud OAuth test merges its nested with statements (SIM117).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
select-admin-state.sql joined select-ldap-auth-source-exists.sql under the shared 'select' prefix, which the prefix-grouping lint rejects. Both now live in files/sql/select/ and their three readers point there.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The admin step ran its Rails runner on every deploy. It now digests its sources, its inputs and the administrator and settings state OpenProject holds, compares that with the admin-provision.sha256 stamp in the instance directory, and runs only on a mismatch or when services.openproject.admin.force is set.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
db:migrate took about 24 s on every deploy. After a successful run the
deploy stamps the web container's image id as the comment of
public.schema_migrations; a deploy whose image matches the stamp skips the
migration, a new image, a failed run or a recreated database migrates. The
stamp is read unmasked because masking short config values would corrupt a
hex image id, and written through a DO/format() block because sqlfluff cannot
parse COMMENT ON ... IS with a placeholder.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
OpenBao keeps no record of who read which secret without an audit device,
and 2.7 refuses to enable one over the API, so the server config declares a
file device writing to stdout, which cannot block and lands in the container
log with secret values only as salted HMACs. Every deploy fails unless the
device is active, a probe secret read through the AppRole session shows up
in the log with path, operation, display_name and time, and the probe value
does not appear in plaintext. openbao becomes a regular shared provider so
consumers such as the landing page role can declare services.openbao.
Requirement 047; taken over from the feature/openbao-audit worktree.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An inventory group web-app-cst-<name> with applications.<id>.custom: true and
a pinned source {url, version, path} is pulled into roles/ before the play,
read raw (Jinja, Ansible tags and credential URLs refused), placed through a
staging directory under an exclusive lock, and owned by a marker that names
its own directory, so a pull or roles-clean never replaces or removes a
directory it did not place and a failed pull leaves roles/ as it was.
Directory inventories, host lists and non-UTF-8 files behave as Ansible
reads them; generated test deploys pass --skip-pull. make roles-pull,
roles-clean and roles list, INFINITO_EXTRA_MOUNTS for local sources, the
web-app-cst category, the reserved prefix lint and a central
repository_files() that also yields the gitignored custom roles let every
lint scan them like in-tree roles. Supersedes the uncommitted 045 External
Roles draft and reuses its fetch, sync and mount design.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A constructor task after the update role and the swarm auto-placement installs,
in one package-manager call, every package id of every role the host's groups
may run: meta dependencies, service edges and every role invocation readable
from tasks and handlers, conditions ignored. Ids that must stay with their role
(a conditional install, a repository added first, a need before update) carry
batch: false in meta/packages.yml and keep their task; roles reference shared
root ids with shared: true. 42 per-role install tasks go, and a lint keeps the
split: no batched id in a role task, none in a role run ahead of the batch.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The onion port probe gave every SOCKS connect 60s, half of Tor's
SocksTimeout (120s by default; torrc does not set it), although its own
docstring argued that abandoning an attempt before SocksTimeout is
destructive. Every attempt authenticates with fresh credentials, so Tor
builds a whole new rendezvous for it and keeps working on it until
SocksTimeout. The probe walked away first and called the port dead. The
other raw onion connects in the repository, the svc-db-* CLI tests and
test-e2e-playwright's wait_onion_reachable.sh, wait 120s per attempt.
Run 40128751852 shows how often an attempt needs more than that. Of 178
svc-net-tor CLI runs in its 89 compose onion jobs, three lost their
first attempt to the 60s timeout and connected on the second within
seconds (Shopware#1, Hugo#1, n8n#1; the CLI test took 77-79s against
the usual 1-16s), and Hugo#1's other run took 40s on a single attempt. When
the stall outlasts the 70s until the second attempt, the probe fails
while the HTTP probe reaches the same onion port seconds earlier:
OpenBao#1 (run 39166850273, job 118050491920, the original failure of
#1048) and Socialhome#1 (run 40088473634, job 121019758461). In the
Socialhome job tor.log holds no line for either attempt, because Tor
never reached its own verdict.
ONION_PORT_TIMEOUT is now 130: SocksTimeout plus room for Tor's
once-per-second expiry sweep, so an attempt ends with Tor's own answer
(reply 0x06, and a "Giving up" notice in tor.log, which the rescue
artifact captures) instead of the probe's. The second attempt now starts
at 140s and ends at 270s, so the probe rides out every stall the old
budget did, and any stall shorter than about 260s instead of about 120s.
A green run still costs one connect per port, and a refused or unmapped
port still fails at once. A unit test pins the window above Tor's
SocksTimeout and fails as soon as torrc sets its own.
That doubles what a dead port costs, and meta/tests.yml caps the whole
CLI test. The largest compose onion deploy in run 40128751852 probes 19
domains and 5 ports; with everything dead that is 19 x 350 + 5 x 260 +
10 = 7960s, beyond the 7200s that 069df39027 set for the same reason.
9000 keeps the full failure output for up to 22 domains.
Closes#1214
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Listing, reading, uploading and activating the Sieve forward scripts each run
as one request for every forwarding account, with one method call per account
addressed through its accountId by the recovery admin, at most
STALWART_JMAP_MAX_METHOD_CALLS per request, instead of one request per account
under act-as. The account lookup joins the limits read, a missing account
stops before the limits are raised, and only changed scripts are uploaded.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The carrier key switched from the tokens file's path and hash to the
sys_token_store_users fingerprint once the token store loaded, so the
stage-wide park re-rendered the applications SPOT (about 11 s) even when the
tokens were identical. Both sources now sign as a digest over exactly the
tokens the users hydration reads; the store helpers move to users/store.py
to keep the package under the file length cap.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
forwarded.conf.j2 includes headers/forwarded_base.conf.j2 by its path from
the repository root, the way Ansible searches templates, so the test's loader
searches the root next to the headers directory.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The guard now includes store_guard/migrate.yml for a store from before 1.0
and re-reads its layout, so the walk feeds that include's result into the
final assert: a migrated store passes, one left before 1.0 still stops.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The file was named after the application, so an app served on several domains (SeaweedFS filer/master, Stalwart mail/webmail) overwrote it once per domain on every deploy. The last domain won its dashboard iframe host and Matomo site id for all others, and every deploy reloaded OpenResty for the flip-flop. The file is now <domain>.lua, the naming store-i18n already uses, and its path comes from lookup('nginx', 'files.lua', domain) for writer, reader and the Matomo reinjection. The reinjection stats the live domain's file instead of globbing leftovers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Takes main's Stalwart v1.0.0 port (server v1.0.0, WebUI v2.0.2, 0.16 store guard). manager_ops.yml keeps the updater's per-account forwards ahead of main's store marker, which stays the last task. The root README drops Akaunting with main.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
lookup('tls', app, 'url.base') always ends in '/', and 39 call sites in 25 files rebuilt the slash-less form by hand with regex_replace('/$', ''), regex_replace('/+$', ''), trim('/') or .rstrip('/'), directly or on a variable holding url.base. The lookup now returns that value as url.origin (scheme://host, url.base minus its final '/'), and every call site reads it; two sites that stripped the slash only to append it again now take url.base as is, and GITLAB_URL and DASHBOARD_JS_CDN_URL lost their last consumer.
test_url_base_no_double_slash gains the rule url-base-strip, so url.origin stays the only way to the slash-less URL; it flags all nine HEAD sites probed and none after the change. The same lint keeps url-base-double-slash and url-base-addon-value, which put the nine Nextcloud integration URLs, the Mattermost and SuiteCRM facts and the XWiki instance on the slash-less form, drop the stray '/' before Joomla's index.php callback, and give BBB its api.url as [base, suffix] | url_join plus the trailing '/' its client appends 'api/<call>' to; bbb.spec.js now requires exactly that form.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Connecting a user to OpenProject failed: the instance URL carried url.base's trailing slash, so Nextcloud redeemed the code at //oauth/token (HTTP 500), integration_openproject stayed in its fresh-setup state so isAdminConfigOk was false and no Connect button rendered, and duplicate OAuth clients piled up across deploys. The coupling now converges on each deploy: a PHP helper creates or repairs the Nextcloud OAuth2 client (numeric id, OpenProject callback, same-name duplicates removed); one rails runner call ensures the storage, its OAuth application and the Nextcloud client on it, renewing the OpenProject secret when Nextcloud holds none or a stale one, or when rotate_oauth_client forces it, and realigning the application's redirect_uri after a host change; nc_oauth_client_id is stored only after that call succeeded. openproject_instance_url has no trailing slash and fresh_project_folder_setup is 0.
No secret travels in argv or environment: on the way to OpenProject they ride the same stdin as the Ruby script, as a PROVISION_INPUT heredoc, and the credentials go into Nextcloud through occ lines on the container shell's stdin, taken verbatim from 7f72e53a58 on feature/store-i18n. A Ruby truth table over seven scenarios, with the staged state as negative control, covers fresh install, rerun, a Nextcloud that lost client and secret, a stale secret, forced rotation and a host change. A new Playwright test connects a user through the OAuth round trip and expects Nextcloud's oauth-redirect below 500 and 'Connected as'.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The validator cut every !vault block with a regex that ran up to the next line at column 0, so the keys after a vaulted password vanished, a whole user among them. utils.cache.yaml has parsed the !vault tag itself since c4556fe72d, so the loader now hands the file to it unchanged and a vaulted value loads as its ciphertext. An inventory-only user that brings its own username no longer warns as unknown.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
render_override takes a service's entrypoint and command from the
output of `docker compose config`, which already prints a literal `$`
as `$$`, and the image's Entrypoint and Cmd from `docker image inspect`,
which are raw. It escaped the combined argv once more, so a service
command reading a variable at runtime (`$$REDIS_PASSWORD` in
compose.yml) reached the override as `$$$$` and the container as `$$`,
which the shell expands to its PID. redis then required
"<pid>REDIS_PASSWORD" and every authenticated ping failed with
WRONGPASS.
Only the image-derived parts are escaped now; the service's own argv
is written as compose config printed it. A new test keeps a service
command's `$$` intact next to an image entrypoint; the existing tests
still require an image Cmd's `$VAR` to become `$$VAR`.
Local Nextcloud update deploy: the redis override holds `$$`, the
container Cmd `$REDIS_PASSWORD`, and the authenticated ping answers
PONG on the first attempt.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
automtu searched the path MTU between its payload bounds, and a lost
echo during that search moved the result between 1400 and 1398 from one
deploy to the next. Every move changed the daemon MTU, rewrote
daemon.json and restarted dockerd.
The probe window is now a single payload: the smaller of 1500 and the
egress MTU automtu reads itself (`--print-mtu egress`), minus the 28
bytes of IP and ICMP header, with `--pmtu-policy max` so one target
answering at that size decides. registry-1.docker.io leaves the
targets because it does not answer ICMP echo. The assert now names both
the egress and the effective value.
Local repeat deploy: the probe took 0.83 s and no handler restarted
dockerd.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Handlers run in definition order, and "Generate CA trust override" was
defined before "Build compose". On a first deploy the override read the
Entrypoint and Cmd of every service image with docker image inspect
before compose build had created the locally built tags, found none,
and left those services without the with-ca-trust wrapper; on later
deploys it read the previous build's Entrypoint and Cmd instead of the
new one. The override and the compose config validation now run after
the build and rebuild handlers and before compose up; pull stays first.
The swarm handlers already build before they generate their override
and use no wrapper, so they are unchanged.
TestFirstDeployFlush replays the shell handlers of one compose-build
plus compose-up flush in the order handlers/compose.yml defines them
and asserts that compose up starts the freshly built image behind the
wrapper.
Local deploy: all six stacks with a built image carry the wrapper in
compose.ca.override.yml, nextcloud for 8 of 8 services.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
recognize's app:install downloads about 700 MB of models and held the
plugin phase for minutes. An addon may now set
config.background_install; the plan files such a plugin under the new
background list unless it is pinned through archive or mandatory.
01_background.yml starts occ app:install as an async job and holds
every later occ call at a gate until the app's schema is applied:
Nextcloud migrations diff a schema snapshot, so DDL running beside
another occ call can be reverted. The gate reads the async job file
and files/php/utils/migration_state.php, which prints the versions
oc_migrations records and the versions the app's migration folder
holds. occ migrations:status is not usable, Nextcloud registers it only
with debug on. While the app directory does not exist yet the
MigrationService constructor throws and lib/base.php's ErrorHandler
only logs it, so the probe exits 0 with nothing on stdout and the gate
waits.
The install loop then runs while the job finishes; the always section
joins every job with async_status and fails on an exit code that a
timeout or signal produced, because the php process may then still run
inside the container. 02_install.yml afterwards answers "already
installed" and enables and configures the plugin, or installs it with
its own retries when the job failed before placing the app.
The stage files are renumbered (01_install -> 02_install, 02_enable ->
03_enable, 03_configure -> 04_configure) so the numbers follow the
include order the task-numbering lint checks.
Local perf run: plugin phase 538 s -> 282 s, gate open after 30.2 s,
recognize 12.0.2 enabled with 8/8 migrations, failed=0.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- test_prefix_grouped_files: files/sql/store_database.sql and
store_layout.sql become files/sql/store/database.sql and layout.sql.
- test_file_spot: test_store_guard.py builds the vars path with
ROLE_FILE_VARS_MAIN instead of a hard-coded "vars" / "main.yml" chain.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CodeQL flagged five notes on PR #1215: four test modules imported unittest both as a module and through from-import, and sys-front-inj-matomo's __init__ exported PROJECT_ROOT without declaring it, so the import from test_site_cache.py read as unused. The modules now bind mock through import unittest.mock as mock, and the package declares __all__, the same fixes main carries from 3610e7b9ef and 6e776a5780. The same pattern is fixed in the new Talk direct-media test and the decommission integration package, and the coturn test package drops an __init__ export nothing imports.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
janus_conf is mounted only when direct media is on, which requires compose mode, and compose ignores a config mode outside swarm and warns about it on every deploy. The file keeps the 0644 it is rendered with on the host.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Removing a role from the inventory left its compose stack running under /opt/compose, holding ports, memory and a stale instance directory, until someone stopped it by hand.
On a full compose deploy with MODE_CLEANUP (no --id), the destructor now takes every directory below DIR_COMPOSITIONS that is named after a known role, carries the full instance layout and is owned by no active role (inventory-wide groups, group_names, the dependency closure and the instances this play prepared), stops it with compose down --remove-orphans without -v, verifies through the project and working_dir labels that no container is left, and only then deletes the directory; named volumes stay. Hand-made, partial or backup directories and instances a non-compose container is named after are left in place and listed. The step is the last block of the destructor, with the timer unlock in its always section, so a failure cannot leave timers locked.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The aio-talk image runs Janus with its stock janus.jcfg: no rtp_port_range and no nat_1_1_mapping, so Janus offered only its bridge address and every stream of a call went through coturn as relay. Calls above about six participants stuttered and failed.
When talk runs bridged next to standalone coturn in compose mode on a host that owns networks.internet.ip4, a rendered janus.jcfg is mounted over the stock one with rtp_port_range 61000-61999, nat_1_1_mapping = ip4 and keep_private_host, and that UDP range is published on ip4. TURN stays the fallback. Before rendering, the deploy reads the image's launcher and stock janus.jcfg and stops when Janus no longer reads the mounted path or the stock settings drifted from the template. Variant 0 publishes 64 ports so dev and CI exercise the path; the call capacity spec asserts a non-relay selected pair on ip4 in that range. With the flag off every rendered file is byte-identical to before.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With host networking and no --relay-ip, coturn put every non-loopback IPv4 of the host into its relay list, Docker bridge gateways included, and handed IPv6 and bridge clients relays on those addresses while advertising the public ip4, so their media went nowhere. It also listened on every bridge address.
In compose mode with host networking, coturn now listens on networks.internet.ip4, 127.0.0.1 for the healthcheck and networks.internet.ip6 when set, and relays on ip4. It pins only when the host owns ip4, and owns ip6 too or leaves it unset; behind 1:1 NAT, without the ip binary or with a foreign ip6 it renders the previous command unchanged, because coturn exits when it cannot bind a listening address. A unit test renders the real template and vars for every cell.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Follow-up to the v1.0.0 port, closing holes a review and a local run found:
- WebUI sign-in: WebUI v2 sends an http:// client-metadata-document id
on the onion origin, which Stalwart v1.0.0 rejects (unknown http ids
get no CIMD fallback, trusted hosts require TLS). The recovery admin
under SSO hit the same wall with the bare id 'stalwart-webui'. Register
a public first-party x:OAuthClient 'stalwart-webui' with the WebUI's
callback and login URIs on the served origin, and always inject it as
the Application's oauthClientId. Native sign-in now shows Stalwart's
consent step; test-login-native.js clicks it and also runs under SSO.
- Store guard: the previous guard only checked a running 0.16 server; a
local run showed v1 writing into a 0.16 store whose server was stopped.
The guard now reads the store whenever one exists (shared: pg_database,
dedicated: volume) and fails on the 0.16 bigint layout. A host marker
written at the end of a successful v1 deploy lets restores and
redeploys with a stopped dedicated database pass; without the marker,
a stopped dedicated database fails with an explicit message instead of
a masked psql error. Queries retry while the database recovers.
- README: WebUI sign-in, guard semantics, restore and redeploy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Follow-up to the bucket-policy deny, closing holes two reviews found:
- Scope: the policy was only applied during a consumer's own deploy, so a
provider-only or partial deploy left other buckets without it. Every
provider deploy now converges all existing consumer buckets, and a
consumer's provisioning applies it right after creating the bucket.
- Ownership: our statement is identified by Sid DenyAnonymousRead; a
public bucket loses only that statement, operator statements survive.
- Errors: only a 404 NoSuchBucketPolicy means "no policy"; any other
status fails the read loudly.
- Credentials: curl expands the admin credential itself via --variable
instead of --user "$VAR", which put it on curl's argv and in the
host's ps.
- Curl runs through the repository's curl filter with timeouts, the
read-only tasks run in check mode, and bucket names are parsed exactly
instead of by substring.
- README: corrected embedded-mode authentication and documented the
convergence.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SeaweedFS 4.49+ honours a public-read object or bucket ACL on an unsigned
GET/HEAD/list even in a bucket the platform treats as private
(seaweedfs/seaweedfs#11605). Since consumers hold WriteAcp, an app could
expose objects through the public S3 API domain just by sending
x-amz-acl: public-read.
Put a bucket policy on every non-public consumer bucket that denies
s3:Get* and s3:List* to principal "*" when aws:username is null. SeaweedFS
evaluates that deny before any AllUsers grant; signed and presigned
requests carry an IAM user ARN, so aws:username is set and they are not
affected. Public buckets keep their anonymous Read grant, and a bucket
that turns public gets our deny lifted again. Embedded sidecars publish
no port and are left as they are.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stalwart v1.0.0 rejects unknown JMAP properties ("Invalid property" ->
notCreated/notUpdated), validates OIDC tokens more strictly and refuses to
open a data store from before 1.0. Port the role and protect existing
installs:
- Milter: hostname/timeoutConnect/timeoutCommand/timeoutData become
host/connectTimeout/commandTimeout/dataTimeout; the existing milter is
looked up by host.
- Onion relay route: address becomes host.
- OIDC directory: requireAudience is gone. Set skipAudienceValidation and
unscopedTokens=interoperable on create and update, so Keycloak tokens
(no Stalwart audience, no mail scope) keep working for the WebUI and
Roundcube XOAUTH2.
- WebUI: send oauthClientId only with SSO enabled; without SSO the WebUI
falls back to its own client metadata document.
- Store guard: before anything changes, a running Stalwart whose table u
still has the 0.16 bigint layout stops the deploy and points to the
stalwart-migrate steps, which the README now documents under
"Upgrading from 0.16".
- Updater: pin the WebUI git_tags source to v2.* to match server 1.x.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SeaweedFS 4.49 (seaweedfs/seaweedfs#11592) authorizes a PutObject that
carries x-amz-acl as s3:PutObjectAcl as well. The consumer identities
only held Read/Write/List/Tagging, so every app that sends an ACL on
upload (Nextcloud, Fider and Pixelfed with private, Mastodon, PeerTube,
BookWyrm and Baserow with public-read) got AccessDenied after the 4.48 -> 4.51
bump in #1226. Main run 40401200083 failed Fider#0, BookWyrm#0,
Baserow#0, Pixelfed#0 and Nextcloud#0/#4/#8 on it.
Add WriteAcp to the s3.json identities and to the sidecar s3.configure
grant.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
csp-checker 3.2.0 checks every URL in its own browser context and takes --concurrency for a worker pool. The role pins 3.2.0, sets HEALTH_CSP_CONCURRENCY to 4, and the systemd unit hands it through script.py to the container command ahead of the -- URL separator, for the clearnet and the onion batch alike. TimeoutStartSec keeps its sequential sizing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
test_include_uniform_apply_keyword flagged legacy_checkout.yml after 11c2d8dcd3 moved its guard onto the include: both tasks repeated become: true. The include in 03_launcher.yml now applies become to the file and keeps the guard; the order test reads include_tasks in its mapping form too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every database consumer called baudolo-seed through the seed task, 37 times and always 'changed' in a full production run (54.4 s), although databases.csv rarely changes. The first seed of a play now reads the file once (05_read-seeded-databases.yml) into svc_bkp_volume_2_local_seeded; baudolo_seed_current compares the consumer's row with it and baudolo_seed_apply advances the cached state the way baudolo-seed writes the file, so the command runs only for a missing or differing row. A file baudolo did not write byte for byte (hand-edited, CRLF, short rows) falls back to one call per consumer; 02_reset forgets the cache with the file.
Replay harness old vs new: databases.csv byte-identical in fresh, empty, converged, drift and reset-mid-play scenarios; converged seeds 37 -> 0, drift 38 -> 2; the negative control without reset invalidation diverges. About 44 s per unchanged redeploy.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each plugin ran its own occ config:app:set per configuration value right after its install or enable, and a redeploy rewrote every value. Install and enable now queue the plugin; the configure stage runs once for the queue: it loads every queued addon spec, reads occ config:list once, keeps only the values Nextcloud does not hold exactly (nextcloud_plugin_config_pending; typed, masked and structured values are always kept), and runs the hooks in queue order afterwards.
Combined with store-i18n's 7f72e53a58: the pending occ config:app:set calls go on the stdin of one container exec -i ... sh -s, so no value (Talk, OnlyOffice, whiteboard, BBB secrets) reaches a host argv; a failing call names its appid/configkey.
Replay harness old vs new: identical final state, updated keys and hook order in first-deploy, converged, drift and failure scenarios (failure stops at the same key); converged config:app:set calls 19 -> 3. Merge note: resolve 03_configure.yml with this side against 7f72e53a58.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every app's compose pass ran three remote tasks before compose up: create the directories, compose ps for running containers, and stat the pending marker (53 apps x ~2.7 s in a production run). compose_instance_prepare does all three in one call from 02_stack_host_prep.yml, sharing the directory creation with directory_batch through utils/directory_batch.py; compose ps runs only in compose mode. up.yml evaluates the registered result, named compose_instance_before_files because it predates 04_files: the running check is skipped whenever the compose, env or override file changed in this run (those changes notify compose-up themselves).
Replay harness old vs new: rendered files and the compose-up, marker and swarm handlers identical in every scenario; the only difference is the skipped running check when only the override changed, where compose-up still runs. About 85 s per full production run.
Merge note: store-i18n's a4c7fda974 hoists the removed marker stat into a block; resolve up.yml with this side.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
