forwarded.conf.j2 includes headers/forwarded_base.conf.j2 by its path from
the repository root, the way Ansible searches templates, so the test's loader
searches the root next to the headers directory.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The guard now includes store_guard/migrate.yml for a store from before 1.0
and re-reads its layout, so the walk feeds that include's result into the
final assert: a migrated store passes, one left before 1.0 still stops.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The file was named after the application, so an app served on several domains (SeaweedFS filer/master, Stalwart mail/webmail) overwrote it once per domain on every deploy. The last domain won its dashboard iframe host and Matomo site id for all others, and every deploy reloaded OpenResty for the flip-flop. The file is now <domain>.lua, the naming store-i18n already uses, and its path comes from lookup('nginx', 'files.lua', domain) for writer, reader and the Matomo reinjection. The reinjection stats the live domain's file instead of globbing leftovers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Takes main's Stalwart v1.0.0 port (server v1.0.0, WebUI v2.0.2, 0.16 store guard). manager_ops.yml keeps the updater's per-account forwards ahead of main's store marker, which stays the last task. The root README drops Akaunting with main.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
lookup('tls', app, 'url.base') always ends in '/', and 39 call sites in 25 files rebuilt the slash-less form by hand with regex_replace('/$', ''), regex_replace('/+$', ''), trim('/') or .rstrip('/'), directly or on a variable holding url.base. The lookup now returns that value as url.origin (scheme://host, url.base minus its final '/'), and every call site reads it; two sites that stripped the slash only to append it again now take url.base as is, and GITLAB_URL and DASHBOARD_JS_CDN_URL lost their last consumer.
test_url_base_no_double_slash gains the rule url-base-strip, so url.origin stays the only way to the slash-less URL; it flags all nine HEAD sites probed and none after the change. The same lint keeps url-base-double-slash and url-base-addon-value, which put the nine Nextcloud integration URLs, the Mattermost and SuiteCRM facts and the XWiki instance on the slash-less form, drop the stray '/' before Joomla's index.php callback, and give BBB its api.url as [base, suffix] | url_join plus the trailing '/' its client appends 'api/<call>' to; bbb.spec.js now requires exactly that form.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Connecting a user to OpenProject failed: the instance URL carried url.base's trailing slash, so Nextcloud redeemed the code at //oauth/token (HTTP 500), integration_openproject stayed in its fresh-setup state so isAdminConfigOk was false and no Connect button rendered, and duplicate OAuth clients piled up across deploys. The coupling now converges on each deploy: a PHP helper creates or repairs the Nextcloud OAuth2 client (numeric id, OpenProject callback, same-name duplicates removed); one rails runner call ensures the storage, its OAuth application and the Nextcloud client on it, renewing the OpenProject secret when Nextcloud holds none or a stale one, or when rotate_oauth_client forces it, and realigning the application's redirect_uri after a host change; nc_oauth_client_id is stored only after that call succeeded. openproject_instance_url has no trailing slash and fresh_project_folder_setup is 0.
No secret travels in argv or environment: on the way to OpenProject they ride the same stdin as the Ruby script, as a PROVISION_INPUT heredoc, and the credentials go into Nextcloud through occ lines on the container shell's stdin, taken verbatim from 7f72e53a58 on feature/store-i18n. A Ruby truth table over seven scenarios, with the staged state as negative control, covers fresh install, rerun, a Nextcloud that lost client and secret, a stale secret, forced rotation and a host change. A new Playwright test connects a user through the OAuth round trip and expects Nextcloud's oauth-redirect below 500 and 'Connected as'.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The validator cut every !vault block with a regex that ran up to the next line at column 0, so the keys after a vaulted password vanished, a whole user among them. utils.cache.yaml has parsed the !vault tag itself since c4556fe72d, so the loader now hands the file to it unchanged and a vaulted value loads as its ciphertext. An inventory-only user that brings its own username no longer warns as unknown.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
render_override takes a service's entrypoint and command from the
output of `docker compose config`, which already prints a literal `$`
as `$$`, and the image's Entrypoint and Cmd from `docker image inspect`,
which are raw. It escaped the combined argv once more, so a service
command reading a variable at runtime (`$$REDIS_PASSWORD` in
compose.yml) reached the override as `$$$$` and the container as `$$`,
which the shell expands to its PID. redis then required
"<pid>REDIS_PASSWORD" and every authenticated ping failed with
WRONGPASS.
Only the image-derived parts are escaped now; the service's own argv
is written as compose config printed it. A new test keeps a service
command's `$$` intact next to an image entrypoint; the existing tests
still require an image Cmd's `$VAR` to become `$$VAR`.
Local Nextcloud update deploy: the redis override holds `$$`, the
container Cmd `$REDIS_PASSWORD`, and the authenticated ping answers
PONG on the first attempt.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
automtu searched the path MTU between its payload bounds, and a lost
echo during that search moved the result between 1400 and 1398 from one
deploy to the next. Every move changed the daemon MTU, rewrote
daemon.json and restarted dockerd.
The probe window is now a single payload: the smaller of 1500 and the
egress MTU automtu reads itself (`--print-mtu egress`), minus the 28
bytes of IP and ICMP header, with `--pmtu-policy max` so one target
answering at that size decides. registry-1.docker.io leaves the
targets because it does not answer ICMP echo. The assert now names both
the egress and the effective value.
Local repeat deploy: the probe took 0.83 s and no handler restarted
dockerd.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Handlers run in definition order, and "Generate CA trust override" was
defined before "Build compose". On a first deploy the override read the
Entrypoint and Cmd of every service image with docker image inspect
before compose build had created the locally built tags, found none,
and left those services without the with-ca-trust wrapper; on later
deploys it read the previous build's Entrypoint and Cmd instead of the
new one. The override and the compose config validation now run after
the build and rebuild handlers and before compose up; pull stays first.
The swarm handlers already build before they generate their override
and use no wrapper, so they are unchanged.
TestFirstDeployFlush replays the shell handlers of one compose-build
plus compose-up flush in the order handlers/compose.yml defines them
and asserts that compose up starts the freshly built image behind the
wrapper.
Local deploy: all six stacks with a built image carry the wrapper in
compose.ca.override.yml, nextcloud for 8 of 8 services.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
recognize's app:install downloads about 700 MB of models and held the
plugin phase for minutes. An addon may now set
config.background_install; the plan files such a plugin under the new
background list unless it is pinned through archive or mandatory.
01_background.yml starts occ app:install as an async job and holds
every later occ call at a gate until the app's schema is applied:
Nextcloud migrations diff a schema snapshot, so DDL running beside
another occ call can be reverted. The gate reads the async job file
and files/php/utils/migration_state.php, which prints the versions
oc_migrations records and the versions the app's migration folder
holds. occ migrations:status is not usable, Nextcloud registers it only
with debug on. While the app directory does not exist yet the
MigrationService constructor throws and lib/base.php's ErrorHandler
only logs it, so the probe exits 0 with nothing on stdout and the gate
waits.
The install loop then runs while the job finishes; the always section
joins every job with async_status and fails on an exit code that a
timeout or signal produced, because the php process may then still run
inside the container. 02_install.yml afterwards answers "already
installed" and enables and configures the plugin, or installs it with
its own retries when the job failed before placing the app.
The stage files are renumbered (01_install -> 02_install, 02_enable ->
03_enable, 03_configure -> 04_configure) so the numbers follow the
include order the task-numbering lint checks.
Local perf run: plugin phase 538 s -> 282 s, gate open after 30.2 s,
recognize 12.0.2 enabled with 8/8 migrations, failed=0.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- test_prefix_grouped_files: files/sql/store_database.sql and
store_layout.sql become files/sql/store/database.sql and layout.sql.
- test_file_spot: test_store_guard.py builds the vars path with
ROLE_FILE_VARS_MAIN instead of a hard-coded "vars" / "main.yml" chain.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CodeQL flagged five notes on PR #1215: four test modules imported unittest both as a module and through from-import, and sys-front-inj-matomo's __init__ exported PROJECT_ROOT without declaring it, so the import from test_site_cache.py read as unused. The modules now bind mock through import unittest.mock as mock, and the package declares __all__, the same fixes main carries from 3610e7b9ef and 6e776a5780. The same pattern is fixed in the new Talk direct-media test and the decommission integration package, and the coturn test package drops an __init__ export nothing imports.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
janus_conf is mounted only when direct media is on, which requires compose mode, and compose ignores a config mode outside swarm and warns about it on every deploy. The file keeps the 0644 it is rendered with on the host.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Removing a role from the inventory left its compose stack running under /opt/compose, holding ports, memory and a stale instance directory, until someone stopped it by hand.
On a full compose deploy with MODE_CLEANUP (no --id), the destructor now takes every directory below DIR_COMPOSITIONS that is named after a known role, carries the full instance layout and is owned by no active role (inventory-wide groups, group_names, the dependency closure and the instances this play prepared), stops it with compose down --remove-orphans without -v, verifies through the project and working_dir labels that no container is left, and only then deletes the directory; named volumes stay. Hand-made, partial or backup directories and instances a non-compose container is named after are left in place and listed. The step is the last block of the destructor, with the timer unlock in its always section, so a failure cannot leave timers locked.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The aio-talk image runs Janus with its stock janus.jcfg: no rtp_port_range and no nat_1_1_mapping, so Janus offered only its bridge address and every stream of a call went through coturn as relay. Calls above about six participants stuttered and failed.
When talk runs bridged next to standalone coturn in compose mode on a host that owns networks.internet.ip4, a rendered janus.jcfg is mounted over the stock one with rtp_port_range 61000-61999, nat_1_1_mapping = ip4 and keep_private_host, and that UDP range is published on ip4. TURN stays the fallback. Before rendering, the deploy reads the image's launcher and stock janus.jcfg and stops when Janus no longer reads the mounted path or the stock settings drifted from the template. Variant 0 publishes 64 ports so dev and CI exercise the path; the call capacity spec asserts a non-relay selected pair on ip4 in that range. With the flag off every rendered file is byte-identical to before.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With host networking and no --relay-ip, coturn put every non-loopback IPv4 of the host into its relay list, Docker bridge gateways included, and handed IPv6 and bridge clients relays on those addresses while advertising the public ip4, so their media went nowhere. It also listened on every bridge address.
In compose mode with host networking, coturn now listens on networks.internet.ip4, 127.0.0.1 for the healthcheck and networks.internet.ip6 when set, and relays on ip4. It pins only when the host owns ip4, and owns ip6 too or leaves it unset; behind 1:1 NAT, without the ip binary or with a foreign ip6 it renders the previous command unchanged, because coturn exits when it cannot bind a listening address. A unit test renders the real template and vars for every cell.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Follow-up to the v1.0.0 port, closing holes a review and a local run found:
- WebUI sign-in: WebUI v2 sends an http:// client-metadata-document id
on the onion origin, which Stalwart v1.0.0 rejects (unknown http ids
get no CIMD fallback, trusted hosts require TLS). The recovery admin
under SSO hit the same wall with the bare id 'stalwart-webui'. Register
a public first-party x:OAuthClient 'stalwart-webui' with the WebUI's
callback and login URIs on the served origin, and always inject it as
the Application's oauthClientId. Native sign-in now shows Stalwart's
consent step; test-login-native.js clicks it and also runs under SSO.
- Store guard: the previous guard only checked a running 0.16 server; a
local run showed v1 writing into a 0.16 store whose server was stopped.
The guard now reads the store whenever one exists (shared: pg_database,
dedicated: volume) and fails on the 0.16 bigint layout. A host marker
written at the end of a successful v1 deploy lets restores and
redeploys with a stopped dedicated database pass; without the marker,
a stopped dedicated database fails with an explicit message instead of
a masked psql error. Queries retry while the database recovers.
- README: WebUI sign-in, guard semantics, restore and redeploy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Follow-up to the bucket-policy deny, closing holes two reviews found:
- Scope: the policy was only applied during a consumer's own deploy, so a
provider-only or partial deploy left other buckets without it. Every
provider deploy now converges all existing consumer buckets, and a
consumer's provisioning applies it right after creating the bucket.
- Ownership: our statement is identified by Sid DenyAnonymousRead; a
public bucket loses only that statement, operator statements survive.
- Errors: only a 404 NoSuchBucketPolicy means "no policy"; any other
status fails the read loudly.
- Credentials: curl expands the admin credential itself via --variable
instead of --user "$VAR", which put it on curl's argv and in the
host's ps.
- Curl runs through the repository's curl filter with timeouts, the
read-only tasks run in check mode, and bucket names are parsed exactly
instead of by substring.
- README: corrected embedded-mode authentication and documented the
convergence.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SeaweedFS 4.49+ honours a public-read object or bucket ACL on an unsigned
GET/HEAD/list even in a bucket the platform treats as private
(seaweedfs/seaweedfs#11605). Since consumers hold WriteAcp, an app could
expose objects through the public S3 API domain just by sending
x-amz-acl: public-read.
Put a bucket policy on every non-public consumer bucket that denies
s3:Get* and s3:List* to principal "*" when aws:username is null. SeaweedFS
evaluates that deny before any AllUsers grant; signed and presigned
requests carry an IAM user ARN, so aws:username is set and they are not
affected. Public buckets keep their anonymous Read grant, and a bucket
that turns public gets our deny lifted again. Embedded sidecars publish
no port and are left as they are.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stalwart v1.0.0 rejects unknown JMAP properties ("Invalid property" ->
notCreated/notUpdated), validates OIDC tokens more strictly and refuses to
open a data store from before 1.0. Port the role and protect existing
installs:
- Milter: hostname/timeoutConnect/timeoutCommand/timeoutData become
host/connectTimeout/commandTimeout/dataTimeout; the existing milter is
looked up by host.
- Onion relay route: address becomes host.
- OIDC directory: requireAudience is gone. Set skipAudienceValidation and
unscopedTokens=interoperable on create and update, so Keycloak tokens
(no Stalwart audience, no mail scope) keep working for the WebUI and
Roundcube XOAUTH2.
- WebUI: send oauthClientId only with SSO enabled; without SSO the WebUI
falls back to its own client metadata document.
- Store guard: before anything changes, a running Stalwart whose table u
still has the 0.16 bigint layout stops the deploy and points to the
stalwart-migrate steps, which the README now documents under
"Upgrading from 0.16".
- Updater: pin the WebUI git_tags source to v2.* to match server 1.x.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SeaweedFS 4.49 (seaweedfs/seaweedfs#11592) authorizes a PutObject that
carries x-amz-acl as s3:PutObjectAcl as well. The consumer identities
only held Read/Write/List/Tagging, so every app that sends an ACL on
upload (Nextcloud, Fider and Pixelfed with private, Mastodon, PeerTube,
BookWyrm and Baserow with public-read) got AccessDenied after the 4.48 -> 4.51
bump in #1226. Main run 40401200083 failed Fider#0, BookWyrm#0,
Baserow#0, Pixelfed#0 and Nextcloud#0/#4/#8 on it.
Add WriteAcp to the s3.json identities and to the sidecar s3.configure
grant.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
csp-checker 3.2.0 checks every URL in its own browser context and takes --concurrency for a worker pool. The role pins 3.2.0, sets HEALTH_CSP_CONCURRENCY to 4, and the systemd unit hands it through script.py to the container command ahead of the -- URL separator, for the clearnet and the onion batch alike. TimeoutStartSec keeps its sequential sizing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
test_include_uniform_apply_keyword flagged legacy_checkout.yml after 11c2d8dcd3 moved its guard onto the include: both tasks repeated become: true. The include in 03_launcher.yml now applies become to the file and keeps the guard; the order test reads include_tasks in its mapping form too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every database consumer called baudolo-seed through the seed task, 37 times and always 'changed' in a full production run (54.4 s), although databases.csv rarely changes. The first seed of a play now reads the file once (05_read-seeded-databases.yml) into svc_bkp_volume_2_local_seeded; baudolo_seed_current compares the consumer's row with it and baudolo_seed_apply advances the cached state the way baudolo-seed writes the file, so the command runs only for a missing or differing row. A file baudolo did not write byte for byte (hand-edited, CRLF, short rows) falls back to one call per consumer; 02_reset forgets the cache with the file.
Replay harness old vs new: databases.csv byte-identical in fresh, empty, converged, drift and reset-mid-play scenarios; converged seeds 37 -> 0, drift 38 -> 2; the negative control without reset invalidation diverges. About 44 s per unchanged redeploy.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each plugin ran its own occ config:app:set per configuration value right after its install or enable, and a redeploy rewrote every value. Install and enable now queue the plugin; the configure stage runs once for the queue: it loads every queued addon spec, reads occ config:list once, keeps only the values Nextcloud does not hold exactly (nextcloud_plugin_config_pending; typed, masked and structured values are always kept), and runs the hooks in queue order afterwards.
Combined with store-i18n's 7f72e53a58: the pending occ config:app:set calls go on the stdin of one container exec -i ... sh -s, so no value (Talk, OnlyOffice, whiteboard, BBB secrets) reaches a host argv; a failing call names its appid/configkey.
Replay harness old vs new: identical final state, updated keys and hook order in first-deploy, converged, drift and failure scenarios (failure stops at the same key); converged config:app:set calls 19 -> 3. Merge note: resolve 03_configure.yml with this side against 7f72e53a58.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every app's compose pass ran three remote tasks before compose up: create the directories, compose ps for running containers, and stat the pending marker (53 apps x ~2.7 s in a production run). compose_instance_prepare does all three in one call from 02_stack_host_prep.yml, sharing the directory creation with directory_batch through utils/directory_batch.py; compose ps runs only in compose mode. up.yml evaluates the registered result, named compose_instance_before_files because it predates 04_files: the running check is skipped whenever the compose, env or override file changed in this run (those changes notify compose-up themselves).
Replay harness old vs new: rendered files and the compose-up, marker and swarm handlers identical in every scenario; the only difference is the skipped running check when only the override changed, where compose-up still runs. About 85 s per full production run.
Merge note: store-i18n's a4c7fda974 hoists the removed marker stat into a block; resolve up.yml with this side.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
DISCOURSE_REPOSITORY_DIR appended the repository URL to the services directory, so the launcher lived under services/https:/github.com/discourse/discourse_docker.git/. It now lives under services/discourse_docker. Before the clone, launcher/legacy_checkout.yml removes the URL-named tree (computed with relpath against the services directory) and prunes its emptied parents; a guard skips the cleanup when that path would escape the services directory or equal the new checkout. The cleanup must precede the clone: git otherwise resolves the old relative path as a URL and aborts with 'protocol ... is not supported'. Discourse data stays in the launcher's named volume and /var/discourse/shared; the first deploy rebuilds once because the bootstrap digest lived in the old containers/ directory.
relpath joins the builtin filters test_defined knows.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The dashboard card of every app opened its primary domain, so Matrix led to the Synapse status page and Stalwart to its admin UI. A role can now declare services.dashboard.card: <key of meta/domains.yml canonical>; docker_cards builds the card URL with lookup('canonical_url', app, key), the Apps menu reuses it, and roles_with_service('dashboard') hands the same domain to the tile e2e spec. Matrix sets card: element, Stalwart card: webmail. Without card nothing changes, and the prometheus/matomo target lists are untouched. A lint requires the key to resolve in the role's own domains.canonical.
The Matrix ansible flavor also sets Synapse's web_client_location to Element through matrix_synapse_configuration_extension_yaml, as the compose flavor already does, so matrix.<domain>/ redirects there. Merge note: feature/iam-profile-rbac defines the same extension variable inside the OIDC block; the next merge must combine both into one block (agreed with that session).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A host that defines CAPTCHA with one provider replaces the group_vars mapping (no hash_behaviour merge), so galaxyserver, which configures only hCaptcha, left CAPTCHA.RECAPTCHA undefined. RECAPTCHA_ENABLED then failed to render, the applications render fallback returned the expression as text, and Keycloak's '| bool' coerced it to False with a deprecation warning that ansible-core 2.23 turns into an error. Both flags now check that their provider mapping is defined first; the effective values stay the same for every existing inventory.
test_captcha_enabled renders every role's services.recaptcha/hcaptcha.enabled through the real templar for defaults, both providers, only the other provider and only this provider, and fails on the raw text (negative control with the old expression: 4 subtests).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every redeploy ran occ maintenance:repair --include-expensive (43.5 s, maintenance mode on) and listed the appstore updates through Tor (49.7 s) although the version had not changed. 03_upgrade.yml now reads occ status and files/mimetype_version after occ upgrade; nextcloud_needs_post_upgrade decides whether the pass is due (version changed in this deploy, or the expensive repair has not run for the running version). A due deploy includes 03_upgrade/01_post_upgrade.yml with today's app update and repair; an unchanged one includes 03_upgrade/02_cached_updates.yml, which updates only the unpinned enabled apps whose release stored by the updatenotification background job is newer (nextcloud_cached_app_updates, PHP version_compare semantics). NEXTCLOUD_PLUGINS_PINNED is the single source of the pinned check.
Verified against echoserver's live occ output (status, marker, updatenotification cache, app:list, with debug log lines): unchanged redeploy is not due and finds no update. Scenario harness 24/24 incl. negative controls on the old file. About 86 s per unchanged redeploy.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each vhost used to run its own sequential reachability probe. The role now queues the vhost and deferred_probe.yml checks the whole queue with one uri_probe_batch call after the shared-service preload and after the server roles. A vhost answering like a stale config restarts OpenResty once; one still unserved afterwards fails the deploy by name. uri_probe_batch takes follow_redirects explicitly so the probe sees the redirect status itself.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
5b7284c4a4 named its reset task with 81 characters, one over the limit of
test_name_emoji_and_length, and 754d35c4de built the path to
roles/web-app-keycloak/vars/main.yml with a pathlib chain that test_file_spot
requires to come from ROLE_FILE_VARS_MAIN. Both commits went in without the
lint suite. The task keeps its behaviour under a 70-character name, the test
reads the same file through the constant. make test-lint: 577 passed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
SAML clients were created only by the realm import, and the import runs only
for a missing realm. echoserver keeps its legacy realm cymais.cloud and ran
SuiteCRM behind oauth2-proxy before v16, so the SuiteCRM SAML client
https://<suitecrm>/saml/metadata never existed there: the SAML hop showed
Keycloak's Invalid Request page and Keycloak logged LOGIN_ERROR
error=client_not_found reason=Cannot_match_source_hash.
The existing-realm reconcile now also loops over the clientId of every SAML
client in the rendered realm dictionary, the same source the import uses,
and creates or updates it through kcadm like the application-declared OIDC
clients. The same path already converged the stalwart-webmail and
stalwart-webui clients in echoserver full-10.
Same change as 2b4db90576 on feature/store-i18n, so a later merge of the two
lines resolves without a conflict, plus a unit test that pins the SAML client
selection and its place in the reconcile loop.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On a host whose Matomo was not deployed yet, every vhost injected before
web-app-matomo asked SitesManager.getSitesWithAtLeastViewAccess again and
waited out uri_retry (30 retries x 2 s): 6 vhosts, 84 to 98 s each, 8.7
minutes in galaxyserver full-2. matomo_site_cache only kept a loaded list.
inject.yml now caches every executed load: the list on success, none on a
failed request or an error mapping. A cache of none skips the cached-id step
and getSitesIdFromSiteUrl, so the existing skip task turns the injection off
for the vhost as before, without the retry wait. web-app-matomo resets the
cache of every play host (delegate_facts, swarm workers included) after
07_config and before 08_reinject_consumers, so the re-injection and every
later vhost load the list from the now running Matomo.
Evidence: the real inject.yml with the real uri_retry against a closed port
and a stub Matomo, 11 vhosts in 4 scenarios: the first probe took 68.6 s, the
next two 0.4 and 0.3 s with matomo off; after the reset a known site came
from the cache, an unknown one went getSitesIdFromSiteUrl -> addSite and was
appended, a later vhost of that domain hit the cache. HEAD inject.yml took
68 s per vhost with the same outcome. The swarm reset reached a second host.
ansible-lint passes; tests/unit/python/roles/sys-front-inj-matomo pins the
gates and the reset order.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
resolve_term follows a redirect source to its target so an alias inherits the TLS settings of the application it points at. A redirect to a domain another host serves ends that chain on a domain this host does not know, and the tls and cert lookups failed: galaxyserver full-1 stopped in web-opt-rdr-domains on 'tls: domain project.infinito.nexus not found' for its project.veen.world -> project.infinito.nexus redirect.
When the chain ends outside the host's domains and web-opt-rdr-domains is deployed, the source now resolves to web-opt-rdr-domains with the source itself as primary domain, which is the role that serves its vhost and whose certificate bundle names it. Only the outermost call falls back, so the source asked for stays the primary domain across a longer chain; without the redirect role, or for a target this host serves, nothing changes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The reconcile task retried every failure thirty times ten seconds apart, so a permission the key lacks surfaced only after five minutes: rdr-3 spent 331 s on a 403 that no retry could change. The reconciler now ends with exit code 3 when Flowise answers 401 or 403, and the task stops retrying on that code while its failed_when still fails the run under MCP_RECONCILE_STRICT.
Only 401 and 403 count as final: authorize may answer a provider that is still starting with another 4xx, and that wait is what the retries are for. Every exit that reports an HTTP status goes through fail(), which keeps the message on the exception, so the existing tests read it unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
mcp_servers narrowed the MCP providers to application_closure(deployment.whitelist), which on a persistent host drops every provider an earlier run deployed and a partial run does not touch. sys-svc-mcp-reconcile then converges each client present on the host against that narrowed set: on echoserver the --id web-opt-rdr-domains run rdr-3 removed the Prometheus registration from Flowise.
deployment now reports ephemeral (runtime act or github), and the closure narrows only there, where a whitelist is all a run deploys; a persistent host narrows nothing, as a full deploy already did with its empty whitelist, so its clients keep every provider it serves. CI keeps its narrowing unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
test_mint_mcp_api_key.py computed PROJECT_ROOT itself from Path(__file__).parents[6], which test_project_root_import forbids outside an __init__.py. The files package now exports PROJECT_ROOT the way its reconcile subpackage does, and the test imports it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The reconciler prunes the managed fixture chatflows of providers it no longer probes, but the scoped API key it gets was minted without chatflows:delete, so the first prune ended in 403 Forbidden after thirty retries; on echoserver rdr-3 stopped on 'FAILED deleting stale infinito:fixture:web-app-prometheus'. Adding the permission alone would not have reached that host, because api_key() reused an existing key unchanged.
KEY_PERMISSIONS now carries chatflows:delete, and api_key() reuses the managed key only while its permissions equal KEY_PERMISSIONS; otherwise it deletes the key through DELETE /api/v1/apikey/<id> and mints a new one, which the reconciler receives in the same run. The script gains its mirrored unit test, so its mirrored-unit-test exemption goes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Host-wide redirects such as a legacy zone came from the top-level inventory variable redirect_domain_mappings, which only the role default declared. Once the role input moved to web_opt_rdr_domains_mappings nothing declared it any more and the inventory validation refused the deploy, because it does not see the lookup's vars_ read. The redirects now live where the role's other settings are: services.rdr-domains.redirects in meta/services.yml, empty by default and set per inventory under applications.web-opt-rdr-domains, and current_play_redirect_domains reads them through the config lookup. They still win over the application aliases, and a caller that includes the role without its own web_opt_rdr_domains_mappings still renders the merged CURRENT_PLAY_REDIRECT_DOMAINS.
Inventories move their redirect_domain_mappings list unchanged to applications.web-opt-rdr-domains.services.rdr-domains.redirects.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every addon ran through the install, enable and configure stages on its own, and each stage asked occ for the plugin's state again. The role now reads occ app:list --output=json once, and the new nextcloud_plugin_plan filter replays the addon order (an enabled entry switches its incompatible_plugins off and itself on, a disabled entry switches itself off, the later entry wins) and splits the app ids into disable, install, enable and configure. One app:disable call takes every id that has to go off; the install, enable and configure includes loop only over the ids that need that stage, so an already enabled plugin skips straight to its configuration. A plugin shipped as a pinned archive still goes through install on every run.
On echoserver the plugin phase took 608 s in full-9 for 51 addons; a repeat deploy is derived at about 195 s instead of about 550 s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
quality-high's autoformat pass rewrote three files that 9301220d05,
49aa920034 and c78d5f906c committed without the hooks. ruff drops three
`noqa: E402` markers as unused (RUF100): E402 does not fire on an import
that follows sys.path.insert. The formatter wraps the long
KEYCLOAK_RBAC_GROUP_LOGIC_DIGEST line and the OpenProject admin runner's
`sh -c` line; the double-quoted scalar and the `>-` block both fold the
break back into a space, and a parse against HEAD gives equal values.
On merge: feature/store-i18n 2bcf59109e carries the same four Keycloak
hunks byte for byte; they merge without a conflict.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The emailVerified update looped over every non-reserved user, uploaded a
shell script per user and ran kcadm with a JVM login each time, and it
always reported changed: 7 users took 23.8 s on echoserver.
email_verified.py uses the shared keycloak_admin client through
tasks/utils/rest_script.yml: it looks each user up by exact username,
skips users already verified, and writes emailVerified with a GET plus a
full-representation PUT, as kcadm update did. The task reports changed
only when a user was updated, and a failed lookup now fails the task
instead of being treated as a missing user. The old lookup piped
kcadm -q username=X into head -n1, which could have hit a longer username
sharing the prefix.
Evidence (local compose, slot 104): summary updated=0 unchanged=8
missing=0 failed=0 in 0.60 s, task ok, staging dir removed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The per-app RBAC groups and LDAP group mappers were reconciled by four
kcadm shell scripts that started a JVM per call and re-synced every app
on every deploy: 582 s for the mappers plus 324 s for the orphan purge on
echoserver (39 apps).
rbac_reconcile.py now talks to the admin REST API from inside the
Keycloak network namespace (nsenter --net) with the admin credentials
read from /proc/<pid>/environ, so nothing secret reaches argv or output.
It drops the shared ldap-roles mapper, purges root groups of undeployed
apps, ensures each app group and mapper, and syncs a mapper only when the
group is new, the mapper config drifted, or the fingerprint stored on the
group (mapper payload, app rbac and domains, group-logic digest) differs.
The per-app payload no longer carries groups.path.group: Keycloak 26.8.1
does not store that key, so a payload with it never matched the stored
mapper and every deploy re-synced every app.
The admin client lives in keycloak_admin.py; tasks/utils/rest_script.yml
stages it with the script in a 0700 temp dir on KEYCLOAK_HOST_NODE, runs
it and removes the dir in an always block.
Evidence (local compose, slot 104): matrix iteration green in both
variants and passes; update runs show synced=1 in 1.1 s, then
synced=0 unchanged=1 in 0.81 s through the final delivery path, no
staging dir left behind.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The parallel-sessions test asserted that more than one of four sessions
appended something, which a loaded host where one worker drains the
shared iterator fails with nothing broken. The sessions now meet at a
barrier before their first append, so a serialised pool still fails the
assertion while a concurrent one always passes.
(cherry picked from commit 657015852a)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
