CI run 41261531965 Nextcloud#3 (multi network) tested Nextcloud on its onion address. integration_openproject builds redirect_uri from the request host, so it sent http://next.cloud.<onion>.onion/.../oauth-redirect, but ensure_nextcloud_storage.rb registered only the URIs built from the canonical NEXTCLOUD_URL; Doorkeeper answered every /oauth/authorize with 400 invalid_redirect_uri (proxy log: 6x 400, OpenProject web log status=400 user=6, trace page 'The requested redirect uri ... doesn't match client redirect URI').
NEXTCLOUD_NETWORK_BASE_URLS comes from lookup('network_urls', NEXTCLOUD_URL), like the Zammad and GitLab base URLs, and reaches the provisioning input as nextcloud_base_urls; the script registers every base URL with the apps/ and index.php/apps/ paths, canonical first, and the base URLs feed the converge digest so an install switched to multi network re-provisions. Clearnet-only and tor-only rows register the same string as before. Zammad, GitLab and Mattermost integrations still register only the canonical URI.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Gate pass 23 failed tests/lint/filesystem/shell/test_no_or_true.py on init-gate.sh: occ_status swallowed every exit code with || true. It now returns 0 with the status on success and maps 1 (occ error), 124 (timeout) and 255 (php fatal while the tree is copied or upgraded) to an empty status, the not-ready state both slots already handle. Any other code, such as 127 from a missing su, timeout or php, ends the gate with that code and a message instead of waiting forever. Both slots now call occ_status directly so that exit reaches the gate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In swarm CI run 41206003651 (Nextcloud#11) the first occ maintenance:install failed on a config.php lock after the upstream entrypoint had already copied version.php. Upstream skips the install whenever version.php exists and never retries, so the replacement task served an uninstalled Nextcloud that occ status still reported as healthy, and slots 2 and 3 waited for installed: true until converge gave up after about 25 minutes.
Slot 1, and compose where TASK_SLOT is unset, now exits 1 with the cause and the recovery when version.php and occ exist, the init lock is free and occ status prints installed: false. The task crash-loops and the stack readiness check ends the deploy within minutes with that message in the service log. occ status runs with a 120 s timeout in both branches. Why the lock failed on NFS is not addressed here.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Swarm run 2 of the 061 Nextcloud step failed on every worker with "Assert Talk has the TURN server its calls relay through". The TURN server comes from web-svc-coturn's group membership (meta/services.yml standalone_enabled), which only the stack host carries; workers never run Talk, so the assert judged hosts it has nothing to say about. It came in with 5728502a19 and exists only on this line, not on main.
The assert now runs where IS_STACK_HOST holds: every compose host and the swarm manager, unchanged. test_talk_turn_assert.py evaluates the block and task guards plus the assertion with Jinja; against the unfixed file only the worker case fails.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in feature/updater 8c14873225: lockdown-probe with the Bluesky
broker key-rotation fix (0a6b6f9130) and the occ-only trusted_proxies
(406998f43a), the ERPNext server_scripts switch, and main with the
hotfixes #1290 (BBB apps-akka READ_COMMITTED), #1291 (FusionDirectory
PHP_MEMORY_LIMIT) and #1293 (Matrix variant 1 force_verification false).
Conflict resolutions:
- web-app-erpnext: create_site.yml takes updater's include of
restart_frappe.yml; that file's compose/swarm gates use
IS_COMPOSE_MODE / IS_SWARM_MODE instead of DEPLOYMENT_MODE literals.
- web-app-mailu, web-app-stalwart vars: store-i18n's RUF from
users.platform_administrator plus the new RUA vars; dmarc.rua_user is
platform_administrator (on store-i18n `administrator` is the host
account), so RUA and RUF share the app mailbox. READMEs follow.
- web-app-stalwart 09_dns-records.yml: store-i18n's block form with the
alias-domain MX task, plus rua=mailto in the DMARC record.
- web-app-pretix: store-i18n's source build stays; the version bump
lands as source_version v2026.9.0.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fresh compose deploy (compose-deploy-req061-4.log) spent 764.9 s in the plugin phase. Two parts of it did no work:
Enable stage after a fresh install: occ app:install runs enableApp and prints "<app> enabled" (core/Command/App/Install.php, stable34), so the app:enable that 03_enable.yml ran next only answered "<app> already enabled". The include, the exec, the classify, the skip notice and the queue cost 90.9 s for the 26 plugins installed fresh. nextcloud_install_status now reports enabled when occ exited 0 and printed an "<app> enabled" line; 02_install.yml then queues the plugin for the configure stage directly and runs 03_enable.yml only for an install that left the plugin off ("already installed" from a pinned archive or the recognize background job, or a future --keep-disabled). Queue order is unchanged.
Configure stage: 49 pending config:app:set calls ran as 49 occ processes in one sh -s and took 62.8 s, about 1.28 s of occ boot each. files/php/utils/occ_batch.php boots the console the way Nextcloud's console.php does (lib/base.php, OC\Console\Application::loadCommands, setAutoExit(false)) and runs Application::run once per call, so every call goes through the same command object, app loading, ConsoleEvent and exit code as occ, with one boot. The filter nextcloud_occ_config_set_batch builds the argv per entry; a unit test proves it equals shlex.split of the former shell line for typed, quoted, multi-line, empty and non-ASCII values. The calls travel as base64 JSON assigned in front of the script on stdin, so no value reaches a host or container argv; the first failing call stops the run and names appid/configkey on stderr, the task still reports changed on "set to".
Expected saving per fresh deploy: about 74 s (enable chain, net of the skipped include and the new set_fact, 0.64 s per plugin) plus about 60 s (configure exec down to one boot), about 134 s of 764.9 s. A converged mode=update deploy installs nothing and writes about 3 values, so it saves about 2.5 s; only a live deploy can confirm both numbers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
make autoformat output for three lines the feature/updater merge brought
in over 88 characters; matches feature/updater-lockdown-probe 88d138ba1c.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
make autoformat rewraps the firewall-backend entry in test_userland_proxy.py, the _direct_media call in test_talk_direct_media.py and the _git listing context in test_files.py, which ran past 88 characters. The text is ruff's own output, the same store-i18n commits, so the next merge between the lines resolves cleanly.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Brings in feature/updater accb22da75, which carries main 5995766f58.
Conflict resolutions:
- web-app-fider: main's tenant bootstrap (01_tenant.yml) runs first on
the stack host, OIDC setup moves to 02_setup_oidc.yml, design to
03_design.yml; flat includes so main's gate test reads them.
- sys-front-tls test_nginx_ssl_header: the tls fake maps the domain term
store-i18n passes back to its app.
- group_vars 04_webserver.yml, sys-svc-proxy CSP header: store-i18n's
webserver_resource lookup plus #1256's WEBSERVER_SECURITY_HEADERS and
the security_headers.lua include after the CSP block.
- web-app-nextcloud: updater's decoupled Talk stack (Janus, signaling,
NATS, recording builds) replaces aio-talk; store-i18n's per-network
Talk backends move from the wrapper script into signaling.conf.j2 urls
(NEXTCLOUD_HPB_SIGNALING_BACKEND_URLS from network_urls, canonical
first); IS_COMPOSE_MODE/IS_SWARM_MODE booleans kept; the integration
OpenProject digest stays delegated to STACK_HOST.
- users lookup docs, token-store write_many, cache README, meta info
lint: both sides combined.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On a clearnet deploy with self-signed TLS, test-talk-recording.js failed three times: the call recording stayed at state 4 (starting audio) because the recording server could not notify Nextcloud. Its log showed CERTIFICATE_VERIFY_FAILED on POST /ocs/v2.php/apps/spreed/api/v1/recording/backend, although compose-ca sets REQUESTS_CA_BUNDLE to the mounted bundle. nextcloud-talk-recording's BackendNotifier sends through `session.send(prepared, verify=True)`, and Session.send() skips the environment merge that only Session.request() does, so requests falls back to certifi's bundle, which does not know the injected root CA. A probe in the recording container's network namespace verified Nextcloud with the mounted bundle (200) and failed without it.
When the CA is injected, the recording image now points certifi's cacert.pem at the mounted CA bundle (the system CAs plus the injected root), in the same build step as the Firefox certificate policy. Verification stays on. Without an injected CA, as with real certificates, the build arg is empty and the image is unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
nextcloud/aio-talk bundled signaling, Janus, NATS and an onboard TURN server behind a start script that rewrote their configs, and AIO publishes only build stamps, so the updaters could not raise it and the role had to guard Janus against config drift. The backend now runs as talk (Janus built from a meetecho/janus-gateway tag, files/Dockerfiles/Janus), talk_signaling (upstream strukturag/nextcloud-spreed-signaling image), talk_nats (official nats image) and talk_recording (built from a nextcloud-talk-recording release tag, files/Dockerfiles/Recording), each semver-pinned, single-replica, healthchecked and resource-limited. The Dockerfiles take every value as a build arg from compose, and Janus and the recording server get their config path through `command:`.
The role renders janus.jcfg, the signaling server.conf and the recording server.conf; they match what AIO stamp 20260409_094910 generated apart from the service addresses and the session keys, which are now derived from the signaling secret. Secrets reach the services only through those files. Janus may join the host network in compose mode, which keeps direct media available there, and an assert refuses host mode in swarm. talk and talk_recording keep their keys, so one deploy replaces the AIO containers in place.
The onboard TURN server is gone with its ports, vars and credential, so Talk uses web-svc-coturn only, and the deploy now stops when Talk is enabled without it instead of handing out no STUN or TURN server. The recording image installs the injected root CA as a Firefox policy when the CA is mounted, so recordings work against self-signed TLS, and the recording scratch volume stays off NFS. The Janus drift guard and the janus_config_drift filter are removed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
services.talk.direct_media was an expression in meta/services.yml that read ansible_facts['all_ipv4_addresses'] and nested config lookups. lookup('config') renders it where the applications are merged without host facts; the render fails there, the lookup hands back the raw expression and `| bool` coerces it to False with a deprecation warning. Direct media therefore never switched on: both the first requirement-061 deploy and CI run 40877408319 logged the coercion, and a render in the development container returned the raw string without facts and true with them.
meta/services.yml keeps only the inventory switch, now `direct_media: false`, so every Talk stream goes through web-svc-coturn unless an operator opts a host in. That is the behaviour the bug produced everywhere, so no host changes on deploy, and it keeps Janus off the public network by default. vars/main.yml decides in task context from the switch, Talk, standalone TURN, the bridged network, compose mode and NEXTCLOUD_HPB_HOST_ADDRESSES, which reads the facts the way web-svc-coturn does. The flag tests render that chain against the merged applications: off by default, on for an opted-in compose host with coturn and an owned address, off for host networking, a foreign or loopback address, swarm, onboard TURN and missing facts. Administration.md documents the opt-in.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The pinned onlyoffice archive was unpacked with tar over custom_apps/onlyoffice on every deploy, so files that a newer release no longer ships stayed behind. On cloud.veen.world that left the 52 files of onlyoffice 9.15.1 (NC32 line) inside the 10.2.1 tree, and the integrity check reported them as EXTRA_FILE; the 52 equal exactly the 9.15.1 files absent from 10.2.1. replace_pinned_app.sh now unpacks the release beside the app, keeps the app when diff -rq finds it identical, and otherwise swaps the whole directory, so the next deploy heals the drifted tree and later deploys report ok. A broken archive fails before the live app is touched.
The real install task, run through ansible-playbook against a reproduced 9.15.1 plus 10.2.1 tree, changed it once to a tree identical to a clean 10.2.1 and stayed unchanged on the rerun; eleven unit cases cover identical, drifted, missing and broken inputs under sh and dash.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The admin overview reported that no server identifier is configured. Nextcloud 34 accepts 1 to 511, treats values below 1 as unset and asks for one NC_serverid per server when config.php is shared, which it is between nextcloud, cron, the task processing worker and the swarm replicas. A single serverid in config.php would let them mint identical Snowflake IDs in the same millisecond. The nextcloud service now carries NC_serverid 1 in compose and its task slot in swarm, so the check passes and replicas never share an id; the other services keep the hostname fallback.
A unit test renders the service environment in both modes and fails when the compose id leaves the accepted range.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Nextcloud 34 flags "Task Processing worker status: never run" because only occ taskprocessing:worker records core/ai.taskprocessing_worker_last_iteration, and the role ran background jobs through the cron container alone; since 32.0.7 the worker command is taskprocessing:worker, so background-job:worker would never clear the check. Add a taskprocessing service that reuses the cron image, mounts, healthcheck, restart policy and networks and runs occ taskprocessing:worker -t 60 in an in-container restart loop that backs off while Nextcloud is not yet installed, so swarm neither replaces the task every minute nor marks the stack fatal on a fresh install. It scales with the swarm topology, since task claims use FOR UPDATE SKIP LOCKED.
Rendered compose and swarm stacks only gain the taskprocessing service and validate against the compose schema; the new unit tests fail when the worker mount is dropped or background-job:worker is used instead.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
68 conflicts, resolved per hunk; resolutions agreed with the Updater
session were applied as agreed:
- svc-db-postgres (superuser password script and core) and
sys-svc-rdbms realign_dedicated_db: updater's argv/stdin redesign with
expand_argument_vars: false, nested into store-i18n's block structure.
- svc-db-redis 01_init: store-i18n's ACL password on stdin.
- sys-svc-proxy forwarded.conf.j2: store-i18n's handover-aware file,
without the forwarded_base include (Host and X-Forwarded-Proto would
be sent twice); forwarded_base serves the inline consumers, e.g.
web-app-seaweedfs nginx.conf.j2.
- basic.conf.j2 and web-app-openproject proxy.conf.j2: issuer redirect,
then handover, else updater's location/sso_acl.conf.j2; the
server-level webserver_extra_configuration hook stays.
- web-app-keycloak: updater's admin-REST settings step and renumbered
update steps; update/01_client.yml removed; store-i18n's design step
and LDAP block kept; a73435cc9d arrived through rename detection.
- web-app-nextcloud OpenProject provisioning: updater's rewrite, all
credentials on the Ruby runner's stdin.
- web-app-semaphore: store-i18n's canonical web root (a1d7835863).
- Role READMEs: no generated Cosmos/Architecture/Quick Setup/Credits
sections; desk-* paths resolved to the dsk-* renames.
Integration fixes for what merged cleanly but did not fit together:
- web-app-matrix ansible vars: drop updater's second
matrix_client_element_configuration_extension_json, which silently
overrode store-i18n's design-aware value.
- url.origin instead of a stripped url.base in 19 places store-i18n
added since 44aaa68924, same lookup targets (the double-slash lint).
- Package batch (18ea75049e, cb4d340de9): installs the deploy-start
batch covers are removed (caffeine, cursor, blender, npm in dsk-gnt-*,
with shared references), the rest carry batch: false and a nocheck
with the verified reason (PyPI fallback, before the batch, vendor
repository, configured flavor only, addon gate); dev-npm references
the root npm id.
- sys-token-store write_many.yml: store-i18n's change-gated block and
IS_SWARM_MODE.
- web-app-openproject: IS_COMPOSE_MODE in 02_migrate/run.yml; the
seeded-admin lock (ccbf8f168d) ported into 04_admin/provision.yml and
covered by the admin-state digest.
- Bucket policy (3b4ac40df5 and follow-ups) moved to web-svc-seaweedfs,
where store-i18n's object store lives; the S3 admin secret travels on
stdin instead of a task-level environment.
- act workflow.sh: no Exception-labelled comment.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ruff format reflows the package batch, registry, plan and invocation code and tests and the Stalwart and Nextcloud tests, ansible-lint rewraps long Jinja lines in folded scalars of sys-service, Nextcloud and SeaweedFS, mbake restores the Makefile's final newline, and the Nextcloud OAuth test merges its nested with statements (SIM117).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Connecting a user to OpenProject failed: the instance URL carried url.base's trailing slash, so Nextcloud redeemed the code at //oauth/token (HTTP 500), integration_openproject stayed in its fresh-setup state so isAdminConfigOk was false and no Connect button rendered, and duplicate OAuth clients piled up across deploys. The coupling now converges on each deploy: a PHP helper creates or repairs the Nextcloud OAuth2 client (numeric id, OpenProject callback, same-name duplicates removed); one rails runner call ensures the storage, its OAuth application and the Nextcloud client on it, renewing the OpenProject secret when Nextcloud holds none or a stale one, or when rotate_oauth_client forces it, and realigning the application's redirect_uri after a host change; nc_oauth_client_id is stored only after that call succeeded. openproject_instance_url has no trailing slash and fresh_project_folder_setup is 0.
No secret travels in argv or environment: on the way to OpenProject they ride the same stdin as the Ruby script, as a PROVISION_INPUT heredoc, and the credentials go into Nextcloud through occ lines on the container shell's stdin, taken verbatim from 7f72e53a58 on feature/store-i18n. A Ruby truth table over seven scenarios, with the staged state as negative control, covers fresh install, rerun, a Nextcloud that lost client and secret, a stale secret, forced rotation and a host change. A new Playwright test connects a user through the OAuth round trip and expects Nextcloud's oauth-redirect below 500 and 'Connected as'.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
recognize's app:install downloads about 700 MB of models and held the
plugin phase for minutes. An addon may now set
config.background_install; the plan files such a plugin under the new
background list unless it is pinned through archive or mandatory.
01_background.yml starts occ app:install as an async job and holds
every later occ call at a gate until the app's schema is applied:
Nextcloud migrations diff a schema snapshot, so DDL running beside
another occ call can be reverted. The gate reads the async job file
and files/php/utils/migration_state.php, which prints the versions
oc_migrations records and the versions the app's migration folder
holds. occ migrations:status is not usable, Nextcloud registers it only
with debug on. While the app directory does not exist yet the
MigrationService constructor throws and lib/base.php's ErrorHandler
only logs it, so the probe exits 0 with nothing on stdout and the gate
waits.
The install loop then runs while the job finishes; the always section
joins every job with async_status and fails on an exit code that a
timeout or signal produced, because the php process may then still run
inside the container. 02_install.yml afterwards answers "already
installed" and enables and configures the plugin, or installs it with
its own retries when the job failed before placing the app.
The stage files are renumbered (01_install -> 02_install, 02_enable ->
03_enable, 03_configure -> 04_configure) so the numbers follow the
include order the task-numbering lint checks.
Local perf run: plugin phase 538 s -> 282 s, gate open after 30.2 s,
recognize 12.0.2 enabled with 8/8 migrations, failed=0.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
roles/web-app-nextcloud/tasks/01_fullstack.yml: both sides added a
replicated-template task at the same spot; store-i18n's Talk signaling
backend list and the updater's Janus config now stay as two tasks with
their own conditions.
roles/web-app-nextcloud/templates/compose.yml.j2: the updater rendered
the talk service's container_volumes a second time inside the direct-media
branch, next to store-i18n's unconditional call, which doubles the
volumes/configs keys once direct media is on. The janus_conf mount carries
its own direct_media `when` in meta/volumes.yml, so the single call
renders it; the duplicate is gone.
DEPLOYMENT_MODE == 'compose' became IS_COMPOSE_MODE | bool in
web-app-nextcloud/meta/services.yml, web-svc-coturn/vars/main.yml and
tasks/stages/03_destructor.yml, as store-i18n's deployment-mode lint
requires; the updater's coturn, Talk and decommission tests now pass
IS_COMPOSE_MODE alongside DEPLOYMENT_MODE. test_talk_direct_media.py
expects store-i18n's talk command line and mount position, and the
"host networking ignores direct media" check moved to the config level,
where direct_media already requires network_mode bridge.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CodeQL flagged five notes on PR #1215: four test modules imported unittest both as a module and through from-import, and sys-front-inj-matomo's __init__ exported PROJECT_ROOT without declaring it, so the import from test_site_cache.py read as unused. The modules now bind mock through import unittest.mock as mock, and the package declares __all__, the same fixes main carries from 3610e7b9ef and 6e776a5780. The same pattern is fixed in the new Talk direct-media test and the decommission integration package, and the coturn test package drops an __init__ export nothing imports.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
janus_conf is mounted only when direct media is on, which requires compose mode, and compose ignores a config mode outside swarm and warns about it on every deploy. The file keeps the 0644 it is rendered with on the host.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The aio-talk image runs Janus with its stock janus.jcfg: no rtp_port_range and no nat_1_1_mapping, so Janus offered only its bridge address and every stream of a call went through coturn as relay. Calls above about six participants stuttered and failed.
When talk runs bridged next to standalone coturn in compose mode on a host that owns networks.internet.ip4, a rendered janus.jcfg is mounted over the stock one with rtp_port_range 61000-61999, nat_1_1_mapping = ip4 and keep_private_host, and that UDP range is published on ip4. TURN stays the fallback. Before rendering, the deploy reads the image's launcher and stock janus.jcfg and stops when Janus no longer reads the mounted path or the stock settings drifted from the template. Variant 0 publishes 64 ports so dev and CI exercise the path; the call capacity spec asserts a non-relay selected pair on ip4 in that range. With the flag off every rendered file is byte-identical to before.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each plugin ran its own occ config:app:set per configuration value right after its install or enable, and a redeploy rewrote every value. Install and enable now queue the plugin; the configure stage runs once for the queue: it loads every queued addon spec, reads occ config:list once, keeps only the values Nextcloud does not hold exactly (nextcloud_plugin_config_pending; typed, masked and structured values are always kept), and runs the hooks in queue order afterwards.
Combined with store-i18n's 7f72e53a58: the pending occ config:app:set calls go on the stdin of one container exec -i ... sh -s, so no value (Talk, OnlyOffice, whiteboard, BBB secrets) reaches a host argv; a failing call names its appid/configkey.
Replay harness old vs new: identical final state, updated keys and hook order in first-deploy, converged, drift and failure scenarios (failure stops at the same key); converged config:app:set calls 19 -> 3. Merge note: resolve 03_configure.yml with this side against 7f72e53a58.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every redeploy ran occ maintenance:repair --include-expensive (43.5 s, maintenance mode on) and listed the appstore updates through Tor (49.7 s) although the version had not changed. 03_upgrade.yml now reads occ status and files/mimetype_version after occ upgrade; nextcloud_needs_post_upgrade decides whether the pass is due (version changed in this deploy, or the expensive repair has not run for the running version). A due deploy includes 03_upgrade/01_post_upgrade.yml with today's app update and repair; an unchanged one includes 03_upgrade/02_cached_updates.yml, which updates only the unpinned enabled apps whose release stored by the updatenotification background job is newer (nextcloud_cached_app_updates, PHP version_compare semantics). NEXTCLOUD_PLUGINS_PINNED is the single source of the pinned check.
Verified against echoserver's live occ output (status, marker, updatenotification cache, app:list, with debug log lines): unchanged redeploy is not due and finds no update. Scenario harness 24/24 incl. negative controls on the old file. About 86 s per unchanged redeploy.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every addon ran through the install, enable and configure stages on its own, and each stage asked occ for the plugin's state again. The role now reads occ app:list --output=json once, and the new nextcloud_plugin_plan filter replays the addon order (an enabled entry switches its incompatible_plugins off and itself on, a disabled entry switches itself off, the later entry wins) and splits the app ids into disable, install, enable and configure. One app:disable call takes every id that has to go off; the install, enable and configure includes loop only over the ids that need that stage, so an already enabled plugin skips straight to its configuration. A plugin shipped as a pinned archive still goes through install on every run.
On echoserver the plugin phase took 608 s in full-9 for 51 addons; a repeat deploy is derived at about 195 s instead of about 550 s.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Nextcloud refuses to skip a major. Before the pinned image starts, the
role reads $OC_Version from version.php in the code volume, plans one tag
per major between that version and the pin, and runs the image of each
step once against the data (occ upgrade through the entrypoint, with the
debug log lines filtered out of the output). Then the regular manager ops
run on the pinned image and assert that occ status reports the pinned
major. A fresh volume plans no step; swarm stops when data needs one.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Declare `__all__` in every package `__init__.py` whose `PROJECT_ROOT`
is imported by its modules (163 packages), so the constant counts as
exported; list every public name where the package defines more.
- Empty the 15 package `__init__.py` files whose `PROJECT_ROOT` no module
imports.
- Declare the public API of `utils.github.variant.axes` in `__all__`,
re-exports included, instead of silencing them with `noqa: F401`; the
list matches the one `feature/store-i18n` already carries.
- Use `import unittest.mock as mock` in `test_applications.py`, as the
rest of the suite does.
- Spell `_NON_PRINTABLE` in `utils/yaml_bootstrap.py` as the equivalent
BMP-only positive class; it matches exactly the same 2111 code points
as the old negated class across U+0000..U+10FFFF.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`make quality-high` on main (fea6f31d9f) rewrote two files in its
autoformat step:
- roles/sys-service/handlers/refresh_systemctl_service.yml: the
handler's `when` was wrapped by hand; the formatter wraps it at line
width. Both forms parse to the same YAML.
- tests/unit/python/roles/web-app-nextcloud/test_onlyoffice_check_deadline.py:
two over-long assertIn calls that ruff format splits.
A second autoformat run leaves the tree unchanged, so main is stable
under the formatter again.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`retries: 30` with `delay: 10` only bounded the wait while each
`occ onlyoffice:documentserver --check` failed fast. onlyoffice v10.2.1
gives the health check and the version command 60 s each and the
conversion 120 s, so a document server that accepts connections but
does not answer held the deploy for up to two hours before it failed.
The retries now also stop once a deadline of
`lookup('timeout', 300)` seconds, set right before the check, has
passed. A per-attempt `timeout:` would not bound the total: ansible-core
2.21 leaves the `until` loop on the first timed-out attempt, so only a
limit below onlyoffice's 60 s would end a hanging server's loop.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The role writes onlyoffice's settings with one occ call per key,
DocumentServerUrl first and jwt_secret fourth. Nextcloud loads app
config once per process, so a cron.php that starts inside that batch
runs onlyoffice's EditorsCheck job, registered when the app was
enabled, with the document server URL but without the secret. The
document server answers "Invalid token", EditorsCheck stores it in
settings_error, and onlyoffice stops loading its Files integration;
EditorsCheck itself skips every later run while that error is set.
Seen in run 38455782184 (Nextcloud#4, swarm): cron.php started at
06:50:00 inside the batch that ran from 06:49:59 to 06:50:04, logged
"Invalid token" at 06:50:22, and both Playwright passes found .docx
files offered for download only.
DocumentServerUrl is now written last, so a process that cached the
config before the batch ended sees no URL and EditorsCheck returns
early. After the batch, `occ onlyoffice:documentserver --check` sets
settings_error from a fresh check: a redeploy clears an error latched
earlier, and a document server that stays unreachable fails the
deploy. A unit test keeps the URL last.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ONLYOFFICE listed onlyoffice 10.2.0 in the Nextcloud appstore on 2026-09-18 at
10:16:31Z and never published the matching GitHub artifact; the tag itself does
not exist. `occ app:install` takes the newest compatible release, so it dies on
a 404 that no retry can heal. CI proved that twice independently: upstream run
35329488812 and fork run 35330645276 both fail Nextcloud#4 at that task after
six attempts, on the same commit, for the same reason.
The fault had three layers, each visible only once the one above it was gone.
Install. meta/addons/onlyoffice.yml pins `config.archive` to v10.1.2, whose
artifact answers 200 and which the appstore also lists for this server major.
01_install.yml stages that archive on the node and unpacks it into custom_apps
through a stdin pipe: the webroot is a named volume, so the node can only reach
the app directory through the container, and piping avoids both `container cp`
and a cleanup step that would have to run as a user which does not own the file.
Enable and configure needed no change at all. With the release already on disk
`occ app:install` reports "already installed", which nextcloud_install_status
has always mapped to runnable, so 02_enable.yml and 03_configure.yml proceed as
they always did. That is why this commit adds no branch to the plugin pipeline.
Update. `occ app:update --all` then dragged the pinned app back to the broken
10.2.0 and killed the deploy. occ offers no exclusion flag, but it does offer
`--showonly` and single app ids, so a pinned app is simply not updated. The old
`--all` call stays byte-identical and now carries a `when`: with nothing pinned,
thirteen of the fourteen variants take exactly the path they took before. This
layer cannot appear in CI at all, because it needs a second deploy with the app
already installed, and the install always died first.
The classifier test is inverted rather than deleted:
test_an_advertised_release_whose_artifact_is_gone_stays_fatal pins the decision
that a dead artifact fails loudly instead of degrading to a silent skip.
Verified by a local compose deploy of variant 4: ok=2508 changed=149 failed=0
and `✓ 36 tests/addons/onlyoffice.spec.js:12:1`, which drives the whole
JWT-authenticated round trip to the document server over its onion address, not
merely an enabled plugin. In the live container beforehand: `occ app:enable`
returned "onlyoffice 10.1.2 enabled", `occ app:install` returned "already
installed", and `occ app:update onlyoffice` reproduced the 404 that the pin now
avoids.
The loop expression was wrong before this commit and is fixed here: Jinja reads
'\1' as an octal escape, so the backreference produced chr(1) instead of an app
id. Evaluated against ansible-core's own filters over four shapes, the tail is
now cut instead of captured, which removes the escaping hazard entirely.
occ output is not localised: with default_language=de and a de_DE.UTF-8 locale
the strings are unchanged, so parsing them is as safe as the five markers the
role already relied on.
Residual: the per-app update branch has never run with a real item inside a
deploy. Its chain and `occ app:update <id>` are each verified separately.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
meta/addons/oidc_login.yml gates on lookup('sso_oidc_plugin'), which the applications render evaluates while it is still in flight. Under that re-entry guard the merged view hands out service flags as their own Jinja text, and bool("{{ ... }}") is True. The lookup read services.sso.enabled and services.ldap.enabled as enabled on a deployment with neither Keycloak nor OpenLDAP and announced oidc_login, so Nextcloud advertised a login hand-off to a provider that is not there.
lookup('config', ...) renders the value it returns, which is why the roughly ninety '| bool' sites in meta/ do not have this problem. 238bc285c7 established reaching the merged config through the lookup loader; this one had not been converted. utils.roles.applications.config.get is the raw accessor and does not render.
Measured against the real render: with only web-app-nextcloud deployed the addon set is now empty; adding Keycloak yields sociallogin; adding OpenLDAP as well yields oidc_login. That is the resolution order the lookup's own docstring documents. All 14 nextcloud variants pin both flags as literal booleans, so no CI-deployed variant changes.
The stub next to the plugin answered every lookup with the applications payload; it now mirrors lookup('config', app, path, default). A stub resolves a path but does not render it, so it cannot see this defect at all: the regression test renders for real.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sociallogin creates a Nextcloud user as "<provider name>-<keycloak sub>",
and the provider name was the Keycloak domain. On an onion variant that
domain alone is 67 chars, so every OIDC callback died with
InvalidArgumentException "Username is too long" and HTTP 500 at
/apps/sociallogin/custom_oidc/auth.<onion> (run 34615516009,
Nextcloud#1, 204 occurrences). The whiteboard and admin OIDC specs failed
on the same login.
This is the second derived identifier an onion domain overflows; the
kernel hostname is the first, clamped to 63 bytes by container_hostname
since the pretix and bookwyrm fixes. The domain-or-entity rule now lives in
one lookup, bounded_name(application_id, max_length): the primary domain
when it fits, otherwise the entity name. container_hostname delegates to it
at 63 and keeps its eight callers and its behaviour.
The provider name is bounded_name('web-app-keycloak', 64 - 1 - 36), the uid
limit minus the separator and the UUID sub. A short clearnet domain stays
the name, so existing users keep their uids; an onion deploy gets
"keycloak", which, unlike a hash of the domain, survives a rotated onion
address.
Tests cover the lookup's fit, boundary and fallback cases and its argument
validation, container_hostname's delegation at 63, and the provider name's
room for the sub.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
