`retries: 30` with `delay: 10` only bounded the wait while each
`occ onlyoffice:documentserver --check` failed fast. onlyoffice v10.2.1
gives the health check and the version command 60 s each and the
conversion 120 s, so a document server that accepts connections but
does not answer held the deploy for up to two hours before it failed.
The retries now also stop once a deadline of
`lookup('timeout', 300)` seconds, set right before the check, has
passed. A per-attempt `timeout:` would not bound the total: ansible-core
2.21 leaves the `until` loop on the first timed-out attempt, so only a
limit below onlyoffice's 60 s would end a hanging server's loop.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The role writes onlyoffice's settings with one occ call per key,
DocumentServerUrl first and jwt_secret fourth. Nextcloud loads app
config once per process, so a cron.php that starts inside that batch
runs onlyoffice's EditorsCheck job, registered when the app was
enabled, with the document server URL but without the secret. The
document server answers "Invalid token", EditorsCheck stores it in
settings_error, and onlyoffice stops loading its Files integration;
EditorsCheck itself skips every later run while that error is set.
Seen in run 38455782184 (Nextcloud#4, swarm): cron.php started at
06:50:00 inside the batch that ran from 06:49:59 to 06:50:04, logged
"Invalid token" at 06:50:22, and both Playwright passes found .docx
files offered for download only.
DocumentServerUrl is now written last, so a process that cached the
config before the batch ended sees no URL and EditorsCheck returns
early. After the batch, `occ onlyoffice:documentserver --check` sets
settings_error from a fresh check: a redeploy clears an error latched
earlier, and a document server that stays unreachable fails the
deploy. A unit test keeps the URL last.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
occ config:app:set asks for a confirmation when it is given a type, and
without a terminal it aborts with exit code 1. The typed Talk server
lists from the previous commit therefore stopped the deploy at the
plugin configuration.
Entries with a type are now written with --no-interaction. Measured on a
live Nextcloud 35 with dummy values: stun_servers, turn_servers and
recording_servers are stored as arrays, Talk lists exactly the written
servers, and a second identical call leaves the value alone.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Talk 25, the release for Nextcloud 35, reads stun_servers and
turn_servers through the typed app config and declares both as arrays.
Talk 24 read them as JSON strings. The role wrote them without a type,
so on Nextcloud 35 the Talk settings showed no STUN and no TURN server
although the deploy had set both.
A plugin configuration entry may now carry a type, which is passed to
occ config:app:set. The three Talk server lists use it. Entries without
a type are written as before.
Not yet run against a live Nextcloud 35.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ONLYOFFICE listed onlyoffice 10.2.0 in the Nextcloud appstore on 2026-09-18 at
10:16:31Z and never published the matching GitHub artifact; the tag itself does
not exist. `occ app:install` takes the newest compatible release, so it dies on
a 404 that no retry can heal. CI proved that twice independently: upstream run
35329488812 and fork run 35330645276 both fail Nextcloud#4 at that task after
six attempts, on the same commit, for the same reason.
The fault had three layers, each visible only once the one above it was gone.
Install. meta/addons/onlyoffice.yml pins `config.archive` to v10.1.2, whose
artifact answers 200 and which the appstore also lists for this server major.
01_install.yml stages that archive on the node and unpacks it into custom_apps
through a stdin pipe: the webroot is a named volume, so the node can only reach
the app directory through the container, and piping avoids both `container cp`
and a cleanup step that would have to run as a user which does not own the file.
Enable and configure needed no change at all. With the release already on disk
`occ app:install` reports "already installed", which nextcloud_install_status
has always mapped to runnable, so 02_enable.yml and 03_configure.yml proceed as
they always did. That is why this commit adds no branch to the plugin pipeline.
Update. `occ app:update --all` then dragged the pinned app back to the broken
10.2.0 and killed the deploy. occ offers no exclusion flag, but it does offer
`--showonly` and single app ids, so a pinned app is simply not updated. The old
`--all` call stays byte-identical and now carries a `when`: with nothing pinned,
thirteen of the fourteen variants take exactly the path they took before. This
layer cannot appear in CI at all, because it needs a second deploy with the app
already installed, and the install always died first.
The classifier test is inverted rather than deleted:
test_an_advertised_release_whose_artifact_is_gone_stays_fatal pins the decision
that a dead artifact fails loudly instead of degrading to a silent skip.
Verified by a local compose deploy of variant 4: ok=2508 changed=149 failed=0
and `✓ 36 tests/addons/onlyoffice.spec.js:12:1`, which drives the whole
JWT-authenticated round trip to the document server over its onion address, not
merely an enabled plugin. In the live container beforehand: `occ app:enable`
returned "onlyoffice 10.1.2 enabled", `occ app:install` returned "already
installed", and `occ app:update onlyoffice` reproduced the 404 that the pin now
avoids.
The loop expression was wrong before this commit and is fixed here: Jinja reads
'\1' as an octal escape, so the backreference produced chr(1) instead of an app
id. Evaluated against ansible-core's own filters over four shapes, the tail is
now cut instead of captured, which removes the escaping hazard entirely.
occ output is not localised: with default_language=de and a de_DE.UTF-8 locale
the strings are unchanged, so parsing them is as safe as the five markers the
role already relied on.
Residual: the per-app update branch has never run with a real item inside a
deploy. Its chain and `occ app:update <id>` are each verified separately.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Run 34421944328 Nextcloud#1 timed out "waiting for a signed-in Nextcloud
shell after the login redirect". 4164f37981 turned sso on in variant 1
while ldap stays off there, so that round logs in through sociallogin
instead of oidc_login - the first variant, on this branch or on main,
that runs sociallogin against an onion issuer. oidc_login goes through
Nextcloud's HTTP client, which honours the proxy in proxy.config.php;
sociallogin builds its own client and reads only
social_login_http_client['proxy'], so its token exchange never reached
the onion Keycloak. The template now sets the same tor SOCKS proxy for
it, inside the existing "sso on and the OIDC host is an onion" branch.
The spec only treated 5xx documents as a page failure, so a refused
callback surfaced as a 60s (x5 over tor) wait for a shell that could not
come. A 4xx on /apps/sociallogin/ or /apps/oidc_login/ now counts as
the page itself failing, reported without the query so no code leaks.
The plugin loop printed every item, including the OIDC client secret,
into the include line of a public CI log; it now labels by plugin key and
honours MASK_CREDENTIALS_IN_LOGS.
_shared.js grew past 250 lines; its page and modal helpers move to
_page.js and _shared.js re-exports them, with an identical export
surface.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The hub serves 94 tools across 23 categories, write tools included, and enabling the ExApp switches every category on. That surface was published on the public vhost and every discovered client was handed an app password for it. svc-ai-mcp-adapter now sits in front in mcp_passthrough mode: clients present credentials.mcp_bearer to contextagentmcp and reach the nine read tools of calendar, contacts and files, which files/mcp/tools.json names and the sidecar enforces per call. Those three categories are the ones whose upstream is_available is unconditional, so the served set does not vary with which optional apps a deployment installs.
Enforcing it in the sidecar rather than in each client is what makes it hold: web-app-openwebui renders no per-tool filter at all, so a client-side include list would have bounded two consumers out of three.
06c13e7f3e withdrew the declared path from the edge for exactly this reason - "that surface is what the adapter exists to narrow". Nextcloud answers the same route under a second spelling, without the /index.php front controller, which try_files rewrites back onto the route after the edge has already declined to block it. The withdrawal becomes a regex covering both, and only where stripping the front controller changes the path, so the four adapter providers without one render unchanged.
The app password is minted after the stack renders, so it reaches the sidecar through upstream.env and rebuild.yml rather than the compose environment: compose up resolves env_file while creating the container, a restart does not. The sidecar is the last service in the template because a Jinja set has no block scope and its env_file would otherwise displace the one talk, whiteboard and talk_recording each carry.
Two suites cover what nothing measured: the vhost template rendered without a container, asserting both spellings are withdrawn, and every shipped tools.json loaded through the adapter own loader, so a bad contract fails in a second instead of as a probe timing out an hour into a deploy. The guest spec probed mcp.endpoint.path, which this change makes the proxy answer 404 for its own reasons; it now probes all three public paths and requires no MCP body.
Not yet deployed: the sidecar build, the upstream.env delivery and the hub answering exactly nine tools are unverified against a running stack.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An MCP client spec signs in through the client's own OIDC button, which
redirects to Keycloak. Every provider put its MCP pairing in the lean variant,
and that variant pinned sso off - so the round that exists to prove the pairing
was the one round without an identity provider. Run 33574056480 shows what that
costs: openwebui's MCP specs reached _shared.js:131 and died on "openwebui OIDC
sign-in button must be visible on the auth page".
The rule now demands two things of the variant that carries the consumers: the
provider pins sso on, and each admitted consumer, in the variant that round
actually pulls in, does not pin it off. The second half matters because a round
deploys every dependency at its own index and falls back to variant 0 only when
the dependency has fewer, so the client shipping alongside is that one and not
the client's variant 0. A role that declares no sso, or pins it to a literal
false because upstream speaks no OIDC, is exempt from a flag it cannot have.
Twenty roles moved to satisfy it. Ten already had a second sso: false pin and
only needed the flip. Six could host the pairing in variant 0, which already
has sso on, and dropped back to two variants. Six kept a third lean variant to
carry the false polarity variants-coverage wants. moodle and mattermost took
swaps instead: moodle's variant 1 is the mandated LDAP-only branch, and
mattermost was already at the three-variant cap with sso on in its variant 2.
nextcloud also turns context_agent on in that round, the ExApp that serves the
endpoint - with it off the round proved nothing it was built to prove.
The rule moves to tests/lint/ansible/services/ because it reads meta YAML and
executes nothing, which is where integration.md says such a check belongs, and
its suppression key joins the catalog it was never listed in.
nextcloud_exapp_reregister becomes nextcloud_exapp_renew: mcp-secret-masking
matches `register:` unanchored, so the substring in the old name read as a task
that prints a secret. register.yml already masks the two tasks that carry one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Context Agent registration ran once and never again, so a rotation of the
app secret left Nextcloud holding the old one. AppAPI then refused both ways -
"Invalid App secret:" in the ExApp and "Invalid signature for ExApp:
context_agent" in nextcloud.log - and the deploy either died at the
registration or later at the endpoint answering 401 with a bearer nothing
would accept.
The role already escalates on that exact signal for the app password: probe
the endpoint, mint a fresh password when it refuses, probe again, and refuse
the deploy if it still will not answer 200. The ExApp joins that ladder as the
next rung. When the endpoint still rejects after a fresh password, the
registration itself is what is stale, so it is dropped and made again, and the
endpoint is asked once more before the assert has the final word.
Registering is deliberately driven by the probe rather than by a recorded
fingerprint of the declared secret. A fingerprint has to be written by a
registration, so every instance already registered would have read no
fingerprint, judged itself stale, and paid a re-registration on the first
deploy after this shipped - and that carries --wait-finish, the ExApp init
that costs a tor round an hour it does not have. Keyed on the probe, a healthy
deployment does nothing at all, and the cost falls only where the endpoint is
already broken.
The caller says which of the two it wants; neither call site leaves it to a
default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The contract check ran as an Ansible task in the middle of every provider deploy, and the two client checks ran the same way. A test that is part of the deploy cannot be run without deploying, cannot be re-run on its own, and reports its verdict as a deploy failure. Twenty-seven task files lose 491 lines to this.
It is now what it always was. Each provider ships files/test/test.sh, templates/test.env.j2 and meta/tests.yml, and test-e2e-cli stages one shared harness beside it: run.sh drives contract.sh in a throwaway container on the provider's own network, then isolation.sh for adapter-fronted providers. The read probe each provider offers moved from the calling task into meta/mcp.yml next to the allowlist it must satisfy, and the schema now rejects a read probe naming a tool outside that allowlist.
The bearer the test presents is the one a client resolves through mcp.credential, via the new mcp_client_bearer lookup over resolve_credential, which moved to utils.roles.applications.mcp so it is the same function mcp_servers renders every client connection with. The probe presented whatever the calling role passed, which for the fifteen adapter-fronted providers is a different secret than any client holds.
Sidecar isolation is read from the engine rather than measured inside the container, so it no longer needs a delegation to the node holding the task: attachment is compared by network name, which drops the gwbridge special case, and a swarm worker says it cannot measure instead of skipping silently.
Retrying is bounded by wall clock against MCP_DEADLINE_SECONDS, which scales through the same lookup('timeout') as the harness budget it has to stay under. A flag that is neither true nor false aborts rather than reading as off.
mcp-provider-probe becomes mcp-provider-cli-test: an enabled provider must ship the test rather than include the task. mcp-probe-bearer becomes mcp-adapter-bearer, tying ADAPTER_BEARER to the declared credential key now that no probe include carries it. A new integration test renders the shared env template for every provider through Ansible's own templar, so a later change to it breaks visibly instead of poisoning twenty deploys quietly.
Hermes and OpenClaw keep their client-side proof as CLI tests of their own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nextcloud seeds the administrator once, at install time, from NEXTCLOUD_ADMIN_USER
and NEXTCLOUD_ADMIN_PASSWORD in the environment file, and no later run touches the
account again. A rotated credential therefore leaves the instance authenticating
against the old secret while every task in the role still reports success -- which
is exactly what the CI proves against, rotating every inventory credential between
the sync and the async pass.
The reset runs only when the declared password differs from the one this script
last applied, compared through a sha256 marker kept in the config volume. An
unconditional reset would have worked too, but it reports changed on every single
run and turns a rotation check into noise. The marker's cost is stated in the
script header: a password changed outside this script is not noticed.
The comparison cannot be an authentication probe. Nextcloud has no occ command
that verifies a password, and the HTTP endpoints that would are not reachable from
inside an fpm image, where no web server listens in the same container.
OC_PASS reaches occ through the environment and never through an argument, so the
secret stays out of the process list on both the host and the container, the same
way dc32574873 took the redis password off the exec command line.
The account this touches is the local administrator. The LDAP resolution in
07_admin_identities.yml only collects additional identities for the same person,
and the MCP service account reset added in e3caa074b2 targets a different user.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1f870b1e5 left two marks the layout rules reject. friendica's utils/ held
patch_config.yml beside patch_db_credentials.yml, which is a shared
delimiter-bounded prefix, and its numbered files ran 00, 01, 02, 04, 05 with
nothing at 03 after the config patch moved out of the sequence.
The prefix becomes the folder it was standing in for, the way 8e8dbbd08c did it
for the AI files. All four roles move, not only the one the lint could see: the
path post_deploy.yml resolves a role's declaration through has to read the same
everywhere, so a folder in one role and a prefix in the other three would be no
domain at all.
utils/patch_db_credentials.yml -> utils/patch/db_credentials.yml (4 roles)
utils/patch_config.yml -> utils/patch/config.yml (friendica)
The numbers close up behind the file that left: 04_addons and 05_user_bootstrap
become 03 and 04.
Four references follow: the include in 01_manager_ops.yml, the lookup in
post_deploy.yml, the caller path in the unit test that reads each role's
declaration, and a requirement doc that linked the old addon file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3831c8f804 gave matomo, moodle and nextcloud a patch each, called from their
own 00_core.yml. That file only runs while the role's run-once flag is unset,
so the patch runs on the deploy's first pass and never on the second -- which is
the only pass where a rotated credential exists. The app keeps the password it
was installed with while the engine moves on, and every request answers 500.
Run 33317483326 shows it on friendica, which had the same shape: svc-db-mariadb
altered the user at 15:39 and again at 16:26, either side of the 15:58 rotation,
while the app's config was patched once. The engine said what followed:
[Warning] Access denied for user 'friendica'@'192.168.201.4' (using password: YES)
Five failed healthchecks in a row, swarm replaced the task, and the convergence
gate reported "tasks not running yet" for a service that started correctly every
time.
The call moves to post_deploy.yml, which runs after every deploy and sits behind
no role's flag. It includes roles/<app>/tasks/utils/patch_db_credentials.yml when
that file exists, so a role declares what to patch and nothing else: friendica
joins the other three with its volume, its config path and its expression.
It cannot live in the role. 00_core.yml has to set the run-once flag as its very
first task or load/app.yml re-includes the role and recurses, which
tests/lint/ansible/roles/test_service_core_first_task_run_once.py enforces, and
everything after that flag is closed to a second invocation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The body was a Jinja-interpolated shell string inside the task. As a file it
can be read, shellchecked and run, and the three parameters travel through the
environment the way the password already did.
That forces one thing into the open. The old expression carried
${PATCH_PASSWORD} and the shell expanded it while assembling the command; a
value arriving in an environment variable is not expanded a second time, so the
substitution has to be explicit. The consumers now write @PASSWORD@ and the
script replaces it, which is also the version a reader can follow without
knowing that rule.
The script refuses to run without its parameters rather than treating an unset
one as empty, because an empty volume name would look exactly like a node that
does not hold the volume: a silent success on every host.
Tests move with it, to the files/ segment every other test of a shipped script
uses. They run it against a stubbed `container` in three states per consumer -
no volume, volume without the config, both present - and assert the marker the
task reads as a change. Five mutations of the script were checked against them:
dropping the substitution, the marker, the existence test, the parameter guards
and the volume filter each turn the suite red.
Layout follows what the linters already require: files/shell/ is enforced by
role-files-layout, and 65e34746fc names ansible.builtin.script as the sanctioned
way to run one from a task.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Baserow's image mounts the MCP app on the ASGI root unconditionally, so
mcp.enabled only decides whether an endpoint record exists. The
application's own vhost kept routing /mcp/*: switched off it answered
401 Endpoint not found., and a refused credential is still a surface.
Switched on it is worse - the adapter narrows fourteen upstream tools to
four, and the publicly routed native path reaches all fourteen for
anyone holding the endpoint key. web-app-homeassistant had the same
shape, an exposure: internal surface reachable publicly. Both now 404
there and stay reachable on the container network.
The public surfaces get the edge limits the contract already declared.
Zones are written per role by the role that owns them, in the shape
web-app-prometheus uses, because one round-scoped file would drop another
application's zones on the next deploy. The rate is concurrent_requests
per second rather than an eighth limit key: a client allowed N in flight,
each bounded by timeout_seconds, cannot sustain more unless every call
returns instantly. A refusal is 429, not nginx's 503, because a client
that should back off must not read it as an outage. stream_seconds
finally binds something - the adapter holds no stream, the proxy does.
The shared location snippet's three timeouts become parameters defaulting
to the 900s they hard-coded, so no existing vhost changes.
Found by the disabled-state assertion 7d8e7e87ff added, which had never
run: its helper was never staged, so the two defects hid each other.
Home Assistant pinned the MCP service account to its read-only group when it created the account and never again, so an account promoted to administrator by hand kept every mutating tool that scope denies while the deploy stayed green, because nothing looked at the existing case. The group is now asserted on every run and a refusal aborts instead of passing.
Nextcloud minted an app password under a fixed managed name and never revoked the previous one, so every run that found the stored password rejected added another valid token while the old ones kept full account access, accumulating across deploys. The mint is now preceded by a listing and the revocation of every token carrying that name. Both new tasks mask their output, because the listing names the account and its token ids even though the JSON strips the secrets.
Prometheus ran with no query timeout, so its two-minute default applied: a query the adapter abandoned after fifteen seconds kept the server busy for the rest of it. The flag is now derived from the adapter's own limit and doubled, which gives the server room past the adapter's deadline while keeping the two from drifting apart.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The context_agent ExApp calls back into the same php-fpm pool it is being
proxied from: its UserAuthMiddleware resolves the bearer against
/ocs/v2.php/cloud/user while a child is already parked forwarding the MCP
request out to the ExApp. Every in-flight MCP request therefore occupies TWO
children. Below twice mcp.limits.concurrent_requests the two legs deadlock, and
because the ExApp's callback carries no timeout the pool never recovers -- which
is why the Context Agent sat unhealthy rather than merely slow.
services.nextcloud.performance.php.fpm now sizes the pool explicitly:
max_children 16, start_servers 4, min/max spare 4/8. request_terminate_timeout
is 900s and must not drop below the proxy's fastcgi_read_timeout; lower kills a
child while nginx is still waiting on it, equal means only work nginx has
already abandoned gets reclaimed.
The readiness path was spending the same scarce workers. /healthz/ready had no
location of its own, so it fell through to `location /`, was rewritten to
/index.php/healthz/ready and booted a full Nextcloud request on a php-fpm child
once per probe interval against a pool of five. It is an exact-match location
returning 200 from nginx now, with access_log off, and never reaches PHP.
The pool file is rendered through render_replicated_templates.yml so swarm peers
receive it too, and mounted from meta/volumes.yml at a path that sorts after
zz-docker.conf, which reopens the [www] section last -- mount it earlier and the
image's own values win while the deploy looks correct.
The context_agent healthcheck moves from a hand-written probe in compose.yml.j2
to the shared container_healthcheck lookup, declared as flavor curl on path
heartbeat with start_period 20m. The ExApp installs dependencies on first boot
and legitimately outlives a 60s start period, which is the other half of why it
reported unhealthy.
Not verified: none of this has deployed. max_children 16 is derived from twice
concurrent_requests rather than measured against real memory use, and the
mount-ordering requirement is read from the image layout rather than confirmed
against a merged pool config in a running container -- if it sorts wrong the
whole change is inert while appearing correct. request_terminate_timeout 900s
also means a genuinely stuck request now holds a child for fifteen minutes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The api_key write was guarded by `(nextcloud_openai_litellm_api_key |
default('')) | length > 0`, so an empty credential SKIPPED the task. Nextcloud
kept whatever integration_openai api_key it already held, or none, and the
deploy reported success -- exactly the silently unauthenticated AI surface
requirement 036 forbids.
The three tasks now sit in one block gated on nextcloud_openai_use_litellm,
which also satisfies the repeated-when lint, and the shared consumer contract
runs first. That contract is what turns the empty-key case from a skip into a
failure, and it also asserts the base URL stays inside the deployment, which
this role never checked.
The key lookup drops its '' fourth argument and is strict again. Ordering
matters: the contract fails first with a readable message, so the strict lookup
is never the thing that raises.
Nextcloud was the last of the nine confirmed roles without the shared contract.
The occ call keeps the space form, `--value '{{ ... }}'`, deliberately. A
foreign commit on fork/feature/svc-net-tor (bbf4ca09cc) converts that to
`--value={{ ... | quote }}` because Symfony ArgvInput reads a token beginning
with a dash as an option. Adopting it here was considered and rejected on
measurement: shlex.quote leaves an sk-prefixed value completely unquoted, so
both forms emit an identical argv token and the change is inert at this call
site; git merge-file yields one conflict hunk either way, because the two sides
differ in the variable name rather than the quoting; and ansible's quote filter
maps None to '', which would turn a null credential into `--value=` -- an empty
write that exits 0 and matches changed_when, defeating the very guarantee this
commit adds. The equals form is still worth having where values are minted
externally, notably integration_openproject_provision.yml and
integration_suitecrm_provision.yml, and belongs in its own change covering
every call site rather than the one where the failure cannot fire.
Not verified: this role has not deployed since the change. On the branch where
Open WebUI is present, nextcloud_openai_use_litellm is false and no contract
runs at all; traffic stays in-cluster because Open WebUI is itself
gateway-backed, but criterion 4 is not asserted on that path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
matomo, moodle and nextcloud keep the database password inside a config
file on their data volume. When the deploy rotates credentials, the
container gets the new password while its persisted config still carries
the old one, and the app fails to authenticate against its own database.
Patch the persisted file before the health gate reads it. The value goes
through the environment rather than the command line, and sed_escape
guards the delimiter, backslash and ampersand that would otherwise corrupt
the file the replacement is meant to repair.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A handler name was doing two jobs at once: the line an operator reads in the
play output, and the identifier a `notify:` binds to. The two pull in opposite
directions -- the log wants "restart the SSH service, with diagnostics", the
binding wants a token that survives quoting, YAML flow style and a shell
round-trip -- and a name that changes for readability silently changes which
handler runs.
Every handler that a notify reached by name now carries a `listen:` with that
exact wording, so the binding is explicit before anything is renamed. The
topics themselves become identifiers: `compose up` reads `compose-up`,
`Restart NFS ganesha` reads `restart-nfs-ganesha`, 22 of them across both
sides of every notify. Where the topic is assembled at runtime, both branches
moved together: the conditional in sys-service carries two literals, and the
LDAP renderer interpolates `import-{{ folder }}-ldif-files`, which is what
makes those handlers reachable at all.
Freed from the binding, the names take the emoji prefix every other task line
already has.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The readiness probe 5bd323999 added for the swarm path dialled the role's public
domain on port 80. wait_for opens a plain TCP socket from the controller, and
once 1d9d355bc gave the domain roles onion variant coverage that target became
a .onion authority, which needs Tor's SOCKS resolver and cannot be reached with
a socket. Variant 6 of run 31823845576 therefore spent the full budget on it -
"Timeout when waiting for lab.git.<onion>:80", elapsed: 600 - while the
preceding resolve_host_cid had already converged, so GitLab itself was up the
whole time.
The repository had already settled this. DOCKER_REACH_HOST
(group_vars/all/00_general.yml:47) is 127.0.0.1, swapped for the Docker bridge
gateway when the controller runs in a container, and every one of the other
eleven wait_for probes under roles/ uses it. The nearest neighbour is the same
task shape - same resolve preamble, same retries and delay - reaching its
partner as `host: DOCKER_REACH_HOST` with `ports.local.http`
(integration_openproject_provision.yml:12-17). This one was written from that
template with the host and port substituted, and the substitution is the defect.
Restoring the idiom keeps the readiness gate 5bd323999 wanted instead of
dropping it, so retries stays at 12.
The lint pins the shape, because the divergence is invisible in review: both
forms read as a plausible readiness probe, and the wrong one only fails where
the domain happens to be unroutable. It parses each wait_for block under roles/,
locates the host key by indentation, and requires DOCKER_REACH_HOST unless the
line carries `# nocheck: wait-for-reach-host` naming what the probe proves - the
opt-out a probe needs when it is verifying the ingress rather than a backend.
Checked against the original line, where it reports
integration_gitlab_provision.yml:14, and against the eleven correct probes,
which it accepts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Symfony ArgvInput reads a token beginning with a dash as an option, so passing the secret as a separate token after --value handed occ two tokens and occ reported the option as missing its value. Doorkeeper mints base64url secrets, so roughly one run in 43 produced one starting with a dash and the OpenProject provisioning died with: The "--value" option requires a value.
The equals form is unambiguous: Symfony splits at the first equals sign and takes everything after it as the value. The quote filter replaces the hand-written single quotes and stays shell-safe. Note that shlex leaves a leading-dash value unquoted, so the equals form is what carries the fix, not the quoting.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The integration_discourse settings page rewrites what the user types, and the
rewrite is unconditional. src/components/PersonalSettings.vue:217-224 upstream:
if (this.state.url !== '' && !this.state.url.startsWith('https://')) {
if (this.state.url.startsWith('http://')) {
this.state.url = this.state.url.replace('http://', 'https://')
An onion service carries no certificate and is served over plain http through
Tor, so on such a node the address is rewritten to an endpoint that does not
exist. Run 31658270172, job 94319317649 shows all three steps: the trace's
action log records the spec typing `http://forum.<onion>/`, the PUT to
/apps/integration_discourse/config carries `{"values":{"url":"https://forum.
<onion>/"}}`, and the connect navigation to `https://forum.<onion>//user-api-
key/new` fails with network status -1, which Playwright reports as
`Received string: "chrome-error://chromewebdata/"`.
The app ships compiled JS, so the fix has to reach the bundle rather than the
source. It is measured against the file the appstore actually installs --
nextcloud-releases/integration_discourse v3.3.0, whose
integration_discourse-personalSettings.mjs is the exact filename the trace
loaded from the container; the nextcloud/integration_discourse releases stop at
v2.0.1 in 2023 and still ship .js, which is why the earlier reading of that repo
was misleading.
Verified rather than argued:
- the anchor `startsWith("https://")&&(` occurs exactly once in that file
- after the patch the guard reads
`…startsWith("https://")&&!this.state.url.includes(".onion")&&(…`
and `node --check` accepts the bundle as valid ESM
- the anchor no longer exists afterwards, so a second application is a no-op:
idempotency is structural, not a marker check
- the task was run through a real ansible-playbook against a real copy of the
bundle with only the exec prefixes stubbed: first run changed=1 and patched,
second run changed=0, so changed_when reports honestly
- busybox sed (alpine:3, "This is not GNU sed version 4.0") produces the same
result as GNU sed, which matters because the image is 34-fpm-alpine
`occ app:getpath` locates the app instead of hardcoding custom_apps, which
appears nowhere in the repo and which the appstore may move. The `grep -q … ||
exit 0` makes the task both idempotent and safe when the file is absent.
Clearnet is untouched: the added condition is false for every non-onion URL, so
the upgrade still happens where TLS is available.
Not verified: what `app:getpath` returns in this installation. If the path is
wrong the task no-ops silently -- the same shape as the WordPress healthcheck,
where msmtp fails and a passing curl hides it. A `test -f` with a loud failure
would close that, and the layer behind the redirect has never been reached by a
green run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The stdin: value concatenating doorkeeper_allow_onion_redirect.rb with
ensure_nextcloud_storage.rb ran past the line budget; it now wraps before the
closing braces.
Whitespace only. yaml.safe_load of the file before and after the change returns
the same document, so the Ruby that reaches the container is byte-identical.
Not authored as part of the surrounding session work; committed separately so it
does not ride along inside an unrelated bundle.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
files/ now mirrors the role's own structure: php, ruby and shell, each split
into addons for the per-integration scripts that tasks/addons/ drives and utils
for what more than one caller shares. set_encrypted_appconfig.php serves gitlab,
mattermost and zammad; doorkeeper_allow_onion_redirect.rb serves zammad and
openproject; init-gate.sh belongs to the role itself. The partner directories
survive because two scripts are both named provision_oauth_app.rb.
NEXTCLOUD_INIT_GATE_FILE had to split. It was the source path under files/, the
name Dockerfile.j2 COPYs into /usr/local/bin, and the ENTRYPOINT, all at once;
moving the script would have produced /usr/local/bin/shell/utils/init-gate.sh.
The basename keeps that name and NEXTCLOUD_INIT_GATE_SRC carries the path.
The same commit closes two of the three faces of one defect. resolve_enabled
reports TLS off for an .onion domain, correctly -- the vhost is plaintext and
Tor carries the transport crypto -- so every redirect URI built from
lookup('tls', ...) is http, and Doorkeeper rejects it. Zammad raises out of
app.save! and fails the task at rc 1; OpenProject discards result.errors, keeps
the unsaved model whose plaintext_secret already exists, reports
oauth_application_created:true, and only dies three tasks later on a null
client_id. The prelude relaxes force_ssl_in_redirect_uri for .onion hosts and
delegates every other host to the previous setting, so clearnet is unchanged at
both partners; the OpenProject script now raises with the real validation
errors.
The third face is PeerTube, whose Keycloak callback is registered http and
requested https, and it stays open: fixing it means changing either PeerTube's
scheme, which is its federation identity, or the registration side.
Evidence: run 31658270172, jobs 94319317307 ("Redirect URI must be an HTTPS/SSL
URI.") and 94319317278 (changed:true, then length on NoneType).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four roles created their MCP service account and never touched its password
again, while the same play authenticates as that account moments later. A
deployment whose declared password moved on therefore failed at the login,
not at the point the drift was introduced.
gitea now runs 'admin user change-password' before the token dance. The
--must-change-password=false flag is mandatory rather than cosmetic: at the
pinned v1.27.1 the flag defaults to true, so omitting it locks the account out
of the very API call that follows.
nextcloud resets through 'occ user:resetpassword --password-from-env' before
minting the app token that verifies the password.
snipe-it compared nothing at all - its docstring promised to pin the password
while the assignment sat inside the create branch. It now rehashes only when
Hash::check says the stored hash no longer matches, so the changed flag stays
honest.
home assistant swallowed a failed credential creation for accounts that already
existed and then logged in with them. It now calls admin_change_password and
falls back to creation only when the reply is credentials_not_found.
jellyfin and zammad already converged; wordpress, gitlab and moodle never
authenticate with this password and are left alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
meta/schema.yml became meta/secrets.yml, but its content still landed at
applications.<app>.credentials — the one meta file whose payload did not sit
under a root named after the file. Every other topic obeys that rule:
meta/volumes.yml goes to volumes, meta/rbac.yml to rbac. Credentials now go to
secrets.credentials.
The path exists exactly once, in utils/manager/credential_key.py, and
OVERRIDE_SECTION is derived from its two segments rather than spelled beside
them, so the dotted form used by --set and the nested form used by the
inventory cannot drift. Six consumers read it: the loader, the generator, the
CLI override parser, the two emit helpers, and the four lookup validators.
--set now accepts exactly one key shape,
applications.<app>.secrets.credentials.<key>. The three short forms are
rejected instead of guessed at: the same short key names different credentials
on different apps. That also retired primary_app_id, which only existed to
disambiguate them.
Three defects surfaced while wiring this up. _apply_one_role_schema still wrote
to the old root, so an app's own credentials would have moved while its
providers' stayed behind. web-app-pgadmin reads through
lookup('applications', ...).credentials.<key>, an access path no regex over
lookup('config', ...) could see, and would have deployed with an empty postgres
password. And _META_NON_GENERIC_TOPICS still listed 'schema' — a bare topic
name, invisible to any path search — so the generic file-root loop assigned the
raw schema under secrets and the follow-up write landed inside the cached
document, truncating keycloak's four credentials to one for every later
reader.
tests/integration/ansible/lookups/config/test_credentials_roundtrip.py pins both
directions of the seam and the cache invariant that last defect broke.
Existing inventories still carry the pre-rename root; migrating them is
deliberately out of scope here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nine roles reached their MCP surface as the deployment administrator or with
an account password kept beside the account in credentials. A Moodle site
admin bypasses every capability check, a Nextcloud app password is not
scopable, a Gitea token reads every private repository, and a GitLab token on
root reads every private project. Each role now declares its own principal in
meta/users.yml and provisions it, so the exposed surface is what the service
role permits. jellyfin, zammad and snipe-it kept their account but took its
password from a credential named token; they read the registry too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
75 commits: the socialhome role reimplemented against the project it actually
deploys, native-login specs for funkwhale and shopware, a window.open IdP
handoff and a settings-link fallback in the shared persona helpers, a swarm
converge budget that gives up on a terminal task, and a long tail of role
fixes.
Eleven conflicts. The shared persona helpers (keycloak.js, logout.js) take the
incoming structure — they gained real behaviour this branch never had — with
this branch's resolveTimeout convention re-applied over it and
confirmKeycloakLogoutIfPrompted re-attached at every return point; without it
the Keycloak SSO session outlives the logout. playwright.config.js keeps this
branch's version: both sides set actionTimeout, but ours routes through an env
var and 15s is too short behind an onion. socialhome's spec takes the incoming
content — 2dbd6eb10 found the role described the wrong upstream project
entirely — and keeps gotoOnion and expectHstsWhenTls, since demanding HSTS over
a plaintext onion is a false failure. Both READMEs were regenerated rather than
merged by hand.
Three collisions arrived with no conflict marker at all, because each side
changed a different place in a different file:
- group_vars/all/18_swarm.yml declared swarm_stack_converge_retries twice.
Same value, so nothing breaks today; editing the documented one would have
silently changed nothing.
- svc-net-tor and web-app-socialhome both claimed 192.168.74.0/24. Socialhome,
the newer role, moves to 192.168.75.0/24.
- web-app-socialhome/meta/schema.yml arrived as a new file, so ef5ea7cb5's
rename never saw it. Its credential would not have resolved and the role
would have deployed with an unusable administrator account.
The incoming specs also needed this branch's conventions: raw timeouts wrapped,
four page.goto calls routed through gotoOnion, a request.get given an
onion-aware timeout, and 21 roles marked playwright-service-flag over the tor
block this branch added to them.
make quality green on the merge result.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
integration_mastodon's connect returned 401 with
'cURL error 60: unable to get local issuer certificate ... for
https://microblog.infinito.example/api/v1/apps', byte-identical across
attempt and retry. The CA plumbing the repo already injects - the CA
mount, SSL_CERT_FILE, CURL_CA_BUNDLE, php.ini curl.cainfo - cannot reach
that call: OC\Http\Client\Client pins Guzzle's VERIFY to the
certificate manager's own bundle, which defaults to the shipped
resources/config/ca-bundle.crt.
occ security:certificates:import is the only lever that reaches it.
Verified in nextcloud:34: the command is registered as
'security:certificates:import' taking a required path, and
addCertificate() calls createCertificateBundle(), which writes the merged
rootcerts.crt that getAbsoluteBundlePath() then returns. Public TLS keeps
working because the import appends to the shipped bundle. Every
integration_* app talking to an internal host over HTTPS was broken the
same way, not just Mastodon.
Registers the GitLab OAuth application with the scopes the client
actually requests: integration_gitlab/src/utils.js:26 sends
'read_user read_api read_repository', and Doorkeeper's ScopeChecker does
plain string-set containment, so 'api' does not subsume 'read_api' and
consent would have been refused. Also least privilege.
The gitlab spec sampled the URL the page settled on after GitLab's 302,
but GitLab keeps the return path in session[:user_return_to], so that URL
carries no query and the assertion looked for evidence GitLab structurally
does not provide. It now captures the authorize request itself and
additionally asserts redirect_uri points back at this Nextcloud and that
the partner answered below 400.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four readiness probes waited on lookup('domain', <app>) port 80 -- a
publication identity used as a machine address. With svc-net-tor deployed
that name is onion-only, because exclusive: true replaces the clearnet
domain rather than adding to it (utils/cache/domains.py).
Nextcloud 5 then burned its full 600s on zammad.helpdesk.<onion>:80. The task
carries no delegate_to but inherits apply: delegate_to: NEXTCLOUD_HOST_NODE
from 02_manager_ops.yml:46-50, which propagates through nested includes, so it
ran on a worker. svc-net-tor is placement: manager, so that worker had no
.onion split-DNS, no TransPort redirect and no :80 listener. wait_for swallows
socket.gaierror in a bare except and retries, so an unresolvable name is
indistinguishable from a slow service -- it just burns the timeout.
Point the probes at DOCKER_REACH_HOST and the service's own
ports.local.http instead. No name resolution, so the onion replacement cannot
reach it; node-local, so the inherited delegation stops mattering; and it hits
the service's own listener rather than the shared front proxy, which is what
the probe was supposed to measure in the first place.
Verified on the failing run's artifacts that this is node-independent in
swarm: zammad_railsserver.1 ran only on mgr-01, yet dockerd listens on *:8077
on all three nodes -- the ingress mesh. Only svc-prx-openresty and svc-net-tor
publish with mode: host; every other role, these four included, goes through
ingress.
Same swap for openproject, suitecrm and wordpress->discourse, which had the
identical construct.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three roles reach their MCP work from a manager-ops file rather than
from the role entry point, and the rename missed them, leaving them
including task files that no longer exist.
Ten provider roles served an MCP surface that other roles consume, but
were never declared as shared services, so the registry did not know
them. Declaring them pulls in the recursion contract every registry
service owes: seven roles move their entry tasks into tasks/00_core.yml
behind the run_once flag, so the service loader cannot include them a
second time.
pretix already had a run_once block, but set the flag last; the flag now
comes first, before the body it protects.
Every MCP role keeps its task tree under tasks/utils/ without numeric
prefixes, so a caller can reach any of them through include_role
tasks_from. utils/roles/mapping.py holds that path as the single source.
Ansible resolves a relative include against the including file's own
directory when that file was reached by path, but against tasks/ when it
was reached through tasks_from. The same file therefore loaded from its
own role and failed from a caller using tasks_from. Every nested include
now names {{ role_path }}/tasks/utils/..., which holds under both bases,
and a lint walks each tasks_from entrypoint transitively to keep it so.
The exec-gating lint learns the same form, otherwise it loses the include
chain and reports inherited IS_STACK_HOST gates as missing.
The expression resolving a provider's stored token appeared eighteen
times. plugins/lookup/mcp_credential.py replaces it, and mcp_probe_owner
disappears from eight roles: it was only ever a step towards the bearer
and no probe read it.
The Hermes and OpenClaw probes stop aborting the play while it is still
running. Roles deploy in dependency order, so a client probing during its
own role can run before a provider exists at all, and failing there
prevents that provider from ever deploying. The verdict moves to the
reconciliation pass, gated on MCP_RECONCILE_STRICT, which the reconciler
sets and group_vars declares so the readers need no default.
The sys-svc-mcp-reconcile role is added here; wiring it into a stage is a
separate change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The gate probed the partner with container inspect on a bare name, which only resolves under compose co-location. In swarm GitLab sits on another node, the probe fails, and failed_when: false turns that into a silent skip: the deploy reports success while client_id/client_secret are never written, so the per-user Connect button leads nowhere. Nothing else writes those two values.
Mirrors integration_zammad: resolve node and container id first, wait for the partner endpoint, then run both execs against the resolved ids. Gate on IS_STACK_HOST like every other addon. The include had to become relative, or the stack-host lint cannot see the gate through a templated path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The eight roles whose MCP server is built into the application now end their MCP
step with the shared contract probe. It builds URL and Authorization header from
the role's own declaration, calls initialize and tools/list, invokes one
deterministic read tool per role, and compares the served surface against the
declared allowlist. It cannot be a Playwright test: the check needs the real
provider credential and a Playwright run would capture it in its trace.
Eleven sites wrote the MCP token into the store under the administrator's
identity while their metadata declared an owner of their own. The schema lint
forbids administrator as a credential owner, so declaration and storage
disagreed, and the probe would have looked for the bearer under the declared
owner and found nothing. They now read the declared owner.
Takes the revert of the rendered system config. The MCP work had added the
proxy service name and its swarm DNS name as trusted domains to
system.config.php.j2, which the revert deletes; both entries move back to
vars/system.yml as trusted_domains 1 and 2, where they lived before aeefb1407.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three occ calls take an exclusive lock on config.php and gave up on the first
failure. In run 30830377785 the config:system:set batch hit
In Config.php line 306:
Could not acquire an exclusive lock on the config file
/var/www/html/config/config.php
after writing seven of nine values, and failed the swarm job. The same task ran
13 times across that run's swarm jobs and passed 12 times, including a second
fedora job on the same commit and topology, so the condition is transient rather
than a defect in the call.
maintenance:repair and app:update in 03_upgrade.yml already carry
retries/delay/until for this reason. This aligns the three that did not:
occ upgrade, maintenance:mode --off, and the config:system:set batch. The batch
is safe to repeat because config:system:set is idempotent and its `set -euo
pipefail` aborts at the failing call rather than continuing half-applied.
This does not mask the failure. Verified with a probe playbook of the same shape
(failed_when plus an until that never becomes true): the play ends failed=1 and
the downstream task does not run, so an exhausted retry still fails the deploy -
two attempts later than before.
The mechanism behind the lock error stays unproven: Config.php calls
flock(LOCK_EX) without LOCK_NB, which blocks rather than returning false, and no
errno appears in any of the 596 rescue-diagnostic files. A retry helps if the
condition is transient and does not if the NFSv4 lock layer returns a hard
error. That is why this is a mitigation of an ~8% flake, not a fix for a proven
cause.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This reverts commit 41063206a.
It broke every swarm nextcloud deploy. system.config.php.j2 opened with
<?php
{%- macro php_str(value) -%}
and the leading dash strips the newline after `<?php`, so under Ansible's
trim_blocks the file renders as `<?phpreturn [`. Run 30848540165:
ParseError: syntax error, unexpected token "=>", expecting "]"
in /var/www/infinito/system.config.php:2
#0 /var/www/html/lib/private/Config.php(229): include()
Config->readData() includes the file, so occ upgrade, every later occ call and
every php-fpm request died on an uncatchable compile error. The template had
been verified by rendering it with plain Jinja defaults, which do not match the
ones Ansible uses - the check confirmed the escaping and missed the line above.
A one-character fix (`{% macro`) is correct and was verified end to end, but the
commit is reverted rather than repaired, because its premise did not survive
review:
- The failure it targeted is not deterministic. The same task ran 13 times
across the swarm jobs of run 30830377785 and passed 12 times, including a
second fedora job on the same commit and topology. It is an ~8% flake.
- It does not remove the mechanism. 03_upgrade.yml:8 runs
`occ maintenance:repair --include-expensive` unconditionally, and upstream
Repair.php sets the maintenance system value twice with no guard - two
exclusive locks on the same NFS config.php per deploy, unretried.
- The benefit is largely illusory. Config.php:279 var_exports the whole merged
cache back into config.php, so the nine values land in the locked file anyway
on the first maintenance:repair, with a larger payload.
- It traded away occ's exit-code confirmation that Nextcloud accepted the
values, and moved them from the shared NFS volume to a node-local bind.
The remaining exposure is the unretried occ calls in 03_upgrade.yml; a bounded
retry there is the smaller and better-targeted change and is left for its own
commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nine sequential `occ config:system:set` calls each open config.php exclusively.
In swarm that file lives on the NFS share behind every replica, and the seventh
call could not take the lock:
In Config.php line 306:
Could not acquire an exclusive lock on the config file
/var/www/html/config/config.php
stdout shows six values written before the abort, so the run left a partial
configuration behind. That failed the swarm web-app-nextcloud 5 job in run
30830377785 while every PLAY RECAP on the other nodes reported failed=0.
The same nine values now come from templates/config/system.config.php.j2, which
rides the additional-config channel the role already uses for redis, ldap and
the rest: rendered to NEXTCLOUD_HOST_CONF_ADD_PATH, mounted read-only, merged by
zzz-infinito.config.php after config.php so its values win. That path never
takes an exclusive lock, and it runs before the stack starts, since all nine
values are statically derived and none needs a live container.
Two behaviours are deliberately not carried over. The --type flags become native
PHP types, which is stricter than the string coercion occ applied. And nothing
confirms the file was accepted any more, where occ's exit code did; a read-only
`config:system:get` would restore that assurance at the cost of an occ call, and
is left out until someone wants it.
php_str() escapes backslashes and single quotes, matching the idiom already used
for the redis password in memcache.config.php.j2. Without it a quote in any
value would be a PHP syntax error, and include_once would drop every additional
config file, not just this one.
vars/system.yml and 02_manager_ops/05_system_config.yml are gone; the later task
files shift down one number so test_task_file_numbering stays satisfied.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both waits are needed and check different things: wait_converged inside
resolve_host_cid asks swarm whether the task reached running, the wait_for asks
whether the service answers. Running the endpoint wait first meant a partner
that is not deployed at all paid both budgets in sequence - up to 600s of HTTP
timeout followed by 120x5s of convergence - before the play gave up.
Ordering the cheap service-level check first halves that without giving up
either guarantee. The resolver needs no HTTP: it runs container service ps,
container ps and set_fact, and its two outputs are first consumed by the rails
call further down, i.e. after both waits.
Task order only; nothing added, removed or reparameterised. 8b2cbcb04 applies
the same ordering to the Zammad twin, where the wait was newly introduced.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
resolve_host_cid's convergence wait is swarm-only (wait_converged.yml:9), so in
compose mode nothing made sure Zammad answers before the rails provisioning ran
- the only budget there was the provisioning task's own 12x10s against a rails
server whose cold start can exceed it. integration_openproject_provision.yml
already guards its partner this way; this restores the symmetry.
The wait sits after the resolver rather than before it. Both waits are needed
and check different things - wait_converged asks swarm whether the task reached
running, the wait_for asks whether the service answers - but ordering the cheap
service-level check first means a partner that is not deployed at all fails
after one budget instead of two. The resolver itself needs no HTTP: it only runs
container service ps / container ps and set_fact.
Not executed: that lookup('domain', 'web-app-zammad') resolves to a name
answering on port 80 is taken from the openproject twin, not measured.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The marker claimed to suppress container-exec-resolver, but that rule does not
fire on this task. Its extractor reads the first argument after `container exec`
and finds `-i`, so it returns no target; the container id is four lines further
down. The position was wrong too: the rule accepts a marker on the same or the
immediately preceding non-empty line, and three lines separated the two.
That leaves a suppression marker carrying a rationale, which is the shape the
comment policy exists to prevent. The reason it stated - that container_address
would expand to a manager-only subshell - is already in 04b8cfead's message.
tests/lint/ansible/shell/container/ stays green without it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
02_manager_ops.yml includes the plugin chain with `apply: delegate_to:
NEXTCLOUD_HOST_NODE`, so the addon hook runs wherever that node happens to be.
On a swarm worker the partner probe could not answer: it used
NEXTCLOUD_ZAMMAD_CONTAINER_ADDRESS, which expands to a
`resolve-container-id zammad railsserver` subshell, and that helper exits 64 off
a manager. The substitution came back empty, `container inspect ""` failed,
`failed_when: false` swallowed it, and the gate
`(nextcloud_zammad_partner_running.stdout | trim) == 'true'` was false.
The OAuth provisioning was therefore skipped, client_id and client_secret stayed
empty, and integration_zammad's personal view fell back to its access-token UI -
so the 'Connect to Zammad' control the spec waits for never existed. That is
what failed web-app-nextcloud 5 in run 30781723102.
The probe and its clause are gone; provisioning now resolves the railsserver's
hosting node and local container id through the standard
sys-svc-compose/utils/swarm/resolve_host_cid.yml and delegates the rails call
there. resolve_host_cid pins its own delegate_to on STACK_HOST and
<prefix>_HOST_NODE, so it is unaffected by the inherited one, and the trailing
occ tasks keep no delegate_to and still run on the Nextcloud node.
Losing the probe also loses its quiet skip. The addon is still gated on
services.zammad.enabled, i.e. 'web-app-zammad' in group_names
(meta/addons/integration_zammad.yml:2), so the exposure is a targeted deploy
where the partner is in the inventory but excluded from the run: wait_converged
then spends 120x5s before failing instead of skipping.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The merged branch renumbered the Nextcloud tasks to 01/02, leaving the
MCP tasks stranded at 09 with a gap the numbering lint rejects; rename
them to 03 and follow every reference.
The Mattermost select statements only formed a prefix group once the
incoming branch's own select landed beside them, so they move into
sql/select/, and the two user-scoped ones again into sql/select/user/.
Pin the object kind on the kata isolation inspect: this repository names
a role's overlay network after the same entity as its main container, so
an untyped inspect can resolve the network and report a missing State key
instead of the missing container. Suppress the address-lookup rule there
because both callers gate the assertion on compose, where the container
name is the address.
Give the sandbox-runtime test the data-root variable the incoming
daemon.json template reads. Both sides were green alone: their template
gained the variable, this test renders it under StrictUndefined.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
