files/ now mirrors the role's own structure: php, ruby and shell, each split
into addons for the per-integration scripts that tasks/addons/ drives and utils
for what more than one caller shares. set_encrypted_appconfig.php serves gitlab,
mattermost and zammad; doorkeeper_allow_onion_redirect.rb serves zammad and
openproject; init-gate.sh belongs to the role itself. The partner directories
survive because two scripts are both named provision_oauth_app.rb.
NEXTCLOUD_INIT_GATE_FILE had to split. It was the source path under files/, the
name Dockerfile.j2 COPYs into /usr/local/bin, and the ENTRYPOINT, all at once;
moving the script would have produced /usr/local/bin/shell/utils/init-gate.sh.
The basename keeps that name and NEXTCLOUD_INIT_GATE_SRC carries the path.
The same commit closes two of the three faces of one defect. resolve_enabled
reports TLS off for an .onion domain, correctly -- the vhost is plaintext and
Tor carries the transport crypto -- so every redirect URI built from
lookup('tls', ...) is http, and Doorkeeper rejects it. Zammad raises out of
app.save! and fails the task at rc 1; OpenProject discards result.errors, keeps
the unsaved model whose plaintext_secret already exists, reports
oauth_application_created:true, and only dies three tasks later on a null
client_id. The prelude relaxes force_ssl_in_redirect_uri for .onion hosts and
delegates every other host to the previous setting, so clearnet is unchanged at
both partners; the OpenProject script now raises with the real validation
errors.
The third face is PeerTube, whose Keycloak callback is registered http and
requested https, and it stays open: fixing it means changing either PeerTube's
scheme, which is its federation identity, or the registration side.
Evidence: run 31658270172, jobs 94319317307 ("Redirect URI must be an HTTPS/SSL
URI.") and 94319317278 (changed:true, then length on NoneType).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four readiness probes waited on lookup('domain', <app>) port 80 -- a
publication identity used as a machine address. With svc-net-tor deployed
that name is onion-only, because exclusive: true replaces the clearnet
domain rather than adding to it (utils/cache/domains.py).
Nextcloud 5 then burned its full 600s on zammad.helpdesk.<onion>:80. The task
carries no delegate_to but inherits apply: delegate_to: NEXTCLOUD_HOST_NODE
from 02_manager_ops.yml:46-50, which propagates through nested includes, so it
ran on a worker. svc-net-tor is placement: manager, so that worker had no
.onion split-DNS, no TransPort redirect and no :80 listener. wait_for swallows
socket.gaierror in a bare except and retries, so an unresolvable name is
indistinguishable from a slow service -- it just burns the timeout.
Point the probes at DOCKER_REACH_HOST and the service's own
ports.local.http instead. No name resolution, so the onion replacement cannot
reach it; node-local, so the inherited delegation stops mattering; and it hits
the service's own listener rather than the shared front proxy, which is what
the probe was supposed to measure in the first place.
Verified on the failing run's artifacts that this is node-independent in
swarm: zammad_railsserver.1 ran only on mgr-01, yet dockerd listens on *:8077
on all three nodes -- the ingress mesh. Only svc-prx-openresty and svc-net-tor
publish with mode: host; every other role, these four included, goes through
ingress.
Same swap for openproject, suitecrm and wordpress->discourse, which had the
identical construct.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
resolve_host_cid's convergence wait is swarm-only (wait_converged.yml:9), so in
compose mode nothing made sure Zammad answers before the rails provisioning ran
- the only budget there was the provisioning task's own 12x10s against a rails
server whose cold start can exceed it. integration_openproject_provision.yml
already guards its partner this way; this restores the symmetry.
The wait sits after the resolver rather than before it. Both waits are needed
and check different things - wait_converged asks swarm whether the task reached
running, the wait_for asks whether the service answers - but ordering the cheap
service-level check first means a partner that is not deployed at all fails
after one budget instead of two. The resolver itself needs no HTTP: it only runs
container service ps / container ps and set_fact.
Not executed: that lookup('domain', 'web-app-zammad') resolves to a name
answering on port 80 is taken from the openproject twin, not measured.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The marker claimed to suppress container-exec-resolver, but that rule does not
fire on this task. Its extractor reads the first argument after `container exec`
and finds `-i`, so it returns no target; the container id is four lines further
down. The position was wrong too: the rule accepts a marker on the same or the
immediately preceding non-empty line, and three lines separated the two.
That leaves a suppression marker carrying a rationale, which is the shape the
comment policy exists to prevent. The reason it stated - that container_address
would expand to a manager-only subshell - is already in 04b8cfead's message.
tests/lint/ansible/shell/container/ stays green without it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
02_manager_ops.yml includes the plugin chain with `apply: delegate_to:
NEXTCLOUD_HOST_NODE`, so the addon hook runs wherever that node happens to be.
On a swarm worker the partner probe could not answer: it used
NEXTCLOUD_ZAMMAD_CONTAINER_ADDRESS, which expands to a
`resolve-container-id zammad railsserver` subshell, and that helper exits 64 off
a manager. The substitution came back empty, `container inspect ""` failed,
`failed_when: false` swallowed it, and the gate
`(nextcloud_zammad_partner_running.stdout | trim) == 'true'` was false.
The OAuth provisioning was therefore skipped, client_id and client_secret stayed
empty, and integration_zammad's personal view fell back to its access-token UI -
so the 'Connect to Zammad' control the spec waits for never existed. That is
what failed web-app-nextcloud 5 in run 30781723102.
The probe and its clause are gone; provisioning now resolves the railsserver's
hosting node and local container id through the standard
sys-svc-compose/utils/swarm/resolve_host_cid.yml and delegates the rails call
there. resolve_host_cid pins its own delegate_to on STACK_HOST and
<prefix>_HOST_NODE, so it is unaffected by the inherited one, and the trailing
occ tasks keep no delegate_to and still run on the Nextcloud node.
Losing the probe also loses its quiet skip. The addon is still gated on
services.zammad.enabled, i.e. 'web-app-zammad' in group_names
(meta/addons/integration_zammad.yml:2), so the exposure is a targeted deploy
where the partner is in the inventory but excluded from the run: wait_converged
then spends 120x5s before failing instead of skipping.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
NEXTCLOUD_DOCKER_EXEC resolved the container id through a subshell evaluated on the manager. When the manager holds no running replica, resolve_container_id.sh falls back to a cluster-wide lookup with no node filter and returns the id of a task on another node, which the local daemon does not know: 'No such container', sixty times, then the wait in 02_config.yml fails the deploy.
08_manager_ops.yml now resolves the hosting node and its node-local container id once via resolve_host_cid.yml and delegates every include there with apply:. Redefining NEXTCLOUD_DOCKER_EXEC in terms of NEXTCLOUD_LOCAL_CID lets all seventeen consumers inherit the fix without touching a task line. It re-resolves after the handler flush and after the upgrade, the two points where a rolling restart can move the task. Compose degenerates to delegate-to-self with the previous container name.
Four call sites built their exec inline because they need -u root or -e flags, and their subshell would have run on the worker where resolve-container-id exits 64 for want of a manager. Without converting them, delegation would have made things worse rather than better.
This is the outermost of four stacked defects and not the reason the job was red: the manager was empty because the object store had no free volume slots, which left a half-installed instance holding the init lock. Those layers are fixed separately. This one is a genuine cross-node bug that the rest of the repo already solved with the same pattern.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The partner probe and both provisioning execs addressed the bare
compose container name zammad-railsserver, which does not resolve in
swarm; the probe silently skipped the whole coupling and the connect
control never rendered. Route them through the container_address
resolver with the shell module (unquoted, subshell expansion), gate the
provision include on IS_STACK_HOST, and retry the rails provision to
absorb Zammad convergence lag. Compose behavior is unchanged: the
lookup returns the bare name there.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The container_address lookup emits $(...) shell syntax in swarm and
documents that callers must use the shell module; the 22 occ tasks
still used command:, so the resolution reached docker literally and
every plugin/LDAP/integration task died on all swarm variants. Convert
them to shell: (args stay single-quoted or [A-Za-z0-9] slugs, so
compose semantics are unchanged).
NFS mounts use local_lock=flock, which makes the stock entrypoint's
init flock node-local: concurrent replicas each won "exclusive" locks
and shredded the rsync install (occ 255, converge livelock). A small
init gate now fronts the entrypoint: only swarm task slot 1 runs the
init, other slots wait for version parity plus config.php; without
TASK_SLOT (compose) it degenerates to a plain exec. The gate ships as
files/nextcloud-init-gate.sh, copied into the build context on the
stack host and referenced through the NEXTCLOUD_INIT_GATE_FILE SPOT.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The container-exec-resolver lint only saw the inline shell form, so the
`command: argv: [container, exec, <name>]` list form (two separate lines)
slipped through, which is how checkmk's bare name reached swarm. Extend the
scanner to parse argv sequences via yaml.compose_all with node line numbers,
and annotate the nine legitimate bare-name execs (mariadb recovery container,
matomo compose-only debug, probe-gated nextcloud partner provisioning, pixelfed
runtime-resolved CID, wordpress launcher-managed discourse) with
# nocheck: container-exec-resolver.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.
- emit emoji names at the source: cli/build/include generator now writes
"🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
integration_zammad reads its OAuth client_id via IAppConfig::getValueString() (plain, relying on the sensitive-value auto-decrypt), unlike integration_gitlab/mattermost which decrypt explicitly via ICrypto getClearAppValue(). The shared set_encrypted_appconfig.php stored every value ICrypto-encrypted, so zammad read back the raw ciphertext as its client_id and the personal connect could not build the /oauth/authorize URL. Add an NC_ENC_MODE switch (default icrypto, unchanged for gitlab/mattermost) and have the zammad provisioning pass sensitive so getValueString() returns the plaintext client_id.
Verified in-container: sensitive round-trips to plaintext, icrypto still yields a pipe-separated ciphertext, and the live Zammad Doorkeeper app uid matches the stored client_id. This also restores idempotency: the ICrypto ciphertext contained a pipe, tripping the re-provision guard on every deploy; the sensitive plaintext does not.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The integration_zammad Connect flow reached the partner /oauth/authorize with a malformed client_id, so Zammad's Doorkeeper rejected the unknown client UID and 302-redirected to its homepage (the spec then captured the partner root, not the authorize page). The encrypted-appconfig helper stored the credentials via setValueString(..., lazy: true) WITHOUT sensitive: true, which mangled the client_id on read. Store it sensitive so the provisioned client_id round-trips intact and the OAuth authorize carries the correct UID.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Provision a Doorkeeper OAuth application on the partner Zammad and write its client_id/client_secret into integration_zammad (Nextcloud-encrypted), so the per-user OAuth connect flow has the credentials the generic SPOT cannot supply. Register both the pretty-URL and /index.php oauth-redirect variants, and drop the hashed-secret fallback so a missing plaintext secret fails loudly instead of persisting an unusable hash.
Strengthen the integration_peertube spec: pin the admin 'instances' value to the exact deployed partner host (new PEERTUBE_BASE_URL) instead of merely 'not Nextcloud', and add a functional OCS search-providers check that fails when the PeerTube provider is not registered. PeerTube has no per-user OAuth (verified upstream: no client_id/oauth_instance_url config, no /oauth/authorize route), so the removed Connect->/oauth/authorize assertion tested a non-existent flow.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
