Commit Graph
151 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 9d940f4040 fix(nextcloud): install onlyoffice from a release that upstream published
ONLYOFFICE listed onlyoffice 10.2.0 in the Nextcloud appstore on 2026-09-18 at
10:16:31Z and never published the matching GitHub artifact; the tag itself does
not exist. `occ app:install` takes the newest compatible release, so it dies on
a 404 that no retry can heal. CI proved that twice independently: upstream run
35329488812 and fork run 35330645276 both fail Nextcloud#4 at that task after
six attempts, on the same commit, for the same reason.

The fault had three layers, each visible only once the one above it was gone.

Install. meta/addons/onlyoffice.yml pins `config.archive` to v10.1.2, whose
artifact answers 200 and which the appstore also lists for this server major.
01_install.yml stages that archive on the node and unpacks it into custom_apps
through a stdin pipe: the webroot is a named volume, so the node can only reach
the app directory through the container, and piping avoids both `container cp`
and a cleanup step that would have to run as a user which does not own the file.

Enable and configure needed no change at all. With the release already on disk
`occ app:install` reports "already installed", which nextcloud_install_status
has always mapped to runnable, so 02_enable.yml and 03_configure.yml proceed as
they always did. That is why this commit adds no branch to the plugin pipeline.

Update. `occ app:update --all` then dragged the pinned app back to the broken
10.2.0 and killed the deploy. occ offers no exclusion flag, but it does offer
`--showonly` and single app ids, so a pinned app is simply not updated. The old
`--all` call stays byte-identical and now carries a `when`: with nothing pinned,
thirteen of the fourteen variants take exactly the path they took before. This
layer cannot appear in CI at all, because it needs a second deploy with the app
already installed, and the install always died first.

The classifier test is inverted rather than deleted:
test_an_advertised_release_whose_artifact_is_gone_stays_fatal pins the decision
that a dead artifact fails loudly instead of degrading to a silent skip.

Verified by a local compose deploy of variant 4: ok=2508 changed=149 failed=0
and `✓ 36 tests/addons/onlyoffice.spec.js:12:1`, which drives the whole
JWT-authenticated round trip to the document server over its onion address, not
merely an enabled plugin. In the live container beforehand: `occ app:enable`
returned "onlyoffice 10.1.2 enabled", `occ app:install` returned "already
installed", and `occ app:update onlyoffice` reproduced the 404 that the pin now
avoids.

The loop expression was wrong before this commit and is fixed here: Jinja reads
'\1' as an octal escape, so the backreference produced chr(1) instead of an app
id. Evaluated against ansible-core's own filters over four shapes, the tail is
now cut instead of captured, which removes the escaping hazard entirely.

occ output is not localised: with default_language=de and a de_DE.UTF-8 locale
the strings are unchanged, so parsing them is as safe as the five markers the
role already relied on.

Residual: the per-app update branch has never run with a real item inside a
deploy. Its chain and `occ app:update <id>` are each verified separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 05:55:51 +02:00
kevinveenbirkenbachandClaude Opus 5 70feb319be feat(nextcloud): front the Context Agent with the MCP adapter
The hub serves 94 tools across 23 categories, write tools included, and enabling the ExApp switches every category on. That surface was published on the public vhost and every discovered client was handed an app password for it. svc-ai-mcp-adapter now sits in front in mcp_passthrough mode: clients present credentials.mcp_bearer to contextagentmcp and reach the nine read tools of calendar, contacts and files, which files/mcp/tools.json names and the sidecar enforces per call. Those three categories are the ones whose upstream is_available is unconditional, so the served set does not vary with which optional apps a deployment installs.

Enforcing it in the sidecar rather than in each client is what makes it hold: web-app-openwebui renders no per-tool filter at all, so a client-side include list would have bounded two consumers out of three.

06c13e7f3e withdrew the declared path from the edge for exactly this reason - "that surface is what the adapter exists to narrow". Nextcloud answers the same route under a second spelling, without the /index.php front controller, which try_files rewrites back onto the route after the edge has already declined to block it. The withdrawal becomes a regex covering both, and only where stripping the front controller changes the path, so the four adapter providers without one render unchanged.

The app password is minted after the stack renders, so it reaches the sidecar through upstream.env and rebuild.yml rather than the compose environment: compose up resolves env_file while creating the container, a restart does not. The sidecar is the last service in the template because a Jinja set has no block scope and its env_file would otherwise displace the one talk, whiteboard and talk_recording each carry.

Two suites cover what nothing measured: the vhost template rendered without a container, asserting both spellings are withdrawn, and every shipped tools.json loaded through the adapter own loader, so a bad contract fails in a second instead of as a probe timing out an hour into a deploy. The guest spec probed mcp.endpoint.path, which this change makes the proxy answer 404 for its own reasons; it now probes all three public paths and requires no MCP body.

Not yet deployed: the sidecar build, the upstream.env delivery and the hub answering exactly nine tools are unverified against a running stack.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-03 20:55:05 +02:00
kevinveenbirkenbachandClaude Opus 5 22425d6f83 fix(nextcloud): keep the ExApp dial off the tor proxy that cannot resolve it
The Context Agent this branch introduces is the first thing Nextcloud dials by
a container name from inside its own server process. proxy.config.php.j2 sets a
global socks5h proxy whenever SSO runs against an onion issuer, and socks5h
hands name resolution to Tor, which has no idea what nextcloud-context-agent
is. Every attempt stalls to cURL's 30s ceiling, the AppAPI retry loop repeats
it, and SYS_TIMEOUT_CONTAINER_EXEC shoots the exec after an hour. The registration
returns 124 and the whole e2e stage behind it never runs.

The exclusion list was written for appstore downloads and lists five public
hosts. It now carries the internal names this same file defines as dial
targets, split so the two kinds stay legible: the role's own proxy and the
Context Agent, both resolved through their mode-aware DNS name so the exemption
holds under compose and swarm alike.

The template is byte-identical to main and the list's gap is as old as the
list, but main declares no context_agent at all, so nothing there ever dialled
an internal name and the gap stayed invisible. This is the branch's own dial
walking into it.

Evidence from run 33574056480: the ExApp was healthy throughout, its log
carrying 142 heartbeat answers, every one of them from 127.0.0.1 and not one
from Nextcloud, while Nextcloud logged eleven cURL error 28 timeouts at 30002ms
spaced 31s apart. The dial dies at the caller.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 17:46:12 +02:00
kevinveenbirkenbachandClaude Opus 5 6ac48b7b8a test(services): hold a compose service guard to its services entry
A compose.yml.j2 that wraps a service key in {% if VAR | bool %} has to reach
that service's own meta/services.yml entry, or the template and the
declaration become two switches for one container: the resource model, the
README graph and the network layer read the entry while the container follows
something else.

The reference may sit anywhere in the chain. The resolver walks the role's
vars/ and defaults/ and follows config paths into the resolved application
config, because nextcloud reaches its Talk entry over
addons.spreed.enabled and a resolver that stopped at role variables called
that a bypass. Only guards indented by at most two spaces count: deeper ones
select content inside a service, not the service.

The rule found one real bypass. NEXTCLOUD_MCP_ENABLED read mcp.enabled while
switching the context_agent container, whose entry admitted two peers where
mcp.enabled admits three, so a round carrying only web-app-openwebui ran a
container its own entry declared off. mcp.endpoint.path routes through
app_api/proxy/context_agent/mcp/, so the container is the MCP surface and the
entry was the stale side. Both ends now meet at the entry.

talk_recording carried no enabled at all and fell back to
_looks_like_container, so the resource model billed it unconditionally. It
now reads services.talk.enabled, which is what its guard already computed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 17:55:59 +02:00
kevinveenbirkenbachandClaude Opus 5 7162b7bf4c Merge fork/feature/svc-net-tor into feature/ai-agent-employees
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.

The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.

Resolutions that were not unions:

Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.

plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.

compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.

tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.

Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.

01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.

Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.

roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.

config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.

READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.

Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 16:32:30 +02:00
kevinveenbirkenbachandClaude Opus 5 43260cdee3 fix(nextcloud): size the php-fpm pool for the Context Agent's callback
The context_agent ExApp calls back into the same php-fpm pool it is being
proxied from: its UserAuthMiddleware resolves the bearer against
/ocs/v2.php/cloud/user while a child is already parked forwarding the MCP
request out to the ExApp. Every in-flight MCP request therefore occupies TWO
children. Below twice mcp.limits.concurrent_requests the two legs deadlock, and
because the ExApp's callback carries no timeout the pool never recovers -- which
is why the Context Agent sat unhealthy rather than merely slow.

services.nextcloud.performance.php.fpm now sizes the pool explicitly:
max_children 16, start_servers 4, min/max spare 4/8. request_terminate_timeout
is 900s and must not drop below the proxy's fastcgi_read_timeout; lower kills a
child while nginx is still waiting on it, equal means only work nginx has
already abandoned gets reclaimed.

The readiness path was spending the same scarce workers. /healthz/ready had no
location of its own, so it fell through to `location /`, was rewritten to
/index.php/healthz/ready and booted a full Nextcloud request on a php-fpm child
once per probe interval against a pool of five. It is an exact-match location
returning 200 from nginx now, with access_log off, and never reaches PHP.

The pool file is rendered through render_replicated_templates.yml so swarm peers
receive it too, and mounted from meta/volumes.yml at a path that sorts after
zz-docker.conf, which reopens the [www] section last -- mount it earlier and the
image's own values win while the deploy looks correct.

The context_agent healthcheck moves from a hand-written probe in compose.yml.j2
to the shared container_healthcheck lookup, declared as flavor curl on path
heartbeat with start_period 20m. The ExApp installs dependencies on first boot
and legitimately outlives a 60s start period, which is the other half of why it
reported unhealthy.

Not verified: none of this has deployed. max_children 16 is derived from twice
concurrent_requests rather than measured against real memory use, and the
mount-ordering requirement is read from the image layout rather than confirmed
against a merged pool config in a running container -- if it sorts wrong the
whole change is inert while appearing correct. request_terminate_timeout 900s
also means a genuinely stuck request now holds a child for fifteen minutes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 14:04:33 +02:00
kevinveenbirkenbachandClaude Opus 5 acc9e25c0f refactor(volumes): one canonical mount surface, context-wired lookups
The mount pipeline was split across two layers: compose_volumes and
container_volumes lived as filter modules that ansible's filter loader
rejected on every start ("has no attribute 'FilterModule'", two warnings
per run), while the same-named lookups wrapped them thinly. The filter
bodies now live inside their lookups (the tor_socks precedent), the
filter files are gone, and the render tests moved next to the lookup
tests.

As lookups they see the templating context, so the application_id term
every call site passed back to its own role is gone too: both lookups
read application_id from context and fail loudly when it is missing.
107 templates and nfs_prep.yml drop the term; the calls collapse to
lookup('compose_volumes') and lookup('container_volumes', service_name).

The container_volumes call is now canonical - four leading spaces,
service_name in the service position, | indent(4) - enforced by a new
lint (test_container_volumes_call_shape). Normalizing the 85 literal
call sites surfaced that erpnext keyed its meta/volumes.yml mounts by
container name instead of compose service key, the only role to do so;
its meta now uses the service keys. wordpress's column-0 call, which
rendered its first line at top level, is indented like every other.

Template logic that only restated meta/volumes.yml is deleted: mailu fed
compose_volumes an extra_volumes dict whose eight names were looked up
from the very meta file compose_volumes reads itself; zammad's init and
railsserver extra mounts were the volumes.yml entries minus the two
services, which the meta now lists (the when-gated OIDC binds included).
Conditionality that meta cannot express stays in code, but in role-local
lookup plugins instead of template loops: matrix_bridge_mounts renders
the per-bridge registration mounts, nextcloud_extra_volumes the
feature-gated whiteboard/recording volumes (type: volume entries are
declared unconditionally by compose_volumes, so gating must happen
before it).

Parity is proven, not assumed: every mechanically rewritten template is
byte-identical to HEAD after the transformation regexes (95/95), and the
semantic roles render identically through the real functions against
HEAD's meta - zammad across six sso/tls polarities, all seven erpnext
services, mailu's top level. make test-lint passes complete.

Also: suitecrm's compose template pins container_extra_hosts with
host_alias (the SOCKS proxy URL host.docker.internal never resolved in
compose+tor), and the five disabled mautrix bridge addons use the
empty-default config lookup so templating the applications dict stops
emitting 125 errors per batch in every CI job.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 23:28:16 +02:00
kevinveenbirkenbachandClaude Opus 5 783bdd1720 fix(nextcloud): keep the appstore mirrors off the tor proxy too
The proxy exists only so libcurl will speak to an .onion issuer at all -- it refuses such a host by name before any resolution (RFC 7686, error 6), as 20ddda8c1 measured. Everything else that crosses it is collateral, and proxyexclude is what keeps it out. apps.nextcloud.com was listed for exactly that reason.

But Guzzle evaluates the list per hop, against the host the handler is handed. The appstore answers with a redirect to a mirror, path identical, host swapped, and garm2/garm3.nextcloud.com were not listed -- so hop 1 went clearnet and succeeded while hop 2 went through Tor and died with error 97. All 24 garm URLs in the failing job are that one endpoint, /api/v1/apps.json; not one is a download.

A suffix entry covers the entry host and every mirror it redirects to, instead of chasing garm2, garm3, garm4 one at a time. The form is already in use in this same list: .githubusercontent.com carries the leading dot for the same reason.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 03:38:40 +02:00
kevinveenbirkenbachandClaude Opus 5 29015311b6 feat(lint): derive container paths from meta/volumes.yml everywhere
meta/volumes.yml says where a volume lands inside its container. Wherever a
second file spelled the same path out, the two drifted independently: the mount
moves, the consumer keeps pointing at the old place, and nothing fails until the
process looks for content that is not there. e031b7f61 fixed exactly this for
web-app-moodle; this generalises it and makes the rule hold.

The lookup gains what a caller needs. `targets` maps each mounting service to
its container path, and `target` is the one path an entry mounts at -- present
only when every mount agrees. An entry that lands at different paths per service
omits `target` rather than picking one, and the caller names the service:

    "{{ lookup('volume', application_id, 'prometheus_config').target }}"
    "{{ lookup('volume', 'web-app-prometheus', 'host_sys').targets['node-exporter'] }}"

66 literals across 48 role files now derive. Replacement only happened where the
path stood alone -- /srv/app inside /srv/app/data is a different path -- and
every rewritten file was re-parsed, Jinja and YAML alike. Three sites the tool
refused rather than guess were done by hand: zammad's path sat in a Jinja list
literal, seaweedfs' inside a quoted string feeding a filter, and openresty's
/var/run/docker.sock is not a mount target at all but the socket on the host,
which only happens to spell the same string. That one carries the single
nocheck.

tests/lint/repository/spot/test_volumes_path_single_spot.py keeps it that way.
roles/*/files/ is out of scope by construction: it is copied verbatim into an
image and executed there, so a Dockerfile or entrypoint has no Ansible to
evaluate a lookup with.

One defect this found, and it was in the tooling rather than the roles: deciding
ambiguity AFTER filtering short paths made web-app-prometheus' host_sys look
single-valued -- it mounts /sys for cadvisor and /host/sys for node-exporter,
and /sys is below the length filter. The template would have rendered an
undefined `.target`. Both the converter and the rule now decide ambiguity over
every target and filter afterwards, and that call reads targets['node-exporter'].

Verified by resolving rather than by reading: all 70 accesses in the diff were
run against the real registry and every one yields a non-empty path equal to the
literal it replaced. make test-lint: 328 passed.

Not verified: no Ansible render, so a call site whose surrounding context cannot
build the applications registry would still fail at deploy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 02:02:29 +02:00
kevinveenbirkenbachandClaude Opus 5 f7bb608927 refactor(nextcloud): sort files by language, and let Doorkeeper accept onion
files/ now mirrors the role's own structure: php, ruby and shell, each split
into addons for the per-integration scripts that tasks/addons/ drives and utils
for what more than one caller shares. set_encrypted_appconfig.php serves gitlab,
mattermost and zammad; doorkeeper_allow_onion_redirect.rb serves zammad and
openproject; init-gate.sh belongs to the role itself. The partner directories
survive because two scripts are both named provision_oauth_app.rb.

NEXTCLOUD_INIT_GATE_FILE had to split. It was the source path under files/, the
name Dockerfile.j2 COPYs into /usr/local/bin, and the ENTRYPOINT, all at once;
moving the script would have produced /usr/local/bin/shell/utils/init-gate.sh.
The basename keeps that name and NEXTCLOUD_INIT_GATE_SRC carries the path.

The same commit closes two of the three faces of one defect. resolve_enabled
reports TLS off for an .onion domain, correctly -- the vhost is plaintext and
Tor carries the transport crypto -- so every redirect URI built from
lookup('tls', ...) is http, and Doorkeeper rejects it. Zammad raises out of
app.save! and fails the task at rc 1; OpenProject discards result.errors, keeps
the unsaved model whose plaintext_secret already exists, reports
oauth_application_created:true, and only dies three tasks later on a null
client_id. The prelude relaxes force_ssl_in_redirect_uri for .onion hosts and
delegates every other host to the previous setting, so clearnet is unchanged at
both partners; the OpenProject script now raises with the real validation
errors.

The third face is PeerTube, whose Keycloak callback is registered http and
requested https, and it stays open: fixing it means changing either PeerTube's
scheme, which is its federation identity, or the registration side.

Evidence: run 31658270172, jobs 94319317307 ("Redirect URI must be an HTTPS/SSL
URI.") and 94319317278 (changed:true, then length on NoneType).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 02:02:27 +02:00
kevinveenbirkenbachandClaude Opus 5 f713ba8350 refactor(credentials): address them at secrets.credentials, like every other meta file
meta/schema.yml became meta/secrets.yml, but its content still landed at
applications.<app>.credentials — the one meta file whose payload did not sit
under a root named after the file. Every other topic obeys that rule:
meta/volumes.yml goes to volumes, meta/rbac.yml to rbac. Credentials now go to
secrets.credentials.

The path exists exactly once, in utils/manager/credential_key.py, and
OVERRIDE_SECTION is derived from its two segments rather than spelled beside
them, so the dotted form used by --set and the nested form used by the
inventory cannot drift. Six consumers read it: the loader, the generator, the
CLI override parser, the two emit helpers, and the four lookup validators.

--set now accepts exactly one key shape,
applications.<app>.secrets.credentials.<key>. The three short forms are
rejected instead of guessed at: the same short key names different credentials
on different apps. That also retired primary_app_id, which only existed to
disambiguate them.

Three defects surfaced while wiring this up. _apply_one_role_schema still wrote
to the old root, so an app's own credentials would have moved while its
providers' stayed behind. web-app-pgadmin reads through
lookup('applications', ...).credentials.<key>, an access path no regex over
lookup('config', ...) could see, and would have deployed with an empty postgres
password. And _META_NON_GENERIC_TOPICS still listed 'schema' — a bare topic
name, invisible to any path search — so the generic file-root loop assigned the
raw schema under secrets and the follow-up write landed inside the cached
document, truncating keycloak's four credentials to one for every later
reader.

tests/integration/ansible/lookups/config/test_credentials_roundtrip.py pins both
directions of the seam and the cache invariant that last defect broke.

Existing inventories still carry the pre-rename root; migrating them is
deliberately out of scope here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 16:51:56 +02:00
kevinveenbirkenbachandClaude Opus 5 28475455db feat(mcp): give every provider its own service account from the registry
Nine roles reached their MCP surface as the deployment administrator or with
an account password kept beside the account in credentials. A Moodle site
admin bypasses every capability check, a Nextcloud app password is not
scopable, a Gitea token reads every private repository, and a GitLab token on
root reads every private project. Each role now declares its own principal in
meta/users.yml and provisions it, so the exposed surface is what the service
role permits. jellyfin, zammad and snipe-it kept their account but took its
password from a credential named token; they read the registry too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 15:44:12 +02:00
kevinveenbirkenbachandClaude Opus 5 e96c752549 Merge fix/playwright/env-template-gate into feature/svc-net-tor
75 commits: the socialhome role reimplemented against the project it actually
deploys, native-login specs for funkwhale and shopware, a window.open IdP
handoff and a settings-link fallback in the shared persona helpers, a swarm
converge budget that gives up on a terminal task, and a long tail of role
fixes.

Eleven conflicts. The shared persona helpers (keycloak.js, logout.js) take the
incoming structure — they gained real behaviour this branch never had — with
this branch's resolveTimeout convention re-applied over it and
confirmKeycloakLogoutIfPrompted re-attached at every return point; without it
the Keycloak SSO session outlives the logout. playwright.config.js keeps this
branch's version: both sides set actionTimeout, but ours routes through an env
var and 15s is too short behind an onion. socialhome's spec takes the incoming
content — 2dbd6eb10 found the role described the wrong upstream project
entirely — and keeps gotoOnion and expectHstsWhenTls, since demanding HSTS over
a plaintext onion is a false failure. Both READMEs were regenerated rather than
merged by hand.

Three collisions arrived with no conflict marker at all, because each side
changed a different place in a different file:

- group_vars/all/18_swarm.yml declared swarm_stack_converge_retries twice.
  Same value, so nothing breaks today; editing the documented one would have
  silently changed nothing.
- svc-net-tor and web-app-socialhome both claimed 192.168.74.0/24. Socialhome,
  the newer role, moves to 192.168.75.0/24.
- web-app-socialhome/meta/schema.yml arrived as a new file, so ef5ea7cb5's
  rename never saw it. Its credential would not have resolved and the role
  would have deployed with an unusable administrator account.

The incoming specs also needed this branch's conventions: raw timeouts wrapped,
four page.goto calls routed through gotoOnion, a request.get given an
onion-aware timeout, and 21 roles marked playwright-service-flag over the tor
block this branch added to them.

make quality green on the merge result.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 14:20:19 +02:00
kevinveenbirkenbachandClaude Opus 5 028e80a471 fix(web-app-nextcloud): let partners resolve locally in development
Nextcloud's SSRF guard answers every server-side call to an RFC1918
address with 401 'violates local access rules'. integration_mastodon
registers its OAuth app dynamically against the partner, so in the swarm
lab -- where every partner resolves inside 192.168.244.0/24 -- the
integration never connects and the addon spec never leaves Nextcloud.

The switch is a service config entry rather than a hardcoded value, so a
deployment whose partners really do sit on a private network can turn it
on without the role stomping the setting on every deploy. Default is on
in development only; production keeps the guard.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 01:08:24 +02:00
kevinveenbirkenbachandClaude Opus 5 20ddda8c1c fix(nextcloud): reach the onion OIDC issuer over Tor SOCKS again
libcurl refuses a .onion host by name (RFC 7686, error 6) before it
consults /etc/hosts, extra_hosts or DNS, so the platform's DNS+iptables
onion egress cannot serve a server-side backchannel at all. Measured:

    curl --noproxy '*'                          -> exit 6
    curl --resolve auth.<onion>:80:127.0.0.1 …  -> exit 6

--resolve fills curl's own cache strictly earlier than getaddrinfo and
is still refused, which makes sys-svc-container's onion_oidc_socks
template -- extra_hosts, despite its name -- a no-op for every libcurl
client. Only a configured proxy makes curl skip the check.

Since 9d91fbea8 emptied proxy.config.php.j2 there is no such proxy, while
svc-net-tor's exclusive/primary defaults replace Keycloak's clearnet
domain and KC_HOSTNAME pins the issuer to the onion. Nextcloud's
oidc_login discovery call therefore dies inside PHP, /apps/oidc_login/oidc
returns 500, and oidc_login_auto_redirect sends /login straight there.

Restoring the old socks5h://host.docker.internal:9050 alone would make it
worse: tor runs as a single manager-pinned replica publishing SOCKS with
mode: host, so only the replica on that node could connect and a
deterministic failure would become a 2-in-3 flake. In swarm, though, tor
is NOT host-networked -- that branch is compose-only -- and it already
binds SocksPort 0.0.0.0, so the shared overlay is a usable path from
every node.

Give svc-net-tor a shared_net overlay with alias 'tor' and dial it by
service name. Attachment is gated by a new consumer kind, onion_sso: the
consumer's services.sso.enabled AND the provider's services.tor.enabled.
Both are group-membership derived, so the conjunction is exactly "the
issuer is an onion" and stays false in clearnet deployments -- which
matters because the service registry is not filtered to deployed roles.
The existing kinds could not express it: keying on sso alone would also
hold without tor, and keying on tor would miss Nextcloud's variant 2,
which disables its own onion yet still needs the client path.

tor_socks_proxy keeps the swarm/compose split in one place so the six
roles sharing this defect need not restate it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 00:50:48 +02:00
kevinveenbirkenbachandClaude Opus 5 5bd3239994 fix(web-app-nextcloud): provision the GitLab OAuth coupling in swarm too
The gate probed the partner with container inspect on a bare name, which only resolves under compose co-location. In swarm GitLab sits on another node, the probe fails, and failed_when: false turns that into a silent skip: the deploy reports success while client_id/client_secret are never written, so the per-user Connect button leads nowhere. Nothing else writes those two values.

Mirrors integration_zammad: resolve node and container id first, wait for the partner endpoint, then run both execs against the resolved ids. Gate on IS_STACK_HOST like every other addon. The include had to become relative, or the stack-host lint cannot see the gate through a templated path.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-08 10:15:07 +02:00
kevinveenbirkenbach 06bc856c32 refactor(mcp): make MCP a role topic instead of a service entry
The block sat under services but never behaved like a service. A service has
exactly one providing role, declared with provides, so the registry can map the
key to that role and consumers bind their flag to its presence. No role declares
provides: mcp. There are seventeen providers and the pairing is done by
allowed_consumers inside the block itself.

Every rule that enforces the service model therefore needed an exemption for it:
a shared-provider suppression in fourteen roles, a playwright-service-flag
suppression, and two freshly added exemptions in the dynamic-flag and
variants-static-override checks. Four independent rules, four escape hatches,
one cause. They are all gone now, together with a bond key that only the
resource budgeter reads and only for entries under services, and a wordpress
addon bridge that pointed at a service block that no longer exists.

The block moves to meta/mcp.yml. The applications loader maps meta/<name>.yml to
applications.<app>.<name> generically, so the move needs no wiring; the file
takes the same shape as meta/rbac.yml. Every config path follows, the variant
overrides are lifted out of their services map, and the vocabulary module leaves
the services package because it no longer describes one. roles_with_service
gains an explicit topic parameter rather than an implicit fallback, so nothing
silently resolves against the wrong container.

Files that are otherwise unrelated to this move are included where the committed
tree would otherwise keep pointing at a path that no longer resolves: thirteen
vars files, eleven playwright env templates and one task file.
2026-08-06 20:36:00 +02:00
kevinveenbirkenbach 342de43111 Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
# Conflicts:
#	roles/svc-ai-ollama/tasks/01_manager_ops.yml
#	roles/web-app-decidim/templates/compose.yml.j2
#	roles/web-app-joomla/templates/compose.yml.j2
#	roles/web-app-matrix/files/flavor/ansible/runner/bootstrap.service
#	roles/web-app-matrix/templates/flavor/compose/services.yml.j2
#	roles/web-app-nextcloud/files/playwright/_shared.js
#	roles/web-svc-cdn/tasks/01_assets.yml
#	scripts/tests/deploy/swarm/utils/trigger_units.sh
#	scripts/tests/deploy/utils/filesystem/docker_dataroot.sh
2026-08-05 02:32:58 +02:00
kevinveenbirkenbach 0b192dac99 Merge remote-tracking branch 'fork/feature/swarm-nfs' into feature/ai-agent-employees 2026-08-05 01:48:14 +02:00
kevinveenbirkenbachandClaude Opus 5 a0e5710018 feat(nextcloud): route the application log to stderr
Nextcloud's log_type was never set, so the default 'file' applied and every
application exception went to data/nextcloud.log on the NFS volume. The rescue
diagnostics collect `docker logs` per service, which only ever saw the FPM
access log - so an exception is invisible in CI artifacts.

That cost a diagnosis. In run 30859490335, job 91840410448, the OIDC login
completed (302 /login -> 302 /apps/oidc_login/oidc -> 303 with the code) and
/apps/dashboard/ answered 500. The access log counts 15 x 303 against 15 x 500,
i.e. five addon specs times three attempts, every single authenticated request
failing - deterministic, not flaky. The trace holds only Nextcloud's generic
error page, "More details can be found in the server log", and that log is not
in the artifact.

'errorlog' sends the exception through PHP's error_log to FPM's stderr and from
there into `docker logs`, which the existing collection already captures. No new
collection code and no reads from the NFS volume. 'syslog' and 'systemd' would
both need a socket inside the container.

This does not explain the 500 in that run; it makes the next one explainable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 08:37:39 +02:00
kevinveenbirkenbachandClaude Opus 5 67aedbeb62 Merge branch 'feature/swarm-nfs' into feature/ai-agent-employees
Takes the revert of the rendered system config. The MCP work had added the
proxy service name and its swarm DNS name as trusted domains to
system.config.php.j2, which the revert deletes; both entries move back to
vars/system.yml as trusted_domains 1 and 2, where they lived before aeefb1407.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 23:51:11 +02:00
kevinveenbirkenbachandClaude Opus 5 fc0d7c95b0 Revert "fix(nextcloud): render system config instead of setting it via occ"
This reverts commit 41063206a.

It broke every swarm nextcloud deploy. system.config.php.j2 opened with

  <?php
  {%- macro php_str(value) -%}

and the leading dash strips the newline after `<?php`, so under Ansible's
trim_blocks the file renders as `<?phpreturn [`. Run 30848540165:

  ParseError: syntax error, unexpected token "=>", expecting "]"
    in /var/www/infinito/system.config.php:2
  #0 /var/www/html/lib/private/Config.php(229): include()

Config->readData() includes the file, so occ upgrade, every later occ call and
every php-fpm request died on an uncatchable compile error. The template had
been verified by rendering it with plain Jinja defaults, which do not match the
ones Ansible uses - the check confirmed the escaping and missed the line above.

A one-character fix (`{% macro`) is correct and was verified end to end, but the
commit is reverted rather than repaired, because its premise did not survive
review:

- The failure it targeted is not deterministic. The same task ran 13 times
  across the swarm jobs of run 30830377785 and passed 12 times, including a
  second fedora job on the same commit and topology. It is an ~8% flake.
- It does not remove the mechanism. 03_upgrade.yml:8 runs
  `occ maintenance:repair --include-expensive` unconditionally, and upstream
  Repair.php sets the maintenance system value twice with no guard - two
  exclusive locks on the same NFS config.php per deploy, unretried.
- The benefit is largely illusory. Config.php:279 var_exports the whole merged
  cache back into config.php, so the nine values land in the locked file anyway
  on the first maintenance:repair, with a larger payload.
- It traded away occ's exit-code confirmation that Nextcloud accepted the
  values, and moved them from the shared NFS volume to a node-local bind.

The remaining exposure is the unretried occ calls in 03_upgrade.yml; a bounded
retry there is the smaller and better-targeted change and is left for its own
commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 23:15:26 +02:00
kevinveenbirkenbach aeefb14076 Merge branch 'feature/swarm-nfs' into feature/ai-agent-employees 2026-08-03 22:24:36 +02:00
kevinveenbirkenbachandClaude Opus 5 e3068a37d8 fix(nextcloud): render system config instead of setting it via occ
Nine sequential `occ config:system:set` calls each open config.php exclusively.
In swarm that file lives on the NFS share behind every replica, and the seventh
call could not take the lock:

  In Config.php line 306:
    Could not acquire an exclusive lock on the config file
    /var/www/html/config/config.php

stdout shows six values written before the abort, so the run left a partial
configuration behind. That failed the swarm web-app-nextcloud 5 job in run
30830377785 while every PLAY RECAP on the other nodes reported failed=0.

The same nine values now come from templates/config/system.config.php.j2, which
rides the additional-config channel the role already uses for redis, ldap and
the rest: rendered to NEXTCLOUD_HOST_CONF_ADD_PATH, mounted read-only, merged by
zzz-infinito.config.php after config.php so its values win. That path never
takes an exclusive lock, and it runs before the stack starts, since all nine
values are statically derived and none needs a live container.

Two behaviours are deliberately not carried over. The --type flags become native
PHP types, which is stricter than the string coercion occ applied. And nothing
confirms the file was accepted any more, where occ's exit code did; a read-only
`config:system:get` would restore that assurance at the cost of an occ call, and
is left out until someone wants it.

php_str() escapes backslashes and single quotes, matching the idiom already used
for the redis password in memcache.config.php.j2. Without it a quote in any
value would be a PHP syntax error, and include_once would drop every additional
config file, not just this one.

vars/system.yml and 02_manager_ops/05_system_config.yml are gone; the later task
files shift down one number so test_task_file_numbering stays satisfied.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 22:02:50 +02:00
kevinveenbirkenbach 255ee13f60 Merge branch 'feature/swarm-nfs' into feature/ai-agent-employees 2026-08-03 04:21:37 +02:00
kevinveenbirkenbachandClaude Opus 5 e3e81e816f refactor(healthcheck): declare probes in services.yml, render them from one lookup
Seven include templates under sys-svc-container decided how a container was
probed, and they read their inputs from template-local `{% set %}` scope.
That scope leaks: a value set for one service silently reached every service
below it, which is how taiga's front and gateway ended up probing a port
declared for a different service, and how bluesky's frontend probed a PDS
endpoint.

Probes now live in utils/docker/healthcheck.py, one class per flavor behind a
PROBES registry, and each service declares its own:

    healthcheck:
      flavor: curl
      path: health/ready
      start_period: 20m

`port_key` picks another entry under ports.internal for the services that
expose their probe elsewhere (keycloak on management, nextcloud talk on
signaling). Without a flavor the entry must carry an explicit test argv, so a
missing declaration fails loudly instead of rendering a probe against nothing.

Rendering HEAD's templates against the new output confirms 49 of 56 call
sites are byte-identical; the remaining seven are covered by an Ansible run.
Three probes change on purpose: erpnext's backend gains a check it never had,
bluesky's frontend drops the leaked PDS path, and msmtp_curl moves from a
hardcoded localhost:80 to the service's own port on 127.0.0.1 like every
other flavor.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 04:05:38 +02:00
kevinveenbirkenbachandClaude Opus 5 c7ddf6eb24 Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Three conflicts, each resolved by combining both sides rather than picking one.

compose.yml passes both env vars through to the container: swarm-nfs adds
INFINITO_SWARM_MGR_IP, this branch adds INFINITO_TOR_SOCKS_PORT, and neither
substitutes for the other.

daemon.json.j2 takes the data-root block from swarm-nfs and keeps HEAD's dns
key. The incoming side writes networks.internet.dns directly; lookup(
'container_dns') already returns that value as its clearnet entry and prepends
the docker0 bridge on a Tor node in compose mode, so the incoming form is a
strict subset and dropping it loses nothing.

The WordPress CA-trust addon spec gains skipUnlessServiceEnabled("sso") from
swarm-nfs alongside HEAD's gotoOnion and resolveTimeout, and asserts on the
Plugin Name the mu-plugin header declares - the mustuse screen renders that,
not the filename - with resolveTimeout applied to the literal the incoming
side left bare.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 00:47:10 +02:00
kevinveenbirkenbachandClaude Opus 5 e6ce6d2df9 refactor(nextcloud): single-source the fpm port and app user
The php-fpm listener port lived as a literal 9000 in the nginx sidecar
config, and the application user as a literal www-data in both
healthchecks, while the role already carried NEXTCLOUD_DOCKER_USER and
declares every other internal port in meta/services.yml.

Declare the listener as services.nextcloud.ports.internal.fpm and read
it through the config lookup, the same shape the moodle and magento
roles use for their fpm listeners and the same shape this role already
uses for the signaling and recording ports. That also satisfies the
port lint, which requires every *_PORT constant in a role's vars to
resolve through the lookup rather than carry a literal.

The rendered output is unchanged: the port is still 9000 and the user
is still www-data. After this change no www-data literal remains in the
role's code; the occurrences under docs/ stay as they are, since those
are copy-paste operator recipes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 00:18:24 +02:00
kevinveenbirkenbachandClaude Opus 5 a2be4f081a feat(nextcloud): serve MCP through the Context Agent ExApp
Nextcloud carries no MCP server of its own, so the role runs the Context
Agent as an ExApp: render its container while MCP is enabled, wait for
its heartbeat, register it with AppAPI, add the MCP route, enable it, and
only then mint the app password clients present.

Split the credentials by origin. The shared secret between the AppAPI
proxy and the ExApp is generated by this deployment and stays a vault
credential; the app password is minted by Nextcloud through occ and is
persisted through sys-token-store instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 22:24:33 +02:00
kevinveenbirkenbachandClaude Opus 5 0efb3e79e7 refactor(nextcloud): give the config loader path a single definition
The path stood verbatim twice in one command line, once for the redirect and once for the chmod. A change to one of them would have written the file to A and set permissions on B, silently.

It now derives from NEXTCLOUD_DOCKER_WORK_DIRECTORY the same way NEXTCLOUD_INTERNAL_OCC_COMMAND already does, so the loader follows the work directory if that ever moves. Rendered value is byte-identical to the literal it replaces.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 14:04:17 +02:00
kevinveenbirkenbachandClaude Opus 5 a963280c1c fix(nextcloud): run occ on the node that hosts the task
NEXTCLOUD_DOCKER_EXEC resolved the container id through a subshell evaluated on the manager. When the manager holds no running replica, resolve_container_id.sh falls back to a cluster-wide lookup with no node filter and returns the id of a task on another node, which the local daemon does not know: 'No such container', sixty times, then the wait in 02_config.yml fails the deploy.

08_manager_ops.yml now resolves the hosting node and its node-local container id once via resolve_host_cid.yml and delegates every include there with apply:. Redefining NEXTCLOUD_DOCKER_EXEC in terms of NEXTCLOUD_LOCAL_CID lets all seventeen consumers inherit the fix without touching a task line. It re-resolves after the handler flush and after the upgrade, the two points where a rolling restart can move the task. Compose degenerates to delegate-to-self with the previous container name.

Four call sites built their exec inline because they need -u root or -e flags, and their subshell would have run on the worker where resolve-container-id exits 64 for want of a manager. Without converting them, delegation would have made things worse rather than better.

This is the outermost of four stacked defects and not the reason the job was red: the manager was empty because the object store had no free volume slots, which left a half-installed instance holding the init lock. Those layers are fixed separately. This one is a genuine cross-node bug that the rest of the repo already solved with the same pattern.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 14:04:17 +02:00
kevinveenbirkenbachandClaude Opus 5 3a1f11fac8 fix(nextcloud): resolve the admin LDAP identities synchronously
07_admin_identities.yml registered 'occ ldap:search <admin>' with async/poll, and ASYNC_POLL is 0 whenever ASYNC_ENABLED is true. The register then holds an async job handle with no stdout, and thirteen lines later the set_fact parses exactly that stdout for the '(identity)' pattern. The list therefore collapsed to [NEXTCLOUD_ADMINISTRATOR_USERNAME] on every swarm run, the LDAP-mapped administrator_<n> never entered the admin group, and failed_when could not complain because it is keyed on the same global flag.

The symptom is a 403 on every admin-scoped route while user routes return 200. The Playwright trace of one session carries data-user=administrator_2981 and _oc_isadmin=false on BOTH - and _oc_isadmin comes straight from IGroupManager::isAdmin(), so it states the group membership rather than the route's verdict. The compose lane stays green only because it runs a synchronous pass first, where the same task does have stdout and adds administrator_3447; its later async pass degenerates identically.

Three tasks become synchronous: the identity resolution, the group:adduser loop that writes the membership, and the LDAP cache warm in addons/user_ldap.yml whose task name promises the users are present. Measured cost 2.533 s + 0.730 s against a 2:30:13 job. The 'not ASYNC_ENABLED and' guards stay: they hang on the global flag, not on the task being async, so removing async does not arm them.

The condition moves into NEXTCLOUD_USER_LDAP_ENABLED and the two tasks sharing it into one block.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 14:04:15 +02:00
kevinveenbirkenbachandClaude Opus 5 9d91fbea83 feat(svc-net-tor): route onion names through tor via dns, not per-app proxies
Reaching an onion address required each role to point its own http client
at the tor socks proxy, which then carried clearnet traffic too — fatal
in the test network, which has no exit, and the reason nextcloud needed a
per-host exclusion list.

The egress router already redirects tor's virtual range to the TransPort
and dnsmasq already forwards .onion to tor's DNSPort; the piece missing
was reachability. dnsmasq listened on loopback only, which docker
discards, so containers never saw it. It now also answers on the docker
bridge, the daemon points there with the clearnet resolver kept as
fallback, and the role's own onion resolves to a container-reachable
backend instead of loopback.

Nextcloud drops its proxy block and the exclusion list; every other role
gets the same behaviour without configuring anything.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 01:13:33 +02:00
kevinveenbirkenbachandClaude Opus 5 d85603782f fix(web-app-nextcloud): keep appstore downloads off the tor proxy
The role routes all outgoing traffic through tor so it can reach
keycloak on its onion address, but the test tor network has no exit, so
every app download ran into a two-minute timeout and the deploy aborted
on the mandatory oidc_login plugin. All 20 required apps resolve to
githubusercontent hosts after redirects, so excluding those plus the
appstore API is enough; onion traffic still goes through the proxy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 23:51:17 +02:00
kevinveenbirkenbachandClaude Fable 5 62a46e472f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Resolves the runner Dockerfile to the lint-standard Acquire::Retries=3
and keeps the collabora TODO wording; hardens the merged tree for the
stricter lints (apt retries in the tor Dockerfile, shell-or-true
markers, comment cleanup).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 16:27:25 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0b2d081daf fix(nextcloud): deliver extra config via a native additional-config file
The infinito config loader was appended to the occ-managed config.php with a shell printf while the image entrypoint was still in its post-install write phase; the entrypoint read config.php before the append and wrote it back after, clobbering the loader, so /var/www/infinito extras (oidc_login_provider_url among them) never reached Nextcloud and /apps/oidc_login/oidc answered 500 in the compose e2e job. Install the loader as config/zzz-infinito.config.php after occ reports installed instead: Nextcloud merges config/*.config.php on every read, occ never rewrites those files, and the entrypoint rsync excludes /config/, so no clobber path remains. A file mount was rejected because a non-empty config/ suppresses the entrypoint's base-config copy on first install. The loader now declares $CONFIG directly (additional config files load after unset($CONFIG)), cron gets the /var/www/infinito bind it silently lacked, and the dead include-path vars and host render go away.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 23:22:48 +02:00
kevinveenbirkenbachandClaude Opus 4.8 b44fea8470 fix(web-app-nextcloud): provision the mattermost OAuth coupling on swarm
The Mattermost <-> Nextcloud OAuth provisioning ran its container probe and
mmctl steps on the manager, but in swarm the mattermost task runs on a
worker, so the probe found no container and the whole coupling was skipped.
The connect button then stayed disabled and the integration e2e timed out.

Resolve the mattermost hosting node and node-local container id via
resolve_host_cid and delegate every mmctl step, and the OAuth-app
provisioning, to that node, mirroring the pattern web-app-mattermost's own
role uses. The OAuth-app step keeps running bash on the host because the
image has no shell, but now on the mattermost node so its localhost URL
resolves. Drop the now-unused running probe and the
NEXTCLOUD_MATTERMOST_CONTAINER_ADDRESS var.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 03:01:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 a51be6f66b fix(web-app-nextcloud): resolve OAuth partner containers via container_address
The mattermost and openproject integration probes plus the openproject provisioning execs addressed the partner by its bare compose container name. Under swarm the running task name differs, so container inspect found nothing, the partner-running probe returned false, and the bidirectional OAuth provisioning was skipped. Route them through the container_address lookup (bare name in compose, resolve-container-id subshell in swarm) so the probe resolves and provisioning fires in both deploy modes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 21:23:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 289a7bdba3 revert(web-app-nextcloud): drop the delegated mattermost bash -s provisioning
Reverts b9a5aa816. It rewrote the Mattermost OAuth provisioning to
'container exec ... bash -s', but the mattermost-team-edition:11.9.0 image
has no shell at all (verified: 'stat /bin/sh: no such file'), so the task
fails rc127 in BOTH swarm and compose (run 29715516546, web-app-nextcloud 3,
jobs 88269803451 + 88270969917). Restores the previous provisioning form.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 12:21:23 +02:00
kevinveenbirkenbach b9a5aa8167 fix(web-app-nextcloud): delegate mattermost oauth 2026-07-20 05:45:26 +02:00
kevinveenbirkenbachandClaude Fable 5 d88f272f9f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 15:31:08 +02:00
kevinveenbirkenbachandClaude Fable 5 898b6a2c4e feat(oidc): resolve OIDC lazily so the issuer family-aligns per consumer
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 20:39:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f78ec52385 fix(nextcloud): stop the config copy-up race and the root-owned config bind
The custom Dockerfile ran mkdir -p /var/www/html/config, which broke the
stock image's empty-/var/www/html invariant: Docker then had image content
at the shared nextcloud_data mount point and did a non-idempotent copy-up
os.Mkdir(config) per container. compose up --force-recreate starts
nextcloud and cron concurrently, so both raced that mkdir and the loser
crashed with 'failed to mkdir .../_data/config: file exists'. Reproduced
char-for-char locally (31/60 with the infinito bind, 7/80 without, so the
bind is only an aggravator); removing the image content gives 0/80.

Drop the Dockerfile mkdir so /var/www/html stays empty and the stock
entrypoint (reached via nextcloud-init-gate.sh -> /entrypoint.sh) creates
config/ as www-data with no copy-up. The mkdir only existed to pre-seed
config as www-data because binding infinito at /var/www/html/config/infinito
forced Docker to create config/ root:root; move that bind (and its
NEXTCLOUD_DOCKER_CONF_ADD_PATH loader path) out to /var/www/infinito so
config/ is owned by www-data and config.php stays writable. Keeps type:
bind, so the swarm config-object fix from 699008a16 is not reverted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 18:28:10 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6dcc564911 fix(nextcloud): make the Zammad OAuth coupling swarm-capable
The partner probe and both provisioning execs addressed the bare
compose container name zammad-railsserver, which does not resolve in
swarm; the probe silently skipped the whole coupling and the connect
control never rendered. Route them through the container_address
resolver with the shell module (unquoted, subshell expansion), gate the
provision include on IS_STACK_HOST, and retry the rails provision to
absorb Zammad convergence lag. Compose behavior is unchanged: the
lookup returns the bare name there.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 04:25:51 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d5aa44715b fix(web-app-nextcloud): route OIDC through Tor SOCKS on an onion node
On an onion node the OIDC issuer (Keycloak) is a .onion; libcurl (oidc_login's Guzzle client) refuses to resolve .onion per RFC 7686, so the server-side OIDC backchannel (discovery/token/jwks) fails. Set Nextcloud's system proxy to socks5h://host.docker.internal:<tor-socks> and expose the host Tor SOCKS to the app container via extra_hosts, both gated on SSO being enabled and the Keycloak domain being an onion. No-op on clearnet nodes.

Hoist the repeated services.sso.enabled and Keycloak-domain lookups into NEXTCLOUD_SSO_ENABLED and NEXTCLOUD_OIDC_HOST constants (mirroring MATRIX_OIDC_HOST) and use them across the role.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 10:50:09 +02:00
kevinveenbirkenbachandClaude Fable 5 4a23e0cb09 feat(services): declare locally-built images via services.<key>.custom
The custom flag moves from 33 hardcoded template kwargs into the merged
service config: 'custom: true' names the shared <entity>_custom build,
a string names a second distinct '<string>_custom' image inside one app
(nextcloud whiteboard). The image lookup reads the declaration as a
fallback, so config consumers (e.g. backup image matching) resolve the
exact deployed reference from one declaration point. nextcloud cron now
runs the nextcloud_custom build like the app service instead of the
upstream image.

A new services lint couples the declaration to reality in both
directions: custom requires a role Dockerfile, and a role Dockerfile
requires a custom declaration. The eight legacy roles deploying local
builds under upstream tags or ad-hoc image= overrides are ratcheted via
'# nocheck: dockerfile-custom'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 19:48:53 +02:00
kevinveenbirkenbachandClaude Fable 5 cf32840586 feat(lint): require *_PORT vars to resolve via lookup, migrate literals
New lint rejects literal port numbers in roles/*/vars/main.yml: the port
belongs in meta/services.yml (services.<svc>.ports.<scope>.<name>) where
band governance and stack rendering read it; vars resolve it via
lookup('config', ...). Migrated the 16 remaining literals across
elasticsearch, registry-docker, erpnext, flowise, funkwhale, magento,
moodle, nextcloud, penpot and shopware.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 03:15:53 +02:00
kevinveenbirkenbachandClaude Fable 5 f19a638eda fix(nextcloud): swarm-capable occ tasks and replica init gate
The container_address lookup emits $(...) shell syntax in swarm and
documents that callers must use the shell module; the 22 occ tasks
still used command:, so the resolution reached docker literally and
every plugin/LDAP/integration task died on all swarm variants. Convert
them to shell: (args stay single-quoted or [A-Za-z0-9] slugs, so
compose semantics are unchanged).

NFS mounts use local_lock=flock, which makes the stock entrypoint's
init flock node-local: concurrent replicas each won "exclusive" locks
and shredded the rsync install (occ 255, converge livelock). A small
init gate now fronts the entrypoint: only swarm task slot 1 runs the
init, other slots wait for version parity plus config.php; without
TASK_SLOT (compose) it degenerates to a plain exec. The gate ships as
files/nextcloud-init-gate.sh, copied into the build context on the
stack host and referenced through the NEXTCLOUD_INIT_GATE_FILE SPOT.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 02:37:28 +02:00
kevinveenbirkenbachandClaude Fable 5 b73706c022 fix(web-app-nextcloud): converge on swarm and run multi-node fpm safely
The fpm/cron occ-status healthcheck rendered a 30s start_period, far shorter
than the (image-serialized) first-boot init, so swarm killed every task with
exit 137 mid-init and the stack never converged. Give both healthchecks a 10m
start_period and raise swarm_stack_converge_retries to 600.

The nextcloud image already serializes first boot across replicas (its own
init lock), so unpin fpm from replicas: 1 and let it run on every node; keep
cron pinned as a genuine singleton. For correct multi-instance runtime, follow
the Nextcloud caching recommendation: memcache.local => APCu in both modes,
memcache.distributed and memcache.locking => Redis in swarm, backed by an
explicit redis connection array (which also fixes the RedisException at occ
upgrade, where OC\Memcache\Redis previously fell back to 127.0.0.1).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 21:05:16 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f2af680a2f test(lint): require container_port to read the internal-port config lookup
Add a lint that rejects hardcoded container_port literals and the
host-published ports.local port; container_port is the port the container
listens on, so it must resolve from
lookup('config', application_id, 'services.<svc>.ports.internal.<name>')
or a constant built that way. Bring 42 roles into compliance: move each
literal into services.<svc>.ports.internal.<name> and reference it via the
lookup, using entity_name/service_name where they equal the service key.
Values are preserved (mattermost/prometheus pointed at ports.local with the
same value as internal). Drop the dead container_port from fider and
prometheus, and the narrative comments the edits pulled into the comment
lint's changed-file scope.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 00:20:10 +02:00