Three hardenings of the mkdir boot lock: reclaim a stale lock (>=1800s dir mtime, beyond the orchestrator kill ceiling of start_period 15m plus retries) so a leader killed mid-install no longer deadlocks every future boot; check staleness BEFORE the install-flag break so a lock leaked after the flag was written (kill during cache:warmup) also heals instead of forcing the loser path forever; release the lock on TERM/INT. After install/cache (which run as root) chown the Symfony and legacy cache trees to www-data: apache regenerates wiped legacy language caches lazily and cannot write root-owned dirs, which served permanent 500s (live-verified: chown turned GET / from 500 to 200).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Workers failed 'docker build: no Dockerfile': the build context, compose file, .env and runner binary were staged on the stack host only while every node deploys its own node-local runner stack. Build and push the image once on the manager (RUNNER_IMAGE_REF via the swarm registry, 04_image.yml) and let workers pull it; render compose.yml (template with lstrip_blocks, matching the replaced stack_host_template action), .env and the extracted run.sh on every node. Add a node_local kwarg to the compose_networks/container_networks/compose_restart/compose_only lookups so this node-local compose stack renders a bridge default network (workers cannot create overlays), keeps restart: unless-stopped (ephemeral runners died after one job) and keeps container_name under swarm; default is unchanged for every other role. Treat build/install.stamp as stale when the venv is missing and exclude ./build from the nested-src copy, closing the nested-e2e ruamel failure class. Regression tests for the node_local render paths; the network include tests now accept lookup kwargs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- tasks/02_bootstrap.yml: set retries=4 delay=30 on owner probe to
stay within n8n's 5-attempt/5-minute rate limit window
- files/playwright: remove test-oidc-login.js - redundant with
test-login-administrator.js which already covers the full OIDC flow
- rebase onto latest main
- tasks/01_core.yml: add when: N8N_SSO_ENABLED | bool to hooks.js copy task
- templates/compose.yml.j2: wrap hooks.js volume mount in SSO conditional
Per Kevin's review - hooks.js should only be staged and mounted
when SSO is actually enabled, adding extra redundancy.
Remove relations: ['role'] from UserRepo.findOne() - in 1.95.3
user.role is a plain string column, not a relation. The relations
array caused EntityPropertyNotFoundError which produced 500 on
every SSO login attempt. Also removed dead RoleRepo/dbCollections.Role
references dropped in 1.95.3's DropRoleTable migration.
Verified: biber@infinito.example now auto-provisioned as global:member
after Keycloak SSO. All 5 Playwright tests pass locally.
- Add bond:1 to all 9 enabled+shared services in web-app-n8n/meta/services.yml
- Fix web-app-n8n/meta/variants.yml V3 postgres shared:false → shared:true (only V2 is standalone)
- Remove dead PIHOLE_SSO_PROXY_CA_CONTAINER_PATH var from web-app-pihole/vars/main.yml
- Add nocheck:url marker for dead link in web-app-mastodon/README.md
- Fix LDAP sign-in button selector in _shared.js (button[type=submit] → getByRole)
n8n is SSO-gated and redirects unauthenticated requests cross-origin to
Keycloak, causing ERR_ABORTED (405) in the headless Chromium probe. Adding
405 opts out of the probe, matching the web-app-prometheus pattern.
1. meta/info.yml: add homepage and video fields.
2. meta/server.yml: add status_codes.default [200, 302] so the CSP/webserver
health check accepts the oauth2-proxy redirect response.
3. Move self-signed CA trust for oauth2-proxy into the shared
oauth2-proxy-keycloak.cfg.j2 template (provider_ca_files, conditional on
TLS_MODE=self_signed). This fixes the root cause for all oauth2-proxy roles
(including web-app-pihole) without role-local lineinfile hacks.
4. Drop compose_handlers_flush: false + explicit meta: flush_handlers from
both web-app-n8n and web-app-pihole — now that no post-render file surgery
is needed, compose_handlers_flush: true lets sys-stk-backend flush normally.
5. Remove the now-unused N8N_SSO_PROXY_CA_CONTAINER_PATH and
PIHOLE_SSO_PROXY_CA_CONTAINER_PATH role variables.
6. README: document the V2 (no-auth) native administrator login flow.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Community edition with edge oauth2-proxy SSO (Decision #1b)
- PostgreSQL backend via svc-db-postgres
- Three variants: V1 sso+ldap, V2 no-auth, V3 ldap-only
- Bootstrap owner account via /api/v1/owner/setup
- LDAP config via direct PostgreSQL upsert (CE API limitation)
- Playwright: baseline, CSP, OIDC, administrator and biber personas
- All 5 active tests pass, 2 skipped correctly for this variant
make autoformat output across role meta files, a task file and 19_tor.yml (single->double quotes on mode values, Jinja line reflow). No semantic changes. Kept separate from the onion CSP / oauth2-proxy fixes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The egress-router trans (9040) and dns (9053) ports had no PORT_BANDS entry and failed the port-band-membership lint. Add the local.trans and local.dns bands next to socks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Over Tor a page served on an .onion still advertised a clearnet-only CSP and oauth2-proxy config, so onion origins were blocked (dashboard could not frame auth.<onion>; oauth2-proxy rejected the redirect back to the onion app). build_csp_header now mirrors the clearnet primary to the node onion per token; the keycloak oauth2-proxy config derives whitelist_domains and cookie_domains from the app resolved domains. Exclusive (onion-only) apps get onion tokens only, primary apps get both, non-tor apps stay clearnet.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove the inline/trailing comments added across the Tor egress-router, onion-domain, mailu-notls and vault round-trip changes, per the repo no-comments convention. Comment-only: no code behaviour changes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Indentation/whitespace normalization from make autoformat. No semantic changes (git diff --ignore-all-space empty for every file). Kept separate from the onion CSP / oauth2-proxy fixes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The upstream-capture copy re-read /etc/resolv.conf on every run and filtered out 127.0.0.1. Once the router has pointed the resolver at dnsmasq (127.0.0.1), that filter yields an empty list, so the copy failed with "src (or content) is required" and aborted the deploy on re-runs. Guard the resolver read and the persist copy on the persisted upstreams file being absent (via a stat), so the capture runs only on first setup and later runs reuse the persisted upstreams.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The email lookups tls default keyed on DOMAIN_PRIMARY, which can stay clearnet while Mailus canonical domain is turned into a .onion by the Tor onion injection. TLS therefore stayed on, but Mailu has no public certificate for the .onion (LE cannot sign it) and disabled TLS, so the SMTP handshake to the local front failed and the msmtp health check errored. Key the .onion detection on the resolved mail host instead (host is now resolved before tls in RESOLUTION_ORDER); an onion mail host relays in plaintext, which Tor already authenticates and encrypts end-to-end.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Indentation/whitespace normalization from make autoformat across role meta files. No semantic changes (git diff --ignore-all-space is empty for every file). Kept separate from the mailu notls-over-onion and Tor-router idempotency fixes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Route any outbound TCP to a *.onion address through Tor, protocol-agnostically (HTTP, SMTP, IMAP, ...), while clearnet traffic resolves normally and goes direct. Implemented on the target host and gated on TOR_EGRESS_ROUTER_ENABLED: torrc gains AutomapHostsOnResolve + AutomapHostsSuffixes .onion + VirtualAddrNetworkIPv4 + TransPort/DNSPort; a dnsmasq drop-in split-resolves *.onion via the DNSPort (clearnet upstreams embedded inline, since an AppArmor-confined dnsmasq may only read /etc/dnsmasq.d/**) and the host resolver points at it; an iptables nat OUTPUT rule redirects the automapped virtual range to TransPort. Verified in-container: http://<app>.<node-onion>/ returns 200 with no SOCKS wrapper, clearnet DNS + HTTPS stay direct.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Indentation/whitespace normalization emitted by make autoformat across role meta/services.yml, meta/variants.yml and a task file. No semantic changes (verified: git diff --ignore-all-space is empty for every file). Kept separate from the Tor egress-router feature.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The handler imported utils.cache.yaml (PyYAML) at module load, which broke python -m cli.meta.env in the stdlib-only .env bootstrap on fresh distro images (manjaro CI failed with ModuleNotFoundError: No module named yaml). Import lazily inside apply() and skip when PyYAML is absent; the SOCKS port is re-set on any later build that has PyYAML.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The !vault YAML constructor returned a plain str, so round-tripping a host_vars
file through utils.cache.yaml (e.g. apply_tor_node_onion writing
services.tor.node) re-emitted every credential as an untagged quoted string.
Ansible then never decrypted them and LDAP/DB admin binds failed with result 49.
Return a VaultScalar so SafeDumper re-emits the !vault literal block; add a
regression test asserting apply_tor_node_onion keeps existing vault credentials
tagged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Every task in 02_extensions.yml repeated delegate_to: MEDIAWIKI_HOST_NODE. Move it to the include_tasks apply: block and drop the per-task lines, satisfying the include-uniform-apply-keyword lint. Functionally identical (apply propagates the keyword to every included task).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
espocrm stays at replicas:1: making it multi-replica needs a control-flow rewrite of its entrypoint (run the image init under a flock via /bin/true so it returns, then own the server launch) plus relocating PHP sessions to the NFS-shared volume. That restructures a currently-working, compose-green deploy and is coupled to the pinned 9.3.10 image's exec behavior, so it could not be verified to the required confidence without a live swarm loop. Add the single-replica nocheck marker and a TODO capturing the path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
cache and messenger already use the mounted REDIS_URL, but the framework.yaml omitted a session handler, so sessions were node-local files and logins/carts broke behind the swarm VIP. Set framework.session.handler_id to the existing REDIS_URL so sessions are shared, and drop the entity replicas:1 pin so web and worker default to the topology count. Re-pin the genuine singletons: init (one-shot install/migrate), scheduler (double-dispatches on >1 poller) and opensearch (discovery.type=single-node). Compose is byte-identical (replicas ignored). Confidence ~94% (session cross-replica reasoned, not live-verified).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
suitecrm_data is NFS-shared and the datastore is external MariaDB, so replicas share /var/www/html, uploads and cache. Two hazards remained: N replicas racing the first-boot suitecrm:app:install, and node-local PHP file sessions. Wrap the installer + cache pass in a flock on the shared volume so exactly one replica installs while the others block then skip, and pin session.save_path to /var/www/html/tmp on the shared volume so sessions are cluster-wide. flock is a no-op with a single replica, so compose is unchanged. Confidence ~95%.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sessions are redis, the filestore /var/lib/odoo is on the NFS-shared odoo_data volume, and cron serializes on Postgres advisory locks, so the only multi-replica hazard was the entrypoint's check-then-init TOCTOU: two replicas booting against an empty DB could both run odoo --init. Replace _schema_present with _guarded_init, which holds a database-scoped pg_advisory_lock across the schema check and the init in one connection (auto-released if the process dies). Uncontended (compose/single-node) it is a no-op, so behavior is unchanged there. Confidence ~95%.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
SESSION/CACHE/QUEUE drivers are redis, the first-boot migration runs once via Ansible with migrate --force --isolated (redis lock), and pixelfed_data (passport keys + init marker) is on the NFS-shared named volume, so the web service is safe at N replicas. Drop the entity replicas:1 pin but keep the worker at replicas:1 because Horizon runs the Laravel scheduler and >1 would double-dispatch scheduled jobs. Confidence ~95%.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
