The net refactor (366dc7b70/cb27ff375) migrated every shared-net provider (postgres/mariadb/openldap/ollama/openresty/keycloak) to the overlay registry but missed seaweedfs and minio, so an object-store consumer like baserow never joined the provider's docker network and its S3 client hit ECONNREFUSED on seaweedfs:8333 (the intended direct data-plane path).
Add networks.overlay to both engines so consumers attach, and complete the multi-service shared-net-provider contract: (1) render.py emits a bare auto-IPAM project default for an own shared_net provider (no name, no subnet) so the entity-named external net created by network/create.yml is not re-requested -- this caused 'invalid pool request: Pool overlaps' on the first attempt. (2) a provider_self_alias flag lets the provider's sidecars (proxy/sso-proxy) JOIN the shared net WITHOUT publishing the <entity> alias, so the name resolves only to the main container and baserow's :8333 never round-robins onto a port-less sidecar. (3) seaweedfs/minio 01_core pre-create the external <entity> net via network/create.yml (mirroring web-app-prometheus) so compose adopts it as the project default instead of building <entity>_default.
Verified: baserow compose e2e test-seaweedfs.js passes (upload stored in the bucket, 3.9s); new TestOwnSharedNetProvider locks the contract; full make test green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
With NFS shared storage the DB dependency runs as its own single-replica swarm service (<dep>_<dep>) instead of a stateful compose container, so 07_wait_converge inspects it via 'docker service ls/ps' (N/N replicas + Running) rather than 'docker inspect' on a container name.
09_seed_content now drains a WORKER node (never the manager, which pins the registry/DB/edge-proxy a rescheduled task depends on) and probes with 'curl -sS' instead of '-fsS': many app roots legitimately 404 (UI/API live elsewhere), so the reschedule check asserts the HTTP server responds, not that / is 2xx. 11_assert_state applies the same -sS reachability semantics.
Verified against swarm #20 (green): db(postgres) 1/1 Running -> converged; drain wrk-01 -> reschedule wrk-02; baserow reachable after reschedule.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Under `act --bind` the working tree is bind-mounted into the runner, so running actions/checkout would clobber the live tree being iterated on. Guard the checkout step with `if: ${{ \!env.ACT }}` so real GitHub runs still check out while local act runs reuse the bind mount.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The all-in-one image runs supervisord with the startup migrate and celery-beat concurrently. With MIGRATE_ON_STARTUP defaulting true, the image's startup migrate races celery-beat (which crash-loops on the un-migrated schema) and deadlocks on the locked_migrate advisory lock; the schema never completes and the external locked_migrate task then blocks forever on that same lock. Set MIGRATE_ON_STARTUP=false so the external task is the sole migrator, and add timeout: 600 to the migrate and bootstrap exec tasks so a stuck exec fails cleanly instead of hanging (retries stay 60; timeout bounds the whole task).
Verified in dev container (deploy-4): locked_migrate now completes in ~6s (was a 30min+ hang); deploy reached ok=1879 end-to-end.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to 37583e807. apply: on include_tasks wraps the included tasks in a Block, so it accepts only Block-valid keywords. The earlier broad hoist wrongly moved leaf-only keywords (changed_when, delay, async, poll) into apply:, which Ansible rejects at parse time ("'<kw>' is not a valid attribute for a Block") and would have broken every compose deploy at the post_deploy include.
Revert changed_when/delay (post_deploy.yml + its 3 include sites), changed_when (bigbluebutton, moodle x2), and async/poll (sys-svc-dns) back onto the per-task lines; the include sites keep only the Block-valid apply: keys (delegate_to, run_once) or revert to the shorthand include. test_include_uniform_apply_keyword now blacklists the leaf-only keywords (async/changed_when/delay/failed_when/poll/retries/until) so only Block-valid keywords are flagged for hoisting.
Verified: no apply: block contains a leaf-only keyword; lint test green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two compose-mode-only gaps that blocked compose deploys while swarm was unaffected.
svc-prx-openresty: the compose restart handler did 'openresty -s reload', which re-reads the file bind-mount but keeps the stale pinned inode after the real nginx.conf is rendered atomically (write-temp + rename). openresty stayed on the bootstrap 'listen 8080' config and the :80 default-vhost assert got connection refused. Force-recreate the container (like swarm's 'service update --force') so the bind remounts the live inode, plus a post-flight 'openresty -t'.
web-app-keycloak: KEYCLOAK_LOCAL_CID was resolved only in the swarm branch (via the swarm service label), so in compose it stayed undefined and the container health check fatalled. Set it from the compose container address when DEPLOYMENT_MODE \!= 'swarm'.
Both changes are gated to the compose path; the swarm deploy is untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add tests/lint/ansible/tasks/test_include_uniform_apply_keyword.py: when a file pulled in via a dynamic include_tasks carries the same hoistable task keyword on every top-level task, that keyword should be set once via apply: on the include site instead of being repeated per task. The rule is blacklist-based: it flags any uniform Ansible task keyword except the per-task-binding ones (name, register, args, vars, when, loop), and never flags the module itself.
Hoist every existing occurrence to apply: across the affected roles (delegate_to, run_once, become, notify, no_log, changed_when, delay, async, poll), dropping the per-task duplication. Behaviour is identical because apply: propagates each keyword to every included task; no_log masking via apply: was empirically verified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The deploy play runs with any_errors_fatal. The old block/rescue latched the play-abort flag on the expected first-deploy permanent-admin login failure, exiting 2 despite failed=0/rescued=1. Replace it with a failed_when:false probe plus a kc_login_perm.rc-gated include_tasks of the extracted 04_login_bootstrap_admin.yml, so no task ever fails and the bootstrap runs only when the permanent admin is absent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
stack_host_copy is the copy: analogue of stack_host_template: it folds the IS_STACK_HOST gate into the module so callers do not silently fail on hosts whose destination tree is owned only by the stack host. In swarm an app's compose / build-context directory (/opt/compose/<app>) exists only on the manager, so a bare copy: into directories.instance also ran on workers and aborted the play with 'Destination directory does not exist'.
Migrate the six compose-dir copy tasks (baserow, jenkins, bookwyrm, postmarks, yourls, openldap) to stack_host_copy and drop the now-redundant when: IS_STACK_HOST. A new drift-guard lint test (test_stack_host_copy_migration) flags any future bare copy: into the compose dir, and a unit test covers the wrapper.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Swarm runs baserow with one replica per node, each taking the shared advisory lock via locked_migrate on startup. The post-deploy ansible task ran an unlocked migrate that raced the replicas and produced DuplicateTable. Switch it to locked_migrate so it serializes on the same lock; this is a no-op for a single compose instance.
The trusted-header SSO build-context staging copied files into baserow's compose directory, which exists only on the stack host. Ungated, it also ran on workers and aborted the play. Gate it with IS_STACK_HOST so the image is staged and built only on the manager; workers pull the built image from the registry. Compose stays unchanged since IS_STACK_HOST is always true on a single host.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On a swarm node, failed tasks self-heal (swarm reschedules them) and their exited replicas linger in docker ps -a; the exited-container check flagged those and exited 2, failing the deploy health gate even though every service had converged. Skip the exited check on swarm and rely on the existing service-convergence check; off-swarm behavior is unchanged. Validated against the live cluster that failed the previous deploy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mark the erpnext primary entry enabled+shared so other roles can declare it as a dependency (erpnext: {enabled, shared}); the service registry now resolves it to web-app-erpnext, matching the web-app-matomo provider pattern.
Drop svc-db-mariadb from run_after — it crossed deploy types (server -> universal), which the service-loader rejects once the entry becomes a provider. The MariaDB dependency and ordering are already handled by the mariadb: shared service flag.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The gated filer/master branch defaulted the proxy upstream to the entity, so in swarm it resolved to seaweedfs:8333 (the S3 API) instead of the admin oauth2-proxy. Set proxy_service_key=sso-proxy + proxy_internal_port (keycloak's services.sso_proxy.ports.internal.http) so swarm routes to seaweedfs_sso-proxy:4180. html.conf.j2 gains an optional internal_port kwarg (default empty = no-op for existing callers; compose ignores it and keeps local_port).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
web-svc-logout has no surface (personas blocked), so its only check is an anonymous GET of each logout-target's canonical URL asserting the HTML contains logout.js. For an oauth2-proxy-gated app that GET 302-redirects to the Keycloak login page, which cannot carry the app's injected logout.js, so the assertion was a false negative. Mark each target with is_proxy_gated (via the sso lookup) in playwright.env.j2 and test.skip gated targets in the spec; their logout.js is verified by that app's own authenticated persona e2e.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mirror compose-playwright for the act-swarm DinD cluster: copy the working-tree's modified files into the node's frozen bootstrap copy (the nodes are not bind-mounted), then rerun via rerun-spec.sh. rerun-spec.sh gains an opt-in TEST_E2E_PLAYWRIGHT_NETWORK_HOST that switches it to --network host (mirroring the deploy-time swarm branch in 02_run_one.yml) so cross-app canonical domains are reachable on the node; the compose path is unchanged. Document the all-green-before-redeploy gate in playwright.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The server args were duplicated across the standalone provider stack (compose.yml.j2) and the local sidecar (service.yml.j2), so the -ip fix had to be applied in two places. They now live in one filter (plugins/filter/seaweedfs.py); both templates keep command: inline and render the args via it, appending -s3.config only when the caller passes the mounted s3.json path (the sidecar omits it). Adds a unit test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The all-in-one server auto-detected the swarm ingress IP as its advertised leader address, which is unreachable for gRPC from inside the container, so weed shell (it follows the leader's advertised address) blocked forever on WaitUntilConnected during bucket provisioning. Advertise localhost instead; -ip.bind stays 0.0.0.0 so the S3 API remains reachable cross-container. Verified on swarm (live cluster) and compose (single-bridge networking).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A hung NFS server refuses 'docker rm -f' and keeps its lab-network endpoint, blocking both the network removal and the fixed container name on the next run. Kill + force-disconnect from the lab network + remove each node before removing the network, so a wedged DinD NFS container no longer leaves the cluster un-redeployable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codifies the swarm/act iteration discipline: validate a candidate fix on the live zombie cluster before triggering another tens-of-minutes rebuild; no commits during an active iteration (commit only after green); use --no-verify if a mid-iteration commit is unavoidable since the pre-commit hook stashes working-tree files a live deploy is reading; make autoformat/make test are safe any time.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The List/Create bucket tasks run 'weed shell' over container exec; weed shell can stall on stdin/master, and without a timeout that hung the whole consumer-provisioning step indefinitely. Add timeout: 60 per attempt plus retries: 3 / delay: 10 (until succeeded) so a stall fails fast and a transient one recovers.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
seaweedfs (new from the objstore merge) lacked the metadata the swarm deploy path needs:
- services.seaweedfs.ports.internal.http: 8333 so the front-proxy proxy_pass SPOT resolves the S3 upstream (seaweedfs:8333) in swarm instead of erroring on a missing internal port.
- default_placement: manager so the single-replica stateful provider stays on the stack host, where its container-exec S3 health check and weed-shell bucket provisioning resolve the local container (mirrors the svc-db-* providers).
- s3.json mounted at mode 0444 (type: config) so the seaweedfs process can read the S3 identities config; 0440 left it unreadable and crashlooped the S3 API.
Verified by the swarm deploy reaching ok=1889 (full stack up).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pure formatting: autoformat line-wraps the long services.discourse.local_login_enabled Jinja expression. No behavior change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
act auto-loads the repo .env (INFINITO_APP_DISCOVERY_RUNNER=docker), which overrode the swarm workflow env: host. The discover job then ran 'docker compose exec infinito' (not running in the act runner), emitted an empty matrix, and aborted the whole swarm run before any deploy. Pin host via ACT_ENV (act --env outranks the auto-loaded --env-file) so discovery uses the venv python. Local act harness only; real GitHub Actions does not auto-load .env.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stay on the Docker Hub image and move to the 4.34 tag. The ghcr.io/seaweedfs/seaweedfs path publishes only the Helm chart (OCI artifact, application/vnd.cncf.helm.config.v1+json), not a runnable container image, and Docker Hub has no 4.34.0 tag — so 4.34 on chrislusf/seaweedfs is the correct runnable bump.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document the unified addon syntax (req-026) and drop the non-existent '# nocheck: playwright-addon-spec' escape hatch so the docs match the no-exemption lint. Exclude desktop desk-* roles from the per-addon Playwright spec requirement — they drive no web surface a spec could exercise — and harden the lint to FORBID spec files under desk-* roles; remove the 8 desk-chromium/desk-firefox/desk-gnome addon specs accordingly. Web-facing roles still require a spec per addon with no exemption.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The matrix-mdad runner's Type=oneshot bootstrap could run before the venv python runtime settled in the privileged systemd DiD container, hit a transient 'ModuleNotFoundError: _posixsubprocess', and die permanently with no retry (oneshot units never restart) — leaving Synapse undeployed and the host probe failing indefinitely. Add a readiness guard that retries importing _posixsubprocess/subprocess/ssl/ctypes (30 x 2s) before running ansible-galaxy/ansible-playbook, aborting cleanly only if the runtime never becomes importable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add tests/lint/ansible/services/test_addons_playwright_spec.py requiring roles/<role>/files/playwright/addons/<id>.spec.js for EVERY addon of EVERY role — no mechanism auto-exempt and no # nocheck escape. Ship the 87 per-addon specs (gated via skipUnlessAddonEnabled so they skip cleanly when the addon is disabled in the current variant), the test-e2e-playwright addon-gating harness (addon-gating.js, playwright.config.js, recursive 02_run_one.yml), the shared odoo helper, and the ADDON_<ID>_ENABLED flags in the six playwright.env.j2 templates. Relax test_env_file_schema to skip Jinja-templated keys in the ADDON loop. Doc updates (suppression.md, requirements/026) follow in a separate commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Relocate the wordpress per-addon task files from tasks/plugins/ to tasks/addons/ (pure renames) and repoint tasks/main.yml, giving every role a uniform tasks/addons/ layout. Remove roles/web-app-xwiki/meta/addons/matomo.yml: matomo is provided as a service via services.matomo, not an Extension-Manager extension, and the org.xwiki.contrib:matomo extension does not exist in the nexus catalog (install plan failed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
github, google and jira nextcloud integrations read client_id/client_secret via lookup('api', '<provider>.client_id'/'client_secret') and enable only when those credentials are present in the API SPOT. Builds on the API SPOT lookup plugin and dict.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Schema-migrate every nextcloud addon to the unified per-file addon syntax (mechanism/source/required/bridges/config); add the non-proprietary integration addons (matrix, mattermost, moodle, suitecrm, zammad) and the local gitlab bridge; wire the bridged service blocks into meta/services.yml and pin every new (service,enabled)/(service,shared) pair in meta/variants.yml for variant coverage; relocate the per-addon user_ldap hook from tasks/plugin/hooks/ to tasks/addons/ for a uniform tasks/addons/ layout, dropping the obsolete vars/plugins README.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace every CLOUDFLARE_API_TOKEN reference (token body, Bearer headers, certbot credentials, assert/fail guards and when: gates) with lookup('api','cloudflare.api_token') across the Cloudflare DNS/cert/redirect roles, and delete the now-orphaned CLOUDFLARE_API_TOKEN default. No code reference to the old variable remains; docs/README re-document the rename. Builds on the API SPOT introduced in the previous commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add group_vars/all/18_api.yml defining the global API mapping (cloudflare, telegram, github, google, jira) and plugins/lookup/api.py — the single sanctioned reader that resolves a dotted 'provider.key' path against it and raises on unknown keys. Routing every proprietary/external credential access through one lookup keeps a single point to change how credentials are sourced. Includes the unit test covering nested resolution, subtree access, error paths, and templar handling.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reconcile main's object-store feature and pre-swarm deploy tasks with the branch's compose/networks/volumes SPOT and swarm lints.
objstore (seaweedfs/minio): standalone compose + embedded service templates migrated to the container_image/container_volumes/container_networks/compose_volumes/compose_networks/depends_on lookups; meta/volumes.yml on the canonical dict schema with s3.json and nginx.conf as type:config; embedded services made swarm-compatible via compose_only/compose_restart.
deploy tasks (erpnext, opentalk, seaweedfs, shopware, snipe-it, wordpress, yourls): container exec routed through the container_address resolver and gated on IS_STACK_HOST, shell module plus stack_host_template, no chdir or compose-verb forms.
Also removed orphaned vars, restored MAILU_INSTANCE_DIR, freed the seaweedfs subnet (192.168.206.0/24), stripped literal quotes from the nextcloud/bookwyrm env templates, and routed main's objstore compose_volumes tests through dir_var_lib.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In swarm the apps are reachable via the node host-mode proxy ports; share the node network namespace so the Playwright container resolves the canonical domains via the node dnsmasq and reaches them, instead of the compose --add-host host.docker.internal path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Realtime apps (websockets/polling) keep the connection busy so page.goto never reaches networkidle; the asset loads the assertion checks are captured during navigation regardless, so swallow the navigation timeout instead of failing the spec.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
roles_with_service returns only consumer roles that are themselves deployed (in group_names) and excludes the service's own provider, so the logout/css/dashboard/matomo/prometheus injection specs no longer target undeployed or surface-less roles in a partial single-app deployment.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
