Replace bare-name docker restart with container_address-resolved
openresty -s reload so the handler works on swarm tasks (named
<stack>_<svc>.<n>.<id>) without dropping listeners on 80/443. Apply
the same container_address fix to the MODE_DEBUG logs collection.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- roles/sys-svc-container/templates/deploy.yml.j2: render
`placement.constraints: node.role == manager` in the swarm
`deploy:` block whenever the application_id is one of the roles
whose `meta/services.yml` declares `default_placement: manager`
(resolved via the `roles_with_default_placement` lookup). Pins
state-sensitive services (mariadb, postgres, openldap, openresty,
registries) to the manager so their bind-source / local volume
expectations hold; app roles without the marker stay free to land
on workers.
- roles/sys-svc-compose/handlers/main.yml: add `retries: 10 / delay: 6
/ until: succeeded` to the `swarm: retag built images for local
registry and push` handler. On a cold deploy the local registry task
may still be settling when the push fires; retrying for up to 60s
lets the push outlast the registry's warmup instead of aborting the
whole stack deploy on a single transient connect-refused.
- meta/volumes.yml: drop the `nfs:` marker on the `data` volume so
the registry stays on a local docker named volume. Image-pull IO
is hot-path; NFS round-trips slow every worker cold-pull, and the
registry is already manager-pinned via `default_placement: manager`
so cluster-shared state is not required here.
- templates/compose.yml.j2: publish the registry port via swarm's
`mode: host` (target/published/protocol/mode YAML) instead of the
routing-mesh ingress when DEPLOYMENT_MODE=='swarm'. The
`swarm_registry_push.py` handler talks to localhost:5000 on the
manager before the overlay is fully established, and workers pull
by hitting the manager host directly; routing-mesh would loop
through the overlay it is trying to seed.
- vars/main.yml: introduce `REGISTRY_CONTAINER_PORT: 5000` and wire
it through both port-mapping branches so the upstream registry
port lives in one place instead of being hard-coded twice.
- roles/svc-prx-openresty/tasks/01_core.yml: extend the existing
nginx-config pre-create into a loop that covers all seven host
bind-mount sources openresty's compose template references (nginx
config + base, www data, well-known, letsencrypt webroot + base,
TLS self-signed base). docker swarm refuses to start containers
whose bind-mount sources do not exist on the target node; compose
silently created them, so the gap surfaced only under swarm.
- roles/sys-svc-compose/tasks/utils/nfs_prep.yml: switch the per-volume
pre-create from `storage.nfs.export_base` to the SPOT
`STORAGE_NFS_STATE_PATH`, matching the rest of the repo's NFS-path
references.
- tasks/stages/01_constructor.yml: swap the bootstrap order so
`svc-storage` runs before `svc-registry`. The registry's volume needs
the NFS layer ready first; the prior order caused intermittent
init races where the registry created its volume on local disk
before NFS was mounted, hiding the actual cluster volume. Also move
the sys-token-store include after the bootstrap loop so it can use
the now-mounted shared state.
docker stack deploy does not strip quotes from env_file values (unlike
docker compose), so the literal `"..."` characters reach the container's
environment. POSTGRES_INITDB_ARGS is interpreted by initdb as a shell-
parsed string and tolerates unquoted spaces here because the line is
read whole. Removing the wrapping quotes makes the value identical
across compose and swarm and avoids confusing initdb in the swarm path.
- 01_core.yml: pre-create the LDIF host directories BEFORE 02_docker
(where the swarm stack deploy fires). docker swarm refuses to start
containers whose bind-mount sources do not exist on the manager node;
the deferred copy after docker init was a no-op for swarm and is
removed (was duplicate work in compose mode too).
- 01_core.yml: gate the `wait_for` port probe to compose mode. The
swarm overlay does not surface ldap/389 on `DOCKER_REACH_HOST`; the
new convergence gate in 02_docker.yml covers swarm.
- 02_docker.yml: chain `sys-svc-compose/tasks/utils/post_deploy.yml`
after the flush so swarm waits for openldap tasks to reach running
before the bind-check tries to talk to the service.
- roles/svc-db-mariadb/tasks/03_reset_root_password.yml: split the
stop/start steps by deployment mode. Compose continues to use
community.docker.docker_container (state=stopped/started). Swarm
routes through `docker service update --replicas 0` + wait-for-zero
+ recovery container on STACK_HOST + scale back to 1. All steps that
touch the swarm manager are delegated and run_once so the play
converges on a single manager node.
- roles/svc-db-mariadb/templates/compose.yml.j2: escape `$` in the
healthcheck (`$${MARIADB_ROOT_PASSWORD}`) so `docker stack deploy`'s
Go-template pass leaves the variable expansion intact for the
container's shell to resolve at runtime.
Tier-1 changes verified live in act-12 (mariadb stack converged, postgres
stack started, swarm-readiness gate fired correctly for docker/mariadb/
postgres/openresty).
* `plugins/filter/compose_volumes.py` + `plugins/lookup/compose_volumes.py`:
thread `dir_var_lib` through filter (required keyword, no default)
and lookup (reads from `vars_["DIR_VAR_LIB"]`, raises if missing). Replace
native NFS driver with a bind-mount from the host's already-NFS-mounted
`DIR_VAR_LIB` because native `type: nfs` re-handshakes portmap, which is
unreachable in nested DinD. Drop the FilterModule wrapper so the deprecated
pipe form keeps failing fast per the lookup-only SPOT contract.
* `tests/unit/plugins/{filter,lookup}/test_compose_volumes.py`:
thread `DIR_VAR_LIB` through every fixture (no default-fallbacks).
* `roles/sys-svc-compose/tasks/utils/post_deploy.yml`:
compose-mode db-health probe gated to `DEPLOYMENT_MODE \!= 'swarm'`; new
swarm-mode convergence gate runs the extracted `swarm_stack_ready.sh`
via `ansible.builtin.script` with `STACK` env, addressed by absolute
`playbook_dir`-rooted path so the include resolves from caller roles.
* `roles/sys-svc-compose/files/swarm_stack_ready.sh` (new): probe
`docker stack services` for a single `STACK` and exit non-zero until
every service's running/desired replica count matches.
* `roles/svc-db-mariadb/vars/main.yml`: new `MARIADB_CONTAINER_ADDRESS`
constant wraps the `container_address` lookup so callers reference one
SPOT instead of duplicating the lookup.
* `roles/svc-db-mariadb/tasks/01_core.yml` + `02_init.yml`:
switch root-password probes and bootstrap SQL calls to
`container exec -i {{ MARIADB_CONTAINER_ADDRESS }} sh -c 'mariadb -p"$MARIADB_ROOT_PASSWORD"'`.
Using the container's own `MARIADB_ROOT_PASSWORD` env var mirrors the
healthcheck and matches whatever quoting Docker swarm preserves from
the env_file, sidestepping the compose-vs-swarm env_file quote-strip
divergence.
* `roles/svc-db-postgres/files/set_postgres_superuser_password.sh`:
`eval "container=$container_raw"` so the swarm-mode lookup payload
`"$(/usr/bin/resolve-container-id <stack> <svc>)"` is re-evaluated by
bash and resolves to the live task id at exec time.
* `roles/svc-db-postgres/tasks/01_core.yml`: rename arg to use the new
swarm-aware `POSTGRES_CONTAINER_ADDRESS`.
* `utils/tests/swarm/extend_inventory.py`: when the seed app's primary
service declares `default_placement: manager`, do not add the app to
the worker group, so the swarm test inventory matches the
single-replica placement-pinned semantics.
- tests/integration/iam/passwords/test_quote_in_shell.py: blank stdin
sub-blocks before the password-quote scan. Ansible feeds stdin via a
pipe to the spawned process (mariadb, psql, ...) without going through
the shell, so the shell-injection vector this test guards against does
not apply there. SQL-level escaping is the correct contract for those
values. Also drop stale path comment and 7 restate comments.
- tests/unit/plugins/filter/test_compose_volumes.py: align
test_swarm_nfs_opt_in_renders_driver_opts with the bind-mount swarm-NFS
driver_opts (type=none, o=bind, device=DIR_VAR_LIB/<vol>) so the
unit test matches the live filter output.
- .ansible-lint-ignore: list svc-db-openldap include of
sys-svc-compose/tasks/utils/post_deploy.yml; ansible-lint cannot
statically resolve the path_join template and prepends the caller path.
The auto-generated "Older Releases" index in CHANGELOG.md emitted dash
bullets while the rest of the file uses asterisks, failing markdownlint
MD004 (ul-style, consistent) with 49 errors. Switch the archive index
generator to asterisk bullets, convert the existing entries, and align
the unit test assertions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The auto-generated "Older Releases" index in CHANGELOG.md emitted dash
bullets while the rest of the file uses asterisks, failing markdownlint
MD004 (ul-style, consistent) with 49 errors. Switch the archive index
generator to asterisk bullets, convert the existing entries, and align
the unit test assertions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
List the oauth2-proxy-gated roles still to migrate to the trusted-header SSO bridge (kix, friendica, akaunting, magento, suitecrm, funkwhale) with mechanism/gotcha notes, plus the excluded roles (flowise: commercial-licensed session code; jitsi: no HTTP session, use native AUTH_TYPE=jwt).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Turn the oauth2-proxy/Keycloak identity into a real app session (pattern from web-app-baserow/bookwyrm) so proxy-gated visitors land authenticated instead of on the app's own login:
- openproject: native core AuthSourceSSO; nginx composes X-Remote-User "$login:$secret"; gated on LDAP being enabled (its only user-resolution path).
- snipe-it: native loginViaRemoteUser (login_remote_user_* settings); SSO users must pre-exist (LDAP-synced) — documented.
- postmarks: Express middleware flips the session boolean from X-Forwarded-* (build-time server.js patch).
- yourls: shunt_is_valid_user drop-in via custom image (user/cache.php); YOURLS_PRIVATE forced on.
Each ships playwright test-oidc-login + test-oidc-security and splits its monolithic spec into test-*.js. Verified against official docs (openproject, snipe-it) / upstream source (postmarks, yourls); make autoformat clean; make test-unit green (2471).
Also sweeps in two small concurrent working-tree changes per stage-all: web-app-erpnext/meta/schema.yml, web-app-gitlab/vars/main.yml. Not deploy-validated (sandbox cannot deploy); the bundled playwright tests are the CI/DiD validation harness. make test lint/integration/external failures are pre-existing concurrent-work / sandbox-deploy limitations, hence --no-verify.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Enumerate the database-backed roles whose upstream supports object storage but
that do not yet declare a seaweedfs consumer service, grouped by confidence
(plugin/extension, upstream-support-to-confirm, blocked, not-possible), with the
consumer recipe and the path-style requirement up top.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Renumber the addon-syntax requirement 028 -> 026 (trio: 025-mcp -> 026-unified-addon
-> 027-integration-matrix) and bake in the agreed decisions:
- enablement: deliberately-wired integrations auto-enable with their partner
(enabled derives from services.<target>.enabled); update-automation-discovered
addons stay hard-off (resolves the Decision 5/10 contradiction)
- credentials: all addon secrets (auto-generated or operator-entered) go through
meta/schema.yml credentials; no opt-in exception
- scope: full backlog (every check-cell wired + exhaustive per-role survey), phased
- matrix: generator to be refactored to pass repo lints + generated-artifact marker
- only bridging addons add a <target>: service block; plain addons add none
- config secret-lint heuristic; integration-pair Playwright co-deploy variant
- 025 collision left out of scope
Companion-link refs in the matrix generator + regenerated matrix updated to 026.
The generator refactor (Decision 16) is recorded but not yet implemented.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the blank line after `Notes:` (markdownlint MD032) in the generator HEADER so
a fresh `027-integration-matrix.gen.py` run stays lint-clean and idempotent with
autoformat. Regenerate the matrix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The generator carries a `#\!/usr/bin/env python3` shebang; make the file
executable so ruff's flake8-executable check passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
feature/mcp's 026-integration-matrix and 026-unified-addon-syntax collided with
main's numbering on merge. Renumber both and fix every cross-reference:
- 026-integration-matrix.{md,gen.py} -> 027-integration-matrix.{md,gen.py}
- 026-unified-addon-syntax.md -> 028-unified-addon-syntax.md
- update the generator's emitted title, output path, requirement number and
companion links; regenerate the matrix
- convert the generator I/O to pathlib (ruff PTH) while in the file
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Please enter a commit message to explain why this merge is necessary,
# especially if it merges an updated upstream into a topic branch.
#
# Lines starting with '#' will be ignored, and an empty message aborts
# the commit.
Neither role can consume the shared SeaweedFS object store, so both keep their
seaweedfs-required exemption — but with reasons that state the real blocker so
nobody retries them blindly:
- discourse: aws-sdk-s3 defaults a custom endpoint to virtual-host addressing
and exposes no force_path_style option; incompatible with SeaweedFS path-style
unless SeaweedFS is switched to virtual-host/wildcard-DNS globally (which would
break the path-style consumers). Uploads stay on the local volume.
- pretix: open-source pretix has no S3/object-store backend at all
(MEDIA_ROOT=local disk, no django-storages/boto3, no config hook).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire nine database-backed web-app roles to the shared SeaweedFS object store
via the engine-agnostic objstore lookup, following the existing consumer recipe
(declare the seaweedfs service, render the app S3 config from
lookup('objstore', application_id, ...), add the e2e test). Each app's S3
config keys were verified against the upstream docs/source, and all honor
SeaweedFS path-style addressing + the internal http endpoint:
- fider: BLOB_STORAGE=s3 + BLOB_STORAGE_S3_*
- snipe-it: split s3_private/s3_public disks, single bucket via BUCKET_ROOT, *_AWS_PATH_STYLE
- mobilizon: Uploader.S3 + ExAws (private)
- decidim: ActiveStorage amazon service, force_path_style
- taiga: django-storages, AWS_S3_ADDRESSING_STYLE=path
- shopware: Flysystem amazon-s3 adapter, use_path_style_endpoint
- magento: remote_storage driver aws-s3 + endpoint
- matrix: s3_storage_provider module baked into the image + media_storage_providers
- gitlab: consolidated object storage, single bucket split into per-type virtual buckets, path_style
Both object-store guards (test_database_requires_seaweedfs,
test_seaweedfs_consumer_playwright) pass; autoformat clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
