resolve_host_cid/wait_converged/guard_instance_dir took a run_host parameter every caller had to pass; STACK_HOST already encodes the correct host in both modes (first swarm manager, degrading to inventory_hostname in compose), so the manager-only queries now delegate to it directly and the parameter plus all caller declarations are gone. Also repairs the swarm guard include: apply/delegate_to are invalid on a HandlerTaskInclude and the previous commit carried duplicate run_once/delegate_to keys, which failed the ansible syntax-check. The keycloak handler drops its masking default; the 02b conditional collapses into the SPOT.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Without LDAP the fallback only seeded the administrator user and no RBAC groups, so oauth2-proxy's allowed_groups check 403'd every group-gated app (prometheus/seaweedfs, matomo/css downstream) and non-admin logins hit user_not_found (biber). 08_users_realm now seeds every non-reserved user with a usable password (parity with the LDAP path, which federates the same set); new 08b_rbac_groups_realm materialises the /roles/<app>/<role> group tree + memberships directly in the realm: build_realm_rbac_groups mirrors build_ldap_role_entries and restricts members to the users 08 actually seeds (by username attribute); ensure_group_path_members.sh walks the path via kcadm (children endpoint, exact user lookup, idempotent membership PUT) and hard-fails on any missed group or membership instead of masking. Both routines are gated to KEYCLOAK_LDAP_ENABLED false; LDAP-enabled deploys are untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Role README Credits now name galaxy_info.author (plain bold or wrapped
in a Markdown link), matched byte-for-byte against the role's own
meta/main.yml, instead of a hard-coded maintainer name. Regenerate all
245 role README Credits blocks to the new 'Implemented by ...' format.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
services.ldap.enabled/shared used host-scoped 'in group_names', which the process-wide applications cache poisons across hosts (a worker seeds false, the keycloak play reads it, LDAP federation is deleted and biber can no longer log in). Key on inventory-wide groups[] instead, which is identical on every host.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
autoformat reformats (settings sort, complexity/model.py, test_database_query.py, keycloak vars wrap); shorten akaunting swarm task name to <=80 chars; delete narration comments from keycloak vars; remove 'invariant' from the comment linter's valid marker set. make autoformat + make test verified green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Swarm: KC_CACHE=ispn + KC_CACHE_STACK=jdbc-ping so replicas share an Infinispan session/realm cache via JDBC_PING2 over the existing Postgres (no multicast, built-in + auto-table in 26.6.3). Compose: KC_CACHE=local (single node). Realm-import scale-back uses KEYCLOAK_REPLICAS/_WANT constants; DB network attached only when the DB is shared (KEYCLOAK_DB_NETWORK_ARG). Does NOT by itself fix the admin-console static-chunk 404 (that needs a shared gzip cache / sticky routing).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert the two resolve_host_cid.yml includes in 02b_updates.yml from the
[playbook_dir, ...] | path_join form to lookup('path_absolute', ...), matching
the repo-wide sweep.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Autoformat (mbake/yamllint) re-indented sequence items under their mapping key
across the regenerated meta files and wrapped one long URL scalar in the matomo
inject task. Whitespace/formatting only, no semantic change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce plugins/lookup/path_absolute.py as the single source for the
repo-root anchor: lookup('path_absolute', 'roles/...') returns an absolute
path under playbook_dir, joining every term. Replace all hand-built
playbook_dir path constructions across .yml/.yaml and .j2 templates with it:
[playbook_dir, '<rel>'] | path_join -> lookup('path_absolute', '<rel>')
playbook_dir ~ '/<rel>' -> lookup('path_absolute', '<rel>')
{{ playbook_dir }}/<rel> -> lookup('path_absolute', '<rel>')
Unrelated repo_path symbols in cli/meta/roles/applications/resolution and the
single bare discover_playwright_roles(playbook_dir) call are left untouched.
Add tests/lint/ansible/test_playbook_dir_via_lookup.py: fails the build if any
of the three construction forms reappears in .yml/.yaml/.j2, pointing callers
at the lookup. Unit test covers the plugin.
Touching these files pulled their pre-existing narration comments into
test_comments_valid scope; clean them (delete restatement/banners, keep real
trip-wires as Workaround/Upstream/Invariant markers). Drop caution, important
and pitfall from the comment-lint marker vocabulary and remove the comments
that used them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In swarm the object updates run delegated to KEYCLOAK_HOST_NODE, but that fact
was resolved early in the login step. Over a long deploy swarm can reschedule
keycloak to another node, leaving the update delegated to a node that no longer
runs keycloak, so kcadm fails with "Failed to create Keycloak object". The
cross-node rescue recovered, but with any_errors_fatal the rescued failure still
aborted the play (exit 2) after a slow recovery loop.
Re-resolve the live hosting node at the start of the update block and run the
update routines with a fresh delegate_to binding, so the first attempt targets
the node currently running keycloak. The rescue stays as a fallback for a
reschedule during the update itself.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.
- emit emoji names at the source: cli/build/include generator now writes
"🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extract the csp, domains, and networks blocks out of roles/*/meta/server.yml
into dedicated meta files, each mapping to its own top-level applications key
via the recursive meta-walk:
csp: -> meta/csp.yml -> applications.<app>.csp
domains: -> meta/domains.yml -> applications.<app>.domains
networks: -> meta/networks.yml -> applications.<app>.networks
Residual nginx/proxy keys (status_codes, client_max_body_size, locations,
ip6_enabled, config_upstream_url) stay in server.yml; an emptied server.yml is
deleted. 98 server.yml -> 249 new files, 74 deleted.
- Consumers repointed applications.<app>.server.{csp,domains,networks} ->
applications.<app>.{csp,domains,networks} across templates, vars, tasks,
lookups and filters.
- Infra readers migrated: canonical_domains_map, application_domain_index,
build_ldap_role_entries, service-registry overlay discovery,
roles_with_service, and the file-based readers utils/domains/list.py +
utils/meta/scan.py (read the now-deleted server.yml directly).
- Detection + mapping: _META_TOPICS gains csp/domains/networks;
ROLE_FILE_META_{CSP,DOMAINS,NETWORKS} constants + ROLE_FILES entries.
- New self-expiring lint guard (test_server_meta_no_split_fields) fails if a
server.yml still carries a split field; auto-fails from 2026-12-28 as a
delete reminder.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Simpleicons: the reachability probe ran inside the controller-side Jinja filter, which has no route to the node's published port in swarm, so every card silently fell back to Font Awesome. Move the probe to a STACK_HOST-delegated uri_retry step and make the filter pure assembly. Compose output is byte-identical.
Tile reachability: add a per-card services.dashboard.iframe capability (default = enabled) so apps whose SPA top-redirects out of an embed (Keycloak admin console, iframe:false) open in a new tab; roles_with_service emits the flag into the target manifest and the spec branches on it. For iframe:false targets the spec also asserts the HTTP layer permits framing (no blocking X-Frame-Options, CSP frame-ancestors allows the dashboard) so the flag can never mask a real header block. Fix the Tab-button locator and reveal it from fullscreen before asserting.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The realm-import scales keycloak 0->N (fresh reschedule across nodes), then the kcadm update block runs delegated to a fixed KEYCLOAK_HOST_NODE; the in-module recovery resolves only a node-local container id, so when a replica reschedules to another node it can never be found -> 'No such container: <dead cid>'. Wrap the update routines (extracted to 02b_update_routines.yml) in a rescue that re-resolves the hosting node + live id and re-delegates the updates, bounded to 5 attempts; compose re-raises immediately. Relax the recovery resolver to status=running so it can find the just-rescheduled, not-yet-healthy replica.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The per-role swarm overlay local subnet was a /28 (14 usable IPs). A role's swarm stack plus its sidecars and per-task network attachments can need more than that, which surfaces as 'node is missing network attachments, ip addresses may be exhausted' and leaves services (sso-proxy, app frontends) unresolvable mid-deploy. Re-allocate all 74 /28 roles to a unique /24 (254 usable) inside 192.168.0.0/16, chosen via the network address suggester so the assignments stay non-overlapping. Validated by tests/integration/infrastructure/networks/test_unique_valid_and_mapped.py and a full make test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When swarm replaces the keycloak replica mid-deploy, the cached container id dies and 'Update Client settings' aborts with 'No such container'. The kcadm module re-resolves a fresh health=healthy id and retries, but the 40 x 3s = 120s budget could expire before the new replica finished its cold boot (jgroups + infinispan + db index check), leaving the per-app SSO client unconfigured. Raise DEAD_CID_MAX_RETRIES to 80 (240s) so the recovery outlasts a keycloak cold boot.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
02a_manager_ops_on_host.yml set 'delegate_to: KEYCLOAK_HOST_NODE' directly on 'include_tasks: 02b_updates.yml', which Ansible rejects ("'delegate_to' is not a valid attribute for a TaskInclude"), so the playbook failed to parse (exit 4) for every SSO-consuming app, since keycloak is preloaded for them. The delegation is already supplied by the caller 02_manager_ops.yml, which includes 02a via apply: { delegate_to: KEYCLOAK_HOST_NODE }, so the include-level delegate_to was both redundant and invalid. Remove it; the sibling include 05_realm_import.yml already relies on the caller's apply, which is why it had no delegate_to.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 'Resolve hosting node + container id' block was gated 'when: KEYCLOAK_HOST_NODE is not defined or none'. A block-level when propagates to the included resolve_host_cid tasks and is re-evaluated per task; resolve_host_cid sets <prefix>_HOST_NODE in its first task, so once KEYCLOAK_HOST_NODE was set the later container-id resolution and the KEYCLOAK_LOCAL_CID set_fact self-skipped, leaving KEYCLOAK_LOCAL_CID undefined. The 'Wait for keycloak container to be healthy' shell then failed finalizing _raw_params with 'KEYCLOAK_LOCAL_CID is undefined' across all 16 SSO-dependent swarm apps. Flatten the block and drop the self-mutating when so resolve_host_cid always runs and sets both KEYCLOAK_HOST_NODE and KEYCLOAK_LOCAL_CID; this also keeps both facts fresh after the realm-import scale reschedules keycloak.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the ten Keycloak update routines (client, mail, RBAC, LDAP, reCAPTCHA, userprofile, users) out of 02a_manager_ops_on_host.yml into 02b_updates.yml so 02a stays within the block-size limit; 02a now includes 02b. Delegation is preserved by the caller 02_manager_ops, which includes 02a with apply: delegate_to KEYCLOAK_HOST_NODE; that propagates to the nested update tasks. The include conditions (LDAP shared/disabled, reCAPTCHA, emailVerified, non-LDAP user provisioning) are moved verbatim.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On swarm the realm-import scales keycloak 0->N replicas; the rescheduled task is gone by the time 'Update Client settings' runs, so the kcadm exec hit 'Error response from daemon: No such container' and the in-module recovery gave up after DEAD_CID_MAX_RETRIES*DEAD_CID_RETRY_DELAY = 5*3 = 15s, far shorter than a keycloak restart + healthcheck + JVM warmup (~30-90s), so the resolver returned empty every attempt and the dead id was never replaced. Raise DEAD_CID_MAX_RETRIES to 40 (120s window) and add --filter health=healthy to KEYCLOAK_CID_RESOLVE_CMD so recovery only re-targets a ready replica, matching the login/realm-import resolvers. Compose is unaffected: the resolver command and current_cid are passed only when DEPLOYMENT_MODE == swarm, so the retry loop never engages and the larger constant is inert.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the per-role swarm host-node/container-id resolve duplicated across fider, matomo, decidim, keycloak (login + realm-import), collabora and listmonk with two shared task utils under roles/sys-svc-compose/tasks/utils/swarm/. resolve_host_cid sets <prefix>_HOST_NODE / <prefix>_LOCAL_CID (swarm: converge + node + cid delegated to the node; compose: inventory host + given cid); wait_converged is the bare convergence gate.
Teach test_variable_definitions to read templated set_fact keys like {{ prefix }}_HOST_NODE as a *_HOST_NODE name pattern (guarded so a pure {{ x }} key cannot become a catch-all), and add the playbook_dir-relative include load-failure exemptions to .ansible-lint-ignore.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hardcoded redis://redis:6379 (no auth, fixed host) made the oauth2-proxy session store crash-loop in swarm, where the shared redis runs with --requirepass (NOAUTH / connection refused). openresty auth_request then returned 502/500 and every sso-gated app failed e2e readiness. Derive the url from the central engine model instead: authenticated central url when shared, embedded redis:6379 when local, matching the established consumer pattern (baserow, gitea, mastodon).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The container_ports migration left make test red; reconcile it:
- Delete 10 port vars orphaned when their compose mappings moved to the
container_ports lookup (OPENLDAP_DOCKER_PORT_OPEN, RABBITMQ_INTERNAL_PORT,
REGISTRY_CACHE_PORT, ERPNEXT_FRONTEND_PORT, FLOWISE_PORT_PUBLIC,
JITSI_WEB_HTTP_PORT, KEYCLOAK_SERVER_HOST, MINIO_API_PORT_INTERNAL,
OPENCLOUD_HTTP_PORT, OPENWEBUI_PORT_PUBLIC).
- Add PORT_BANDS.public entries for the fixed protocol ports the migration
declared (smtp/smtps/submission/pop3/pop3s/imap/imaps/sieve, rtmp, c2s/s2s).
- Route the container_ports internal-declared lint through the SPOT constants
(ROLE_FILE_TEMPL_COMPOSE / ROLE_FILE_META_SERVICES), cached read_text, and
load_yaml_any, per repo conventions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Take the host IP as an optional third element of each term
([service, protocol, host_ip]) instead of an ip= kwarg, and build the whole
ports: block from declared ports.local/public (host) + ports.internal
(container). Convert every compose template across 84 roles, folding the
remaining raw var/numeric/specific-ip mappings (including mailu's per-interface
mail ports bound to MAILU_IP4_PUBLIC). Multi-term calls render one term per line.
Add the compose-ports-must-use-container-ports lint: it forbids any raw
"<host>:<published>:<container>" publish item under a ports: block and requires
every term to declare ports.internal. Remove
tests/lint/roles/test_swarm_internal_ports.py, now subsumed.
Fix a container_depends_on rendering regression. The lookup self-indented
(default indent=4), so the added 4-space placement + | indent(4) triple-indented
the block. Flip the lookup default to column 0 (consistent with
container_networks), make every call use the 4-space + | indent(4) form, and add
a lint enforcing it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- web-app-keycloak: KEYCLOAK_CID_RESOLVE_CMD filtered status=running AND
health=healthy, so during swarm converge (no replica healthy yet) it returned
empty; the kcadm dead-container recovery then looped against the same dead cid,
burned its retries and fail_json'd. Drop health=healthy so a freshly
rescheduled running replica is usable.
- sys-svc-compose/swarm_registry_sync.py: a service sharing a sibling's locally
built <entity>_custom image (no build: of its own, already registry-prefixed)
fell into the prefix branch and ran `docker pull <entity>_custom` against the
registry -> 404. Collect the locally-built refs first and skip the upstream
pull when the prefix-stripped ref is one of them (it is retagged and pushed by
the building service).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The compose-flavour realm-import file runs `compose stop/run/up` and chdirs into
directories.instance, but it is only ever included via the DEPLOYMENT_MODE-driven
`05_realm_import/{{ DEPLOYMENT_MODE }}.yml` switch, so those compose-only calls
never reach a swarm host. Add the
`# include-gated: when: DEPLOYMENT_MODE == "compose"` file-header marker so the
compose-verb / compose-chdir lints treat the file as exempt; it was missing since
the 05_realm_import flavour split.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
DEPLOYMENT_MODE is binary (swarm / compose), so a negated `!= 'swarm'` gate
always means `== 'compose'`. Replace every negated gate across roles/ and
group_vars with the positive form for readability, and add a lint
(tests/lint/ansible/variables/test_no_deployment_mode_negation.py) that forbids
`DEPLOYMENT_MODE !=` in .yml/.yaml/.j2 with a `# nocheck:
deployment-mode-negation` escape.
The compose-verb / chdir / jinja-gate lint guidance messages now suggest the
positive form too; the gate matcher (utils/annotations/task_gate.py) and its
test inputs keep recognising the legacy `!= 'swarm'` so a nocheck-marked
exception is still detected as compose-only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
05_realm_import.yml becomes a thin orchestrator: probe whether the realm exists, then (when import is enabled and the realm is missing) include the per-mode body 05_realm_import/{{ DEPLOYMENT_MODE }}.yml and re-run 04_login. The compose path (stop/import/restart) moves verbatim to 05_realm_import/compose.yml; the swarm scale path moves to 05_realm_import/swarm.yml (was 05b_realm_import_swarm.yml); 05a_realm_import_run.yml is absorbed and removed. Pure structural move, no behavior change; ansible-lint clean (production profile).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The role cached keycloak's swarm container ID into KEYCLOAK_LOCAL_CID and reused it for every 'container exec <CID> kcadm.sh'. In swarm the realm-import scale-up reschedules keycloak, so the cached CID dies mid-block; the next kcadm exec aborts with 'No such container' and the module falls into the create branch, surfacing as 'Failed to create Keycloak object' on Update Client settings (15 swarm apps).
Make kcadm resilient: keycloak_kcadm_update.py re-resolves a fresh healthy CID via the swarm service label and retries on 'No such container' (bounded). Quiesce churn first: 05b waits for replicas X/X stable across 3 consecutive polls before re-resolving (stability poll extracted to files/wait_replicas_stable.sh). Add --filter health=healthy to CID resolution in 04_login and 05b. The 26.6.3 pin is unrelated. Swarm-gated; compose runs the module exactly once as before.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- keycloak: pin the image from floating 'latest' to 26.6.3 for reproducible deploys.
- sso_proxy: the oauth2-proxy sidecar was pinned to 26.6.3 (the keycloak image
version, mis-copied in 1db235b96), which is not a valid oauth2-proxy tag, so the
CA-trust ca-inject pull failed (manifest unknown) and every sso-proxy-gated app's
compose deploy aborted at the Generate CA trust override handler. Pin to v7.15.3
(live-confirmed pullable).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the floating 'latest' tag on the oauth2-proxy sso_proxy sidecar with a concrete version for reproducible deploys.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In swarm keycloak is unpinned and can land on a worker, where the realm-import host bind /opt/compose/keycloak/volumes/import does not exist (it is created only on the manager), so the task is Rejected and never converges. The running 'start' service reads the realm from the database; the swarm import is an independent manager-pinned one-shot (05b) that mounts the dir itself, and the bind is only needed by the compose import path (05a 'compose run keycloak import'). Gate the mount with when: DEPLOYMENT_MODE \!= 'swarm'.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A role's vars/main.yml is auto-loaded into play scope and persists, so proxy
override vars (webserver_*) leaked into the central front-proxy render of OTHER
roles (re-evaluated against the wrong application_id) — silently applying one
role's value elsewhere, or hard-failing when the lazy lookup hit a role lacking
the key (the seaweedfs<->matrix client_max_body_size crash).
- Templates (sys-svc-proxy html/upload/ws_generic, matrix synapse homeserver)
resolve client_max_body_size from the role's own server config
(lookup('config', application_id, 'server.client_max_body_size', '100m')).
- Move every role-level webserver_* out of vars/main.yml into task-scoped
include params (9 roles); they apply only to that role's proxy and can't leak.
- sys-stk-front-proxy resets webserver_* to safe defaults via task-scoped vars
on the render tasks, so a leaked value from any other source is neutralized;
a role's own include params (#20) still override the reset (precedence-tested,
direct + nested).
- Add test_no_leaky_proxy_role_vars lint forbidding webserver_* in vars/main.yml
(suppressable via '# nocheck: leaky-proxy-var').
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`bond` is a 0..1 factor for how strongly a service is tied to the overall
integration: bond: 1 = always required, bond: 0 = omittable. Groundwork for
tooling that later scores service importance and decides what can be dropped.
Named `bond` (not `bound`) to avoid clashing with the resource bounds/limits.
Add test_service_bond.py (lint): every enabled + shared service must declare a
valid `bond` (number in 0..1). Seed `bond: 1` as a placeholder on all 638
enabled+shared service entries across 88 roles, to be refined later.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The swarm e2e now iterates one provision+deploy+e2e round per app variant,
mirroring the compose matrix deploy, so the keycloak TOTP env-gate is no
longer needed: the variant's services.keycloak.realm.totp_enabled=false is
baked into host_vars per round and resolves at deploy time in swarm too.
New cli/administration/deploy/swarm/matrix.py drives the rounds by reusing
the compose planner (plan_dev_inventory_matrix, _resolve_variant_payloads,
_bake_overrides): per round it provisions with the baked --vars overlay plus
--app-variants, extends the swarm topology, deploys (e2e in-deploy), waits
for convergence, and purges prior-round stacks between rounds via the new
purge_stacks.sh. The workflow collapses its per-step provision/deploy/verify
into one orchestrator step; chaos (seed/drain/assert) still runs once.
- cli/administration/deploy/swarm/matrix.py: NEW round orchestrator
- scripts/tests/deploy/swarm/purge_stacks.sh: NEW per-round swarm purge
- scripts/tests/deploy/swarm/05_provision_inventory.sh: per-round inv dir plus
conditional --vars / --app-variants (standalone path unchanged)
- .github/workflows/test-deploy-swarm.yml: single orchestrator step, timeout 360
- default.env, utils/env/handlers/passthrough.py: register INFINITO_VARS_PAYLOAD
plus INFINITO_APP_VARIANTS passthrough keys
- roles/web-app-keycloak/vars/main.yml: drop the ENVIRONMENT env-gate
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The net refactor (366dc7b70/cb27ff375) migrated every shared-net provider (postgres/mariadb/openldap/ollama/openresty/keycloak) to the overlay registry but missed seaweedfs and minio, so an object-store consumer like baserow never joined the provider's docker network and its S3 client hit ECONNREFUSED on seaweedfs:8333 (the intended direct data-plane path).
Add networks.overlay to both engines so consumers attach, and complete the multi-service shared-net-provider contract: (1) render.py emits a bare auto-IPAM project default for an own shared_net provider (no name, no subnet) so the entity-named external net created by network/create.yml is not re-requested -- this caused 'invalid pool request: Pool overlaps' on the first attempt. (2) a provider_self_alias flag lets the provider's sidecars (proxy/sso-proxy) JOIN the shared net WITHOUT publishing the <entity> alias, so the name resolves only to the main container and baserow's :8333 never round-robins onto a port-less sidecar. (3) seaweedfs/minio 01_core pre-create the external <entity> net via network/create.yml (mirroring web-app-prometheus) so compose adopts it as the project default instead of building <entity>_default.
Verified: baserow compose e2e test-seaweedfs.js passes (upload stored in the bucket, 3.9s); new TestOwnSharedNetProvider locks the contract; full make test green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two compose-mode-only gaps that blocked compose deploys while swarm was unaffected.
svc-prx-openresty: the compose restart handler did 'openresty -s reload', which re-reads the file bind-mount but keeps the stale pinned inode after the real nginx.conf is rendered atomically (write-temp + rename). openresty stayed on the bootstrap 'listen 8080' config and the :80 default-vhost assert got connection refused. Force-recreate the container (like swarm's 'service update --force') so the bind remounts the live inode, plus a post-flight 'openresty -t'.
web-app-keycloak: KEYCLOAK_LOCAL_CID was resolved only in the swarm branch (via the swarm service label), so in compose it stayed undefined and the container health check fatalled. Set it from the compose container address when DEPLOYMENT_MODE \!= 'swarm'.
Both changes are gated to the compose path; the swarm deploy is untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add tests/lint/ansible/tasks/test_include_uniform_apply_keyword.py: when a file pulled in via a dynamic include_tasks carries the same hoistable task keyword on every top-level task, that keyword should be set once via apply: on the include site instead of being repeated per task. The rule is blacklist-based: it flags any uniform Ansible task keyword except the per-task-binding ones (name, register, args, vars, when, loop), and never flags the module itself.
Hoist every existing occurrence to apply: across the affected roles (delegate_to, run_once, become, notify, no_log, changed_when, delay, async, poll), dropping the per-task duplication. Behaviour is identical because apply: propagates each keyword to every included task; no_log masking via apply: was empirically verified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The deploy play runs with any_errors_fatal. The old block/rescue latched the play-abort flag on the expected first-deploy permanent-admin login failure, exiting 2 despite failed=0/rescued=1. Replace it with a failed_when:false probe plus a kc_login_perm.rc-gated include_tasks of the extracted 04_login_bootstrap_admin.yml, so no task ever fails and the bootstrap runs only when the permanent admin is absent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reconcile main's object-store feature and pre-swarm deploy tasks with the branch's compose/networks/volumes SPOT and swarm lints.
objstore (seaweedfs/minio): standalone compose + embedded service templates migrated to the container_image/container_volumes/container_networks/compose_volumes/compose_networks/depends_on lookups; meta/volumes.yml on the canonical dict schema with s3.json and nginx.conf as type:config; embedded services made swarm-compatible via compose_only/compose_restart.
deploy tasks (erpnext, opentalk, seaweedfs, shopware, snipe-it, wordpress, yourls): container exec routed through the container_address resolver and gated on IS_STACK_HOST, shell module plus stack_host_template, no chdir or compose-verb forms.
Also removed orphaned vars, restored MAILU_INSTANCE_DIR, freed the seaweedfs subnet (192.168.206.0/24), stripped literal quotes from the nextcloud/bookwyrm env templates, and routed main's objstore compose_volumes tests through dir_var_lib.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Give each reverse-proxied app a swarm-resolvable upstream: add services.<entity>.ports.internal.http to the role metadata and rename the main compose service from the generic 'application' to the entity name, so its short DNS alias on the shared overlay is unique. Thread the service name through compose, vars (*_SERVICE_NAME), volume mounts and the oauth2 origin host. Consolidate the split image/ports blocks in bridgy-fed and postmarks into the entity block.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Centralise the nginx proxy_pass directive in one source of truth (utils/networks/proxy.py + plugins/lookup/proxy_pass.py). Compose keeps the literal 127.0.0.1:<local> upstream; swarm resolves the service over the shared overlay at request time (variable upstream + http resolver) so a rescheduling backend never blocks openresty startup. Converts the html/media/upload/ws location partials, the keycloak oauth2 and login-iframe endpoints, and the logout endpoint to the lookup; adds the swarm resolver and the front-proxy service_key plumbing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
List the oauth2-proxy-gated roles still to migrate to the trusted-header SSO bridge (kix, friendica, akaunting, magento, suitecrm, funkwhale) with mechanism/gotcha notes, plus the excluded roles (flowise: commercial-licensed session code; jitsi: no HTTP session, use native AUTH_TYPE=jwt).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
