The frontend nginx only listens on 8080; container port 9000 belongs to
the separate websocket service, which the frontend reaches internally
via its /socket.io proxy (SOCKETIO env). Publishing 4004->9000 on the
frontend therefore pointed at a port with no listener, and the swarm
reachability probe failed with 'manager port 4004/tcp unreachable'.
Publish only the http port. Realtime traffic is unaffected: it enters
through the http vhost and is proxied to websocket:9000 inside the
stack, matching upstream frappe_docker which publishes the frontend
alone.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Container-name vars derived their value inconsistently (hard-coded literals, entity_name composites) instead of the role-owned SPOT. Add a lint requiring every *_CONTAINER var to resolve through lookup('config', app, 'services.<x>.name'), the bare entity_name form, or the engine lookup; give each multi-container worker a name: in meta/services.yml (with realistic resource limits) and switch its var to the lookup. Extend the engine lookup with a shared-aware 'container' field (central provider container when shared, embedded <entity>-<engine> otherwise) and gate zammad's embedded memcached sidecar on not-shared so its container_name never collides with the central memcached.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
docker stack deploy does not interpolate $$ back to $, so the swarm
backend received a literal "$$@" and gunicorn never started. Passing
the full command inside the bash -c script removes the interpolation
dependency; compose renders the identical argv as before.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
resource_filter now falls back service_name -> entity_name -> hard
default, so entity-level limits keep applying to per-service compose
entries. Compose service names drop the entity prefix (kix, jitsi,
zammad, bluesky); the docker container_name keeps it via compose_only.
A new lint forbids '<entity>-'-prefixed service names.
Service names and internal ports are declared once: vars/main.yml
holds <ROLE>_<SERVICE>_SERVICE constants (container names derived via
entity_name), meta/services.yml declares ports.internal consumed
through config lookups. sso-proxy upstreams stack-qualify the origin
host in swarm because short aliases are ambiguous on shared overlays.
Stale proxy.service_key entries fixed for erpnext/shopware/nextcloud.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
docker stack deploy does not append the image CMD when a service overrides only the entrypoint, so the backend's gated entrypoint ran /usr/local/bin/entrypoint.sh without a command: it linked assets and exited 0 in a ~5s replace loop (every other erpnext service carries an explicit command and ran fine). Declare the upstream gunicorn invocation as the service command; in compose it overrides the image CMD with the same value, in swarm it reaches the gate as $@.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every compose.yml.j2 now declares services as '{{ service_name }}:' with
its '{% set %}' knobs on two-space lines above the key, includes at four
spaces and if/for tags aligned to the content they wrap. The layout
renders correctly because stack_host_template enables lstrip_blocks for
compose.yml.j2; indent(4, True) in the container base and '{%+' guards
in whitespace-sensitive scalars keep the shared fragments identical
under both render modes. A new lint test enforces the skeleton.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
docker stack deploy interpolates $ in the compose file and rejects a bare $@ ("invalid interpolation format"); docker compose tolerates it. $$@ normalises to $@ in both, so the entrypoint is compose- and swarm-compatible.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a 'timeout' lookup plugin that returns round(base * TIMEOUT_FACTOR)
(group_vars default 1, so value-identical to the old literals). One knob
in group_vars stretches every wait in the stack for a slow uplink:
TIMEOUT_FACTOR: 2 doubles them all.
- plugins/lookup/timeout.py + unit tests (identity, factor scaling,
string coercion, error paths).
- group_vars/all/17_resource.yml: TIMEOUT_FACTOR: 1.
- Convert all 34 literal 'timeout:' task keywords / module args across 25
task+handler files to "{{ lookup('timeout', <base>) | int }}".
- tests/lint/repository/yaml/test_timeout_uses_lookup.py enforces the
lookup in every task/handler timeout (opt-out: # nocheck: timeout-lookup).
- swarm.yml pre-deploy build handler: add the missing timeout (same
network-build hang vector as compose.yml's Build compose).
- Drop 13 pre-existing narration comments in the touched files (comment
lint enforces whole-file on any edited file).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to the swarm CID/redis fix that make quality flagged:
- Extract the site-bootstrap sequence into 02b_create_site.yml (renamed from
02b_restart_frappe.yml; now new-site + restart + re-resolve + wait) and include
it with when: rc \!= 0, so the oversized when-block disappears and the sub-file
is not parsed at all when the site already exists.
- The backend entrypoint now execs the image's real /usr/local/bin/entrypoint.sh
after the config-ready gate, so the sites/assets symlink that entrypoint creates
is preserved instead of relying on a sibling replica to recreate it.
- Drop the now-dead ERPNEXT_BACKEND_CONTAINER_ADDRESS var, shorten two over-long
OIDC restart task names, and remove stale narration comments.
Also suppress the INFINITO_MAIN_REMOTE/BRANCH env lints in the git pre-push helper:
they are maintainer tooling knobs, not deployment keys.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The manager-op tasks ran 'container exec {{ ERPNEXT_BACKEND_CONTAINER_ADDRESS }}'
where the address resolves a LOCAL container id. Under swarm the backend task can
run on another node (and 'service update --force' reschedules it), so the id came
back empty, collapsing the command and failing 'apply API-bot' with a bogus 'No
such file'. Resolve the backend's hosting node + local id via resolve_host_cid,
delegate_to that node, and re-resolve after each forced restart. The four apply_*.py
tasks can no longer use a host '< file' redirect once delegated, so feed the script
through 'stdin:' instead; the OIDC on-change restart splits into compose vs swarm.
docker stack deploy strips depends_on conditions, so queue/websocket/scheduler
booted before the one-shot configurator wrote redis_* into common_site_config.json
and wedged on frappe's 127.0.0.1 defaults. Gate each worker (and the backend
entrypoint) to block until redis_socketio appears, closing the race in swarm and
staying a no-op in compose where the config already exists.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Role README Credits now name galaxy_info.author (plain bold or wrapped
in a Markdown link), matched byte-for-byte against the role's own
meta/main.yml, instead of a hard-coded maintainer name. Regenerate all
245 role README Credits blocks to the new 'Implemented by ...' format.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On swarm the bare compose container names (erpnext-backend, ...) do not exist,
so `container restart` failed with "No such container". Split the restart into a
compose path (container restart) and a swarm path (service update --force
<entity>_<service>), extracted into 02b_restart_frappe.yml to keep the bootstrap
block within the size limit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Autoformat (mbake/yamllint) re-indented sequence items under their mapping key
across the regenerated meta files and wrapped one long URL scalar in the matomo
inject task. Whitespace/formatting only, no semantic change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.
- emit emoji names at the source: cli/build/include generator now writes
"🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extract the csp, domains, and networks blocks out of roles/*/meta/server.yml
into dedicated meta files, each mapping to its own top-level applications key
via the recursive meta-walk:
csp: -> meta/csp.yml -> applications.<app>.csp
domains: -> meta/domains.yml -> applications.<app>.domains
networks: -> meta/networks.yml -> applications.<app>.networks
Residual nginx/proxy keys (status_codes, client_max_body_size, locations,
ip6_enabled, config_upstream_url) stay in server.yml; an emptied server.yml is
deleted. 98 server.yml -> 249 new files, 74 deleted.
- Consumers repointed applications.<app>.server.{csp,domains,networks} ->
applications.<app>.{csp,domains,networks} across templates, vars, tasks,
lookups and filters.
- Infra readers migrated: canonical_domains_map, application_domain_index,
build_ldap_role_entries, service-registry overlay discovery,
roles_with_service, and the file-based readers utils/domains/list.py +
utils/meta/scan.py (read the now-deleted server.yml directly).
- Detection + mapping: _META_TOPICS gains csp/domains/networks;
ROLE_FILE_META_{CSP,DOMAINS,NETWORKS} constants + ROLE_FILES entries.
- New self-expiring lint guard (test_server_meta_no_split_fields) fails if a
server.yml still carries a split field; auto-fails from 2026-12-28 as a
delete reminder.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bench new-site never passed --db-host, so Frappe defaulted to 127.0.0.1 (MySQLdb OperationalError 2002). The DB host is only written to common_site_config.json by the one-off configurator, which swarm never gates (docker stack deploy ignores depends_on; a restart:none one-off shows 0/1 forever, so the convergence wait can't gate on it either). Pass host/port explicitly; the vars already resolve to mariadb:3306.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add tests/lint/ansible/services/test_variants_shared_false_once.py enforcing the one-local-provider-variant limit on services.<key>.shared: false pins, document the variants-shared-false-once suppression marker, and flip the database service (mariadb/postgres/redis) to shared: true across the erpnext, funkwhale, shopware, snipe-it, suitecrm, taiga, xwiki and zammad variants to comply.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Conflicts resolved keeping the branch's structure and taking main's version bumps: web-app-pihole kept the branch's lookup-based implementation (container_image/ports/volumes plus compose_volumes/networks lookups and meta/volumes.yml); web-app-erpnext took main's image version v16.25.0; web-app-gitlab took main's 19.1.1-ee.0; CHANGELOG took main's [10.1.1] entry. The branch's swarm-nfs structural changes (erpnext internal ports/proxy/backend, gitlab redis) were preserved. make test is green on the merged tree.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The container_ports migration left make test red; reconcile it:
- Delete 10 port vars orphaned when their compose mappings moved to the
container_ports lookup (OPENLDAP_DOCKER_PORT_OPEN, RABBITMQ_INTERNAL_PORT,
REGISTRY_CACHE_PORT, ERPNEXT_FRONTEND_PORT, FLOWISE_PORT_PUBLIC,
JITSI_WEB_HTTP_PORT, KEYCLOAK_SERVER_HOST, MINIO_API_PORT_INTERNAL,
OPENCLOUD_HTTP_PORT, OPENWEBUI_PORT_PUBLIC).
- Add PORT_BANDS.public entries for the fixed protocol ports the migration
declared (smtp/smtps/submission/pop3/pop3s/imap/imaps/sieve, rtmp, c2s/s2s).
- Route the container_ports internal-declared lint through the SPOT constants
(ROLE_FILE_TEMPL_COMPOSE / ROLE_FILE_META_SERVICES), cached read_text, and
load_yaml_any, per repo conventions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Take the host IP as an optional third element of each term
([service, protocol, host_ip]) instead of an ip= kwarg, and build the whole
ports: block from declared ports.local/public (host) + ports.internal
(container). Convert every compose template across 84 roles, folding the
remaining raw var/numeric/specific-ip mappings (including mailu's per-interface
mail ports bound to MAILU_IP4_PUBLIC). Multi-term calls render one term per line.
Add the compose-ports-must-use-container-ports lint: it forbids any raw
"<host>:<published>:<container>" publish item under a ports: block and requires
every term to declare ports.internal. Remove
tests/lint/roles/test_swarm_internal_ports.py, now subsumed.
Fix a container_depends_on rendering regression. The lookup self-indented
(default indent=4), so the added 4-space placement + | indent(4) triple-indented
the block. Flip the lookup default to column 0 (consistent with
container_networks), make every call use the 4-space + | indent(4) form, and add
a lint enforcing it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Generalize tests/lint/roles/test_swarm_internal_ports.py from http-only to every
published port kind (http, websocket, ssh, ...) referenced in a compose port
mapping (ports.<local|public>.<kind>): each must declare the matching
ports.internal.<kind> so the role's swarm port surface is complete and
consistent. Fix the roles it flagged:
- pihole, shopware: ports.internal.http (80, 8000)
- erpnext, mastodon, odoo: ports.internal.websocket (9000, 4000, 8072)
- gitea, gitlab: ports.internal.ssh (22)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In swarm the default replicas=node-count made erpnext's 7 services overflow its
per-stack /28 overlay (16 IPs), so swarm rejected tasks with "ip addresses may
be exhausted". Rather than pin everything to one replica, let the stateless
services scale and size the overlay to fit:
- backend (gunicorn), queue-short/queue-long (RQ workers) and the nginx frontend
scale to node-count: state lives in MariaDB + Redis, jobs are claimed
atomically from Redis, and swarm reaches each service via its overlay VIP (the
published port is ingress, so DOCKER_BIND_HOST binding does not pin them).
- websocket (no socket.io redis adapter, needs sticky sessions), scheduler
(singleton, else duplicate scheduled jobs) and configurator (one-shot) stay at
one replica via a single service_replicas = 1 above those blocks.
- widen the erpnext overlay from the packed 192.168.105.176/28 to a fresh
192.168.214.0/24 (4 scalable x node-count + 3 singletons + VIPs exceed a /28).
Also relocate the configurator to the end of the template: its
{% set docker_restart_policy = 'no' %} is template-scoped and was leaking
restart:no / swarm condition:none onto every long-running service. With it last
the one-shot policy stays scoped to the configurator; depends_on is by name so
file order is safe, and compose is unaffected (compose_replicas emits nothing in
compose mode).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
container_address(app, key) resolves services.<key>.name; a missing key or
empty name raises at task-arg finalization and killed the compose deploys for
erpnext (no 'backend' service) and friendica (service had no name).
- web-app-friendica: add name to the friendica service
- web-app-bigbluebutton: add names for postgres/greenlight (Compose default
<project>-<service>-<index>, upstream sets no container_name)
- web-app-erpnext: short-key all 7 sub-containers (configurator/backend/
websocket/scheduler/queue-short/queue-long/frontend) with prefixed
container_names, so the swarm resolver (<stack>_<key>) and compose exec
(<container_name>) both resolve; backend addressed via container_address('backend')
- tests/lint: new guard replays container_address._resolve_bare_name over the
static services config so a missing service/name is caught before CI deploys
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mark the erpnext primary entry enabled+shared so other roles can declare it as a dependency (erpnext: {enabled, shared}); the service registry now resolves it to web-app-erpnext, matching the web-app-matomo provider pattern.
Drop svc-db-mariadb from run_after — it crossed deploy types (server -> universal), which the service-loader rejects once the entry becomes a provider. The MariaDB dependency and ordering are already handled by the mariadb: shared service flag.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
About 38 web-app/web-svc roles now consume the central svc-db-* and svc-dns engines via lookup('engine', ...) instead of embedded sidecars: meta/services flags, variant matrices, and env/conf hosts point at the shared services. Lets these apps run unpinned and NFS-shareable in swarm.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`bond` is a 0..1 factor for how strongly a service is tied to the overall
integration: bond: 1 = always required, bond: 0 = omittable. Groundwork for
tooling that later scores service importance and decides what can be dropped.
Named `bond` (not `bound`) to avoid clashing with the resource bounds/limits.
Add test_service_bond.py (lint): every enabled + shared service must declare a
valid `bond` (number in 0..1). Seed `bond: 1` as a placeholder on all 638
enabled+shared service entries across 88 roles, to be refined later.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mark the erpnext primary entry enabled+shared so other roles can declare it as a dependency (erpnext: {enabled, shared}); the service registry now resolves it to web-app-erpnext, matching the web-app-matomo provider pattern.
Drop svc-db-mariadb from run_after — it crossed deploy types (server -> universal), which the service-loader rejects once the entry becomes a provider. The MariaDB dependency and ordering are already handled by the mariadb: shared service flag.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mark the erpnext primary entry enabled+shared so other roles can declare it as a dependency (erpnext: {enabled, shared}); the service registry now resolves it to web-app-erpnext, matching the web-app-matomo provider pattern.
Drop svc-db-mariadb from run_after — it crossed deploy types (server -> universal), which the service-loader rejects once the entry becomes a provider. The MariaDB dependency and ordering are already handled by the mariadb: shared service flag.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reconcile main's object-store feature and pre-swarm deploy tasks with the branch's compose/networks/volumes SPOT and swarm lints.
objstore (seaweedfs/minio): standalone compose + embedded service templates migrated to the container_image/container_volumes/container_networks/compose_volumes/compose_networks/depends_on lookups; meta/volumes.yml on the canonical dict schema with s3.json and nginx.conf as type:config; embedded services made swarm-compatible via compose_only/compose_restart.
deploy tasks (erpnext, opentalk, seaweedfs, shopware, snipe-it, wordpress, yourls): container exec routed through the container_address resolver and gated on IS_STACK_HOST, shell module plus stack_host_template, no chdir or compose-verb forms.
Also removed orphaned vars, restored MAILU_INSTANCE_DIR, freed the seaweedfs subnet (192.168.206.0/24), stripped literal quotes from the nextcloud/bookwyrm env templates, and routed main's objstore compose_volumes tests through dir_var_lib.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Give each reverse-proxied app a swarm-resolvable upstream: add services.<entity>.ports.internal.http to the role metadata and rename the main compose service from the generic 'application' to the entity name, so its short DNS alias on the shared overlay is unique. Thread the service name through compose, vars (*_SERVICE_NAME), volume mounts and the oauth2 origin host. Consolidate the split image/ports blocks in bridgy-fed and postmarks into the entity block.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
