PHP_MEMORY_LIMIT and PHP_OPCACHE_MEMORY_CONSUMPTION were both the host's
RAM divided by 30, read from two keys in meta/services.yml: 533M on a
16 GB host, 2133M on a 64 GB one, beside a container limited to 3g either
way. They are now half and an eighth of that limit through
lookup('service_memory_mb'), which renders 1536M and 384M on every host.
The lookup sits in env.j2 and the two meta keys are gone. It reads the
applications payload, and meta/services.yml is part of that payload, so
a call from there would run back into the render that asked for it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The init gate is the image entrypoint, so it decides when a swarm task
starts php-fpm. For task slots other than 1 it spun on
flock -n "$INIT_LOCK" true
against /var/www/html, which is an NFS4 export in swarm. flock(1) opens
a path argument O_RDONLY and takes an exclusive lock; over NFS flock is
emulated as a whole-file POSIX write lock, which requires the descriptor
to be open for writing, so the call returns EBADF on every iteration.
The loop reads any non-zero exit as "lock still held" and never breaks.
Slots 2 and 3 therefore never reached the entrypoint and never started
php-fpm, while their occ-status healthcheck kept passing off the shared
volume, so swarm left them in the service VIP backend set and roughly
two thirds of fastcgi connects were refused.
The version.php comparison that preceded the lock is not a sufficient
gate on its own: version.php is rsynced before occ maintenance:install
runs, so it flips true mid-install.
Replace both loops with a readiness poll that only passes once the
instance reports installed, out of maintenance, with no pending database
upgrade, and at the image version. That needs no file locking, so the
mount semantics stop mattering, and it holds for the upgrade path too.
Matching is plain POSIX case rather than grep so busybox flag variants
cannot change the outcome.
occ refuses to run as root, so the poll needs the application user. The
gate ships from files/ and cannot template, so the user arrives as an
environment variable fed from the existing NEXTCLOUD_DOCKER_USER single
point of truth. The lookup sits inside the non-primary branch, which
keeps compose and slot 1 starting even without it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
New roles/sys-svc-container/templates/env.j2 sets TZ={{ HOST_TIMEZONE }} and LANG=C.UTF-8. Every role env.j2 includes it at the top; it renders via stack_host_template, the same engine that resolves the includes in compose.yml.j2, so it works in both compose and swarm. TZ/LANG are removed from the individual role env.j2 so each value stays single-source.
New lint tests/lint/ansible/test_env_base_include.py verifies every role env.j2 includes the base and never re-sets a base-owned key (keys derived from the base file).
mattermost and gitea meta/volumes.yml drop the /etc/timezone host-path bind: swarm hard-rejects a task whose bind source is missing on the node (Debian-trixie nodes have no /etc/timezone). TZ now comes from the base env. /etc/localtime bind also dropped; the container timezone follows TZ.
Non-directive comments were removed from the touched env.j2 (banners, restatements, reference URLs, narration) so the changed files pass the comment-lint; only tool directives remain.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Variant 1 runs redis without a password; the unconditional env line handed the image an empty REDIS_HOST_PASSWORD, phpredis sent AUTH to a passwordless redis (ERR Client sent AUTH), the session backend died and every login bounced in a CSRF loop. Emit the line only when the engine actually has a password; the rendered env is byte-identical whenever one is set.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Docker env-files take everything after '=' literally, so cosmetic column
alignment (KEY= value) leaks the spaces into the value (the cause of the
MariaDB "Access denied for user ' friendica'" failure). Add a lint scanning
every *env.j2 for whitespace before/after '=' or trailing whitespace, and strip
the existing offenders in espocrm, friendica, nextcloud and phpldapadmin.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
matomo: force-restart the swarm service after patching config.ini.php DB creds (the compose-up handler is a no-op in swarm), extracted into 10_resolve_host_cid. nextcloud: gate the redis PONG wait on the engine being local. prometheus: config 0444 (swarm config objects are root) and gate the host-monitor binds plus privileged cadvisor/node-exporter to compose only. dashboard: probe the public icon URL in swarm.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reconcile main's object-store feature and pre-swarm deploy tasks with the branch's compose/networks/volumes SPOT and swarm lints.
objstore (seaweedfs/minio): standalone compose + embedded service templates migrated to the container_image/container_volumes/container_networks/compose_volumes/compose_networks/depends_on lookups; meta/volumes.yml on the canonical dict schema with s3.json and nginx.conf as type:config; embedded services made swarm-compatible via compose_only/compose_restart.
deploy tasks (erpnext, opentalk, seaweedfs, shopware, snipe-it, wordpress, yourls): container exec routed through the container_address resolver and gated on IS_STACK_HOST, shell module plus stack_host_template, no chdir or compose-verb forms.
Also removed orphaned vars, restored MAILU_INSTANCE_DIR, freed the seaweedfs subnet (192.168.206.0/24), stripped literal quotes from the nextcloud/bookwyrm env templates, and routed main's objstore compose_volumes tests through dir_var_lib.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
docker stack deploy passes env-file values verbatim, including outer
double quotes - the container reads KEY="value" instead of KEY=value
and DB auth, URL parsers, and int(os.environ['KEY']) consumers break.
Compose keeps stripping the quotes as before.
- plugins/filter/dotenv.py: read DEPLOYMENT_MODE from the render
context via @jinja2.pass_context. compose mode keeps the old quoting
(escape $$, \, "). swarm mode passes the value through verbatim.
Missing or unknown mode falls back to compose quoting so a typo
never silently leaks unquoted values into compose. Direct Python
callers without a Jinja context keep the compose-style default.
- tests/unit/plugins/filter/test_dotenv_quote.py: keep the 12
direct-call cases, add 8 TestDotenvQuoteModeAware cases rendering
the filter through a real jinja2.Environment.
- 4 OIDC_TITLE templates (bookwyrm, confluence, jira, pretix): replace
the manual "{{ X | replace('\"','\\\"') }}" Compose-only escape
with {{ X | dotenv_quote }}.
- 3 env.j2 templates (nextcloud, taiga, wordpress): drop literal
outer quotes that were Compose-only decoration; keep | dotenv_quote
on the password / token assignments.
- sys-svc-rdbms/templates/env/mariadb.env.j2: strip literal quotes
unconditionally - the file name is not env.j2 so the dotenv-quote
integration test does not require the filter and the generated DB
passwords are alphanumeric.
Switch penpot, nextcloud, gitea, mattermost, openwebui, funkwhale, baserow, openproject, pixelfed, mastodon, bookwyrm, listmonk, peertube and opentalk to the engine-agnostic objstore lookup: the minio service entries are pinned off, a seaweedfs service (gated on the provider being in group_names, public for the media apps) replaces them, and the matrix variants pin the new flag. OpenTalk keeps object storage mandatory through an embedded fallback and provisions its bucket via weed shell.
Add objstore lookup SPOT keys consumed by these roles: endpoint (host:port), public_domain, public_bucket (public S3 host plus bucket) and public_url (scheme plus public_bucket, scheme derived from the provider TLS state). This removes the repeated lookup('config', lookup('objstore', ...), 'server.domains.canonical.api') / host:port concatenations from the consumer templates. Listmonk settings render their JSON values via the to_json filter.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).
req-008 — role meta layout
* Move every role's metadata under meta/<topic>.yml with file-root
convention (no wrapping key matching the basename):
- schema/main.yml + config/main.yml.credentials → meta/schema.yml
- users/main.yml → meta/users.yml
- config/main.yml.server → meta/server.yml
- config/main.yml.rbac → meta/rbac.yml
- config/main.yml.compose.services + every other top-level key
of config/main.yml inlined under <entity>.<key>
→ meta/services.yml
- config/main.yml.compose.volumes → meta/volumes.yml
* Drop the legacy schema/, users/, config/ directories from every role.
* meta/schema.yml: support nested credential keys, optional `default:`
field (Jinja literal, not rendered/validated/regenerated), implicit
`algorithm: plain`.
* Materialised paths drop the `compose.` prefix:
compose.services.<e>.<…> → services.<e>.<…>
compose.volumes.<k> → volumes.<k>
<former-top-level>.<…> → services.<entity_name>.<former-top-level>.<…>
* Update every Python consumer (utils/cache, utils/manager, utils/applications,
plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
and task file to the new paths.
* No legacy fallback. Repository-wide grep for `compose.services.` /
`compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
* Add tests/lint/repository/test_role_meta_layout.py guarding against the
old paths returning, and an explicit lint guard that meta/variants.yml
files do not carry a `compose:` wrapper.
req-009 — per-role networks/ports
* Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
dns_resolver}.
* Delete group_vars/all/09_ports.yml entirely; ports now live on the
service entity that exposes them at
roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
* Rename + int-cast every former top-level `services.<entity>.port`
field to `services.<entity>.ports.inter` (single int).
* Always-category-keyed maps for ports.local / ports.public; relay
spans use {start, end} integers.
* Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
of truth for the per-<scope>.<category> band ranges, consumed by both
the new CLI helpers and the lint test.
* New CLI helpers (cli meta ports suggest, cli meta networks suggest)
walk the live role tree gap-first then increment-fallback, with
deterministic output and non-zero exit on capacity overflow.
* cli create role consumes both helpers when scaffolding a new role.
* Lint extensions: bare-int port shape rejection, host-bound port
collision check (single ints + relay span integers, inter excluded),
band-membership check with documented allow-list for the legacy
web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.
req-010 — run_after / lifecycle migration
* Move galaxy_info.run_after and galaxy_info.lifecycle out of
meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
lifecycle}; <primary_entity> is the value returned by
get_entity_name(role_name).
* For multi-entity roles whose primary entity has no compose service
(web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
gen roles), create a dedicated metadata-only top-level entry under
<primary_entity>.
* Introduce utils/roles/meta_lookup.py with get_role_run_after /
get_role_lifecycle that resolve <primary_entity> once and read from
meta/services.yml; degrade gracefully when meta/services.yml is
absent OR the field is absent; raise MetaServicesShapeError when the
file exists but is malformed.
* Update every consumer (utils/service_registry, utils/invokable,
cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
use the helper.
* Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
run_after/lifecycle never on a non-primary entity; empty run_after
lists rejected.
Also folded into this change set
* services_disabler now writes `services.<svc>.{enabled,shared}` into
host_vars instead of the old `compose.services.<svc>.{...}` path; the
discover/find pair walks meta/services.yml and only treats entries
carrying enabled/shared/provides as provider declarations (so
metadata-only entries from req-010 do not create spurious provider
rows).
* service_registry skips `run_after` deps whose target was filtered
out via SERVICES_DISABLED instead of failing the whole load.
* cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
consume the new applications.<role>.services.<svc>.image shape
(mirror.md updated to match).
* utils/domains/list.build_applications_from_roles now reads
meta/server.yml directly (it had been silently returning {} since
config/main.yml stopped existing).
* utils/roles/resource_validation.filter_roles_by_min_storage reads
meta/services.yml.<entity>.min_storage instead of the legacy
compose.services path.
* web-app-bluesky pre-clones the social-app build context BEFORE
sys-svc-compose so the build context is on disk by the time
sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
* 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
web-app-wordpress, web-app-keycloak) lose their `compose: services:`
wrapper so deep-merge over meta/services.yml lands on the new
file-root shape.
Validation
* Unit tests: 1571 passed.
* Lint tests: 50 passed.
* Integration tests: 161 passed (1 skipped).
* Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
discourse,mattermost}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Moves the SYSTEM_EMAIL_* resolution logic out of group_vars/all/03_email.yml
into a dedicated Ansible lookup plugin, so every role template reads outgoing
mail settings via lookup('email') (global) or lookup('email', application_id)
(per-app). Per-app compose.services.email is merged on top of the resolved
defaults; SYSTEM_EMAIL_* variables remain usable as an inventory escape hatch.
Also splits the email docs: administrator-facing configuration stays under
docs/administration/configuration/, plugin contract and extension rules move
to docs/contributing/design/services/email.md. services.md is relocated to
services/base.md so the services/ subtree can host per-service design pages.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces inline `applications`/`users` merging with dedicated lookup
plugins and runtime hydration. Introduces `add_csp_hash` filter,
integrates sort into `find_dock_val_by_bkp_entr`, splits the Keycloak
one-shot realm import into `05a_realm_import_run.yml`, and fixes the
Mastodon administrator task YAML. Removes obsolete `csp_hashes` filter,
`sys-token-store` Jinja filter, and empty CLI setup modules.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add 07_admin_identities.yml: resolves LDAP-backed admin identities
for OCC calls that require the actual NC username, not the LDAP DN
- vars: add NEXTCLOUD_HPB_TURN/STUN_ACTIVE_CONFIG selecting standalone
or onboard TURN based on enabled flags; add TALK_ADMIN_URL
- config: talk container gated on spreed plugin flag
- env.j2 + playwright.env.j2: expose active TURN/STUN config and
Talk admin URL for container env and E2E tests
- playwright.spec.js: extend with Talk TURN/STUN E2E assertions
- schema: add spreed.enabled validation
- coturn compose: move relay port range before TLS block so ports are
always set regardless of TLS mode
- docs/requirements: document TURN relay requirement
- test_resolver: cover new resolution paths
Replace usages of the get_app_conf filter with the config lookup plugin
across roles and templates. Adjust quoting where required to keep YAML
and Jinja syntax valid. No functional changes intended.
https://chatgpt.com/share/69860678-cef8-800f-8402-7fe6231fdc2d
- Mount includes.php read-only instead of docker cp
- Ensure include loader starts with proper PHP header
- Add overwriteprotocol, overwritehost, and trusted_proxies via system config
- Simplify env handling for protocol and trusted proxies
- Remove redundant permission and copy tasks
https://chatgpt.com/share/697f7aff-859c-800f-bfaa-0d4c2378fd85
- Remove deprecated filter_plugins/tls.py and all dependent filters
- Introduce strict lookup plugin lookup_plugins/tls_resolve.py as single TLS authority
- Migrate all roles, templates, vars, and tasks to use tls_resolve (protocols, ports, URLs, cert paths)
- Enforce explicit TLS context (domains, applications, TLS_ENABLED, TLS_MODE)
- Support letsencrypt and self_signed with SAN handling and per-app overrides
- Update TLS provider roles to assert resolver state instead of setting facts
- Add comprehensive unit tests for tls_resolve lookup
https://chatgpt.com/share/6972b49b-b074-800f-a323-dd9fe69d02eb
with explicit per-application TLS resolution.
Key changes:
- Introduce sys-front-tls-layer as the single TLS abstraction
- Centralize certificate paths via TLS_CERT_FILE / TLS_KEY_FILE / TLS_CA_FILE
- Replace sys-svc-letsencrypt SSL templates with generic TLS layer includes
- Add tls_* Jinja filters (enabled, mode, protocol, port, websocket)
- Resolve TLS mode per application (letsencrypt | self_signed | off)
- Remove implicit defaults and WEB_* globals to fail fast on misconfiguration
- Support self-signed certificates with SAN awareness
- Make TLS behavior explicit and consistent across all roles
This decouples application logic from certificate providers and
allows switching TLS flavor without touching app roles.
https://chatgpt.com/share/69727c61-8e68-800f-a267-473a7e627978
- Replace DEFAULT_SYSTEM_EMAIL merge with flat SYSTEM_EMAIL_* variables
- Update templates across apps to new variable names
- Fix msmtp health check to use -t and local username fallback
- Improve postfix localhost config (localdomain, mydestination, timeouts)
- Extend sys-hostname to ensure /etc/hosts mapping once
https://chatgpt.com/share/695e8f39-0694-800f-b91c-a74f4b78d7c8
- Migrate Mailu credentials from legacy mailu_token to tokens['web-app-mailu']
- Update all affected application templates to use the new token structure
- Adjust inventory validation to no longer special-case mailu_token
- Minor typing and formatting cleanups in inventory creation and tests
https://chatgpt.com/share/69554915-87c8-800f-afd5-12118dbb1107
- Added role tasks to deploy templated Dockerfile for Whiteboard service
- Configured build context and custom image name (nextcloud_whiteboard_custom)
- Increased PID limits and shm_size for stable recording
- Adjusted user ID variable naming consistency
- Integrated path_join for service directory variables
- Fixed build permissions (install as root, revert to nobody)
Reference: ChatGPT conversation https://chatgpt.com/share/68f771c6-0e98-800f-99ca-9e367f4cd0c2
- Added Chromium headless flags and writable font cache/tmp volumes
- Enabled WebSocket proxy forwarding for /whiteboard/
- Verified and adjusted CSP and frontend integration
- Added Whiteboard-related variables and volumes in main.yml
See ChatGPT conversation (20 Oct 2025):
https://chatgpt.com/share/68f655e1-fa3c-800f-b35f-4f875dfed4fd
- Replace inline get_app_conf secrets in env.j2 with dedicated vars (TURN, signaling, internal)
- Correctly model signaling_servers as object {servers, secret} in spreed.yml
- Use UDP stun_turn port instead of TLS for transport=udp
- Add fallback logic for standalone Coturn role in main.yml
- Remove obsolete Greenlight section from BBB override
Ref: https://chatgpt.com/share/68d74e25-c068-800f-ae20-d0e34ac8ee12
- Switch to custom BBB Docker repository
- Externalize Coturn and Collabora by default
- Add dedicated 03_dependencies.yml for dependency handling
- Improve env templating with lowercased feature flags
- Add conditional healthcheck for Greenlight
- Refactor TURN/STUN/relay handling with role variable _BBB_COTURN_ROLE
- Extend Collabora/Greenlight dependency wiring in override file
- Nextcloud Talk: refine vars and enable/disable logic with separate plugin/service flags, add network_mode support and conditional nginx proxy block
Ref: https://chatgpt.com/share/68d741ff-a544-800f-9e81-a565e0bab0eb
- Split internal/external Coturn for BBB and Nextcloud
- Added dedicated relay port ranges per app
- Updated env and compose overrides for coturn
- Ensure coturn role is loaded conditionally
- Standardize credential/env passing for coturn
@See https://chatgpt.com/share/68d6f376-4878-800f-b4f7-62822caa49ea
• Add config.performance.async.wait_for and expose as nextcloud_wait_for_async_enabled to toggle waiting for async jobs.
• Split system/admin/index maintenance into separate tasks: 02_add_missing_indices.yml, 03_admin.yml, 04_system_config.yml.
• Refactor plugin flow: rename 02_plugin→05_plugin, 03_plugin_routines→06_plugin_routines, 04_plugin_enable_and_configure→07_plugin_enable_and_configure; remove old 03_plugin_routines and 05_system.
• Harden async handling: filter async_status loops by ansible_job_id; conditionally wait only when nextcloud_wait_for_async_enabled; reduce delay to 1s.
• Reorder main.yml to run system steps before plugin setup; keep handlers flush earlier.
• env.j2: simplify get_app_conf lookups (drop extra True flag).
• vars/main.yml: add nextcloud_host_nginx_path and nextcloud_wait_for_async_enabled.
https://chatgpt.com/share/689c9d4a-1748-800f-b490-06a5a48dd831
