The task above this one is named 'Flush handlers to make DB available before password reset' and was the last line in the file: the reset it flushed for did not exist. The upstream entrypoint sets the password, but inside its install branch next to create-admin-user, so it only ever runs when the app is first installed. The swarm matrix rotates credentials between the sync and the async pass, and the async pass then typed a password EspoCRM had never been told about; in run 33923190220 espocrm#1 re-rendered the login form with the credential still filled in.
The invocation mirrors the entrypoint's own: bin/command is literally 'include __DIR__ . "/../command.php"', so php command.php set-password is the same program, and SetPassword reads through IO::readSecretLine, which is fopen('php://stdin') plus fgets and needs no terminal. Executed against the image: the stty -echo call complains on stderr without a tty and fgets still returns the piped value. The image's working directory is /var/www/html, so the relative path resolves.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
espocrm_daemon and espocrm_websocket exited 0 every six seconds while the main
container sat healthy for thirty-nine minutes, so the deploy waited out its retries
on
not converged: espocrm_daemon espocrm_websocket
espocrm_daemon.1 desired=Shutdown current=Complete
The service log shows our own gate working -- "[entrypoint] Exec: docker-websocket.sh"
-- and upstream's failing right after it. docker-*.sh call exitIfNotReady, which runs
`bin/command app-check` and exits 0 when that fails. CommandManager resolves the name
to an AppCheck class, and espocrm/espocrm carries no AppCheck.php at 10.0.6, so the
gate can never pass. isLegacy does not spare us either: it returns true only when
/var/www/html is itself a mount point, and we mount three subdirectories.
Only the broken half is replaced. applyConfigEnv stays, because the websocket service
is the only carrier of ESPOCRM_CONFIG_WEB_SOCKET_*, and those reach the shared
config.php nowhere else. Readiness is the caller's job already: --wait-ready blocks on
the marker the main container writes once its install has finished, which is a
strictly better-informed signal than app-check.
The logic lives in files/shell/sidecar_boot.sh rather than a bash -c string, following
the route this role already uses for docker-entrypoint-custom.sh and seed_config.php:
copied into the build context, COPYed and chmod'ed in the Dockerfile, addressed
through ESPOCRM_SCRIPT_SIDECAR_DOCKER. That buys shellcheck and bash -n coverage the
embedded string never had, drops one layer of quoting, and turns a missing argument
into a named error instead of a php invocation with no script.
ESPOCRM_APP_DIR replaces the second hardcoded /var/www/html in vars.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A handler name was doing two jobs at once: the line an operator reads in the
play output, and the identifier a `notify:` binds to. The two pull in opposite
directions -- the log wants "restart the SSH service, with diagnostics", the
binding wants a token that survives quoting, YAML flow style and a shell
round-trip -- and a name that changes for readability silently changes which
handler runs.
Every handler that a notify reached by name now carries a `listen:` with that
exact wording, so the binding is explicit before anything is renamed. The
topics themselves become identifiers: `compose up` reads `compose-up`,
`Restart NFS ganesha` reads `restart-nfs-ganesha`, 22 of them across both
sides of every notify. Where the topic is assembled at runtime, both branches
moved together: the conditional in sys-service carries two literals, and the
LDAP renderer interpolates `import-{{ folder }}-ldif-files`, which is what
makes those handlers reachable at all.
Freed from the binding, the names take the emoji prefix every other task line
already has.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The task name templated item.key, which Ansible renders before the loop binds item (object of type str has no attribute key). Use a static name and surface the per-item key via loop_control.label.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
container exec ... < {{ role_path }}/files/X reads the file on the
executing host. Under swarm the manager-ops execs are delegated to the
node hosting the container, where the controller checkout does not
exist, so the redirect failed with 'No such file or directory'.
Read the script on the controller via lookup('file') and pass it as
module stdin instead, matching the existing set_siteurl.php pattern.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
02_manager_ops ran container exec for the config check, upgrade and flag-setter against ESPOCRM_CONTAINER_ADDRESS on the manager, but in swarm the espocrm container lives on a worker -> "No such container" (silently swallowed by failed_when on the earlier steps). Resolve the hosting node + local CID via resolve_host_cid and delegate_to it (also for the config-patch include), matching the erpnext pattern. Drop the now-unused ESPOCRM_CONTAINER_ADDRESS. Compose unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.
- emit emoji names at the source: cli/build/include generator now writes
"🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A role's vars/main.yml is auto-loaded into play scope and persists, so proxy
override vars (webserver_*) leaked into the central front-proxy render of OTHER
roles (re-evaluated against the wrong application_id) — silently applying one
role's value elsewhere, or hard-failing when the lazy lookup hit a role lacking
the key (the seaweedfs<->matrix client_max_body_size crash).
- Templates (sys-svc-proxy html/upload/ws_generic, matrix synapse homeserver)
resolve client_max_body_size from the role's own server config
(lookup('config', application_id, 'server.client_max_body_size', '100m')).
- Move every role-level webserver_* out of vars/main.yml into task-scoped
include params (9 roles); they apply only to that role's proxy and can't leak.
- sys-stk-front-proxy resets webserver_* to safe defaults via task-scoped vars
on the render tasks, so a leaked value from any other source is neutralized;
a role's own include params (#20) still override the reset (precedence-tested,
direct + nested).
- Add test_no_leaky_proxy_role_vars lint forbidding webserver_* in vars/main.yml
(suppressable via '# nocheck: leaky-proxy-var').
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace compose-mode-specific `{{ BIN_COMPOSE }} exec ... {{ SERVICE }}`
+ `args.chdir: lookup('container', ..., 'directories.instance')`
invocations with `container exec -i {{ ..._CONTAINER_ADDRESS | quote }}`
(plus `--workdir` flag where chdir was load-bearing). The container
exec wrapper resolves the actual runtime container name on both
compose (compose-suffixed name) and swarm (stack-task-id-suffixed
name) via the `container_address` lookup, so the task no longer
assumes the compose-mode service-name address.
Pure-syntax adaptation only — no behavioural change, same register /
when / failed_when / no_log semantics preserved on every task.
Roles touched (12 task files):
- web-app-espocrm (1)
- web-app-friendica (2)
- web-app-listmonk (1)
- web-app-openproject (2)
- web-app-snipe-it (3)
- web-app-taiga (1)
- web-app-wordpress (2)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a mode-aware `container_address` lookup plugin and a
`resolve-container-id` helper script that resolve swarm task IDs at
exec time (compose mode returns bare names unchanged). Migrates ~25
roles + ~28 task files from `command:` to `shell:` so the lookup's
`$(...)` subshell evaluates correctly, and extends `database` lookup
with an `address` key for cross-role DB exec.
Three new lint tests enforce the contract: every container exec
target goes through the resolver, lives behind an IS_STACK_HOST gate
(walks the include graph one hop deep), and uses `shell:` rather than
`command:` when the target is a Jinja reference. Existing nocheck
suppressions on bare-name DB exec sites (nextcloud redis, listmonk,
moodle OIDC, mailu) are replaced with the lookup-based form.
Fixes the systemd Environment= injection in sys-ctl-cln-disc-space
(it never expanded $(...)) by passing STACK / SERVICE_KEY parts and
resolving inside the script. Extracts shopware admin and mediawiki
extension install bodies into dedicated .sh files invoked via env.
Rewrites password-bearing shell tasks to either `| quote` directly or
the `-e VAR=... | quote` + `sh -c '... ${VAR} ...'` env-bridge pattern.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The old name suggested "is the swarm manager up?" but the variable
actually answers "is this host the one that holds the compose stack?".
It is true on the single host in compose mode and on every swarm
manager in swarm mode. The SWARM_ prefix is also misleading in compose
mode where no swarm exists.
Replace every reference across group_vars and the gating call sites
(sys-svc-compose, web-app-espocrm, web-app-bigbluebutton). Definition
itself moves to group_vars/all/18_swarm.yml line 6; the body is
unchanged so the truth value at runtime stays identical for both modes.
Everything past `sys-stk-full` in this role wrote to /opt/compose/espocrm/
or talked to the running container via `container exec` / `compose
exec`. Both only exist on the host that actually runs the espocrm
compose stack. In compose mode that is the single host; in swarm mode
only the swarm manager. Without gating, the file deploys aborted on
workers because /opt/compose/espocrm/volumes did not exist there, and
the follow-up container execs would have aborted just past that.
Move the post-stk-full task list into `02_manager_ops.yml` and include
it from main.yml with `when: SWARM_MANAGER_ACTIVE | bool`. The fact is
true on every host in compose mode (DEPLOYMENT_MODE \!= 'swarm') so
single-host deploys are unchanged. In swarm mode only the manager
runs the include, which keeps file deploys, the patch_config include,
the upgrade command and the seed runner all on the host that holds
the stack. Template rendering for compose.yml.j2 / env.j2 is already
gated by SWARM_MANAGER_ACTIVE in sys-svc-compose/tasks/04_files.yml.
Pure 1:1 rename (69 insertions / 69 deletions across 66 files) to
finish the ``docker_*`` → ``compose_*`` alignment started in
8373468e9. The flag configures whether the compose include flushes
handlers right after run; the ``docker_`` prefix was misleading
(handlers are Ansible-side, not docker-side) and the variable is part
of the same ``sys-svc-compose`` SPOT cluster as ``compose_repository_*``.
Touches the role default, the implementation
(``sys-svc-compose/tasks/utils/up.yml``), all 60+ consumer roles,
the network-utility wrapper + its README, and the
``test_network_create_via_util.py`` lint (both the docstring guidance
and the asserted diff).
Breaking change for any external override that still sets the old
name — no alias is provided.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The four `Update DB host/name/user/password (idempotent)` tasks shared
identical grep-then-sed logic, but their YAML folded-scalar layout
landed `&& echo CHANGED;` on a separate line in some cases. Inside the
container's dash, `&&` at the start of a line is a syntax error, so
`Update DB name` aborted the deploy whenever the seed value already
differed from the desired one (CI run 25634428082).
Extracted the loop body into roles/web-app-espocrm/files/patch_config_key.sh
(POSIX sh, env-driven contract: CONFIG_KEY/CONFIG_VALUE/CONFIG_FILE,
escapes \, &, |, / before the sed rewrite) and collapsed the four
tasks into a single Ansible loop that pipes the script via
`container exec -i ... sh < {{ role_path }}/files/patch_config_key.sh`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh:
- rename run_sort_claude_settings worker to run_claude (matches the
renamed scripts/lint/claude.sh).
- drop the EXIT trap that wiped status_dir; cleanup happens explicitly
after `wait` returns, so a parallel worker that interrupts the
parent never races a vanished status dir.
- each worker now captures its tool's exit code and writes one of
OK / SKIP / FAIL. Tolerated `expected violations remain`-style exits
(markdownlint-cli2 rc≤1, ansible-lint rc≤2) still count as OK; only
real failures count as FAIL.
- all workers run to completion regardless of individual tool failures
(so the user always sees the full picture); after the summary, the
script exits 1 if any worker reported FAIL.
* scripts/lint/sort_claude_settings.sh -> scripts/lint/claude.sh
(history preserved via git mv); update test_claude_settings_sorted.py
fix-hint accordingly.
Plus working-tree drift across roles/, group_vars/, inventories/,
requirements/, and .pre-commit-config.yaml accumulated from the
make-test setup phase regenerating generated artefacts during recent
test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New repository lint catches multi-line `php -r '...'` and `php <<EOF`
bodies inside *.sh / *.bash files under roles/, scripts/, .github/.
The escape-layering of bash + (template/YAML) + PHP makes such blocks
brittle and quote-injection-prone. The fix is to drop the PHP body
into a sibling files/<name>.php and pipe it via stdin (preferred from
Ansible: `shell:` with `stdin: "{{ lookup('file', '<name>.php') }}"`).
Three pre-existing violations transformed by replacing the .sh wrapper
with one or more .php sidecars and rewriting the caller task:
- web-app-espocrm: set_siteurl.sh -> set_siteurl.php
Single shell task pipes the .php via stdin; ESPOCRM_URL forwarded
through `compose exec -e` so the URL never lands in the PHP literal.
- web-app-snipe-it: ldap.sh (wrapper + 2 PHP blocks) -> 3 ansible
tasks + wait_settings.php + apply_ldap_config.php
The bash retry-loop becomes Ansible retries/delay/until; LDAP env
vars are forwarded via `compose exec -e` (not via task-level
environment: on a wrapper script). APP_KEY is preserved for
Laravel Crypt::encrypt.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Inline-script-size lint regex previously matched only bare keys
(shell:/command:/script:/raw:/run:). Extended to also catch FQCN
forms (ansible.builtin.*, ansible.legacy.*, ansible.windows.win_shell)
which are interchangeable in Ansible and must be linted identically.
Four pre-existing violations exposed and fixed by extracting the
shell bodies to files/ and calling them via ansible.builtin.script:
- sys-cleanup/main.yml (21 logical lines)
-> files/cleanup_caches.sh
- svc-db-openldap/06_migrate_005.yml (30 logical lines)
-> files/migrate_005_purge_stale.sh (4 args, no_log preserved)
- dev-nix/02_non_arch_installer.yml (24 logical lines)
-> files/aggressive_prune_nix.sh (1 arg)
- web-app-espocrm/01_patch_config.yml (13 logical lines)
-> files/set_siteurl.sh (URL via env, fixes quote injection
risk in original Jinja-into-PHP-single-quoted body)
Removed cargo-culted args.executable: /bin/bash from the four
script: tasks; the script: module ignores it (shebang controls
interpreter).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Rename lookup plugin from `docker` to `container`
- Switch all lookups from `lookup('docker', ...)` to `lookup('container', ...)`
- Normalize compose file keys:
- docker_compose → compose
- docker_compose_override → compose_override
- docker_compose_ca_override → compose_ca_override
- Centralize compose path resolution via docker_paths_utils
- Update compose_file_args to always derive paths from DIR_COMPOSITIONS
- Align all roles, templates, and vars with container/compose naming
- Update and rename unit tests accordingly
This prepares the codebase for engine-agnostic container backends
(e.g. Docker ↔ Podman) and removes legacy docker-specific assumptions.
https://chatgpt.com/share/6987a7cc-d374-800f-99ad-bb4c8f92c59e
Replace all `docker compose build` invocations with the `compose` wrapper
to ensure automatic env-file detection and override handling. Update
handlers, notify targets, direct commands, and log labels consistently
across roles.
https://chatgpt.com/share/69866b76-ce6c-800f-ac9e-dba6bba27008
- Replace lookup('docker', ..., want='...') with lookup('docker', ..., 'path')
- Allow both top-level keys (directories/files) and dotted paths
- Simplify docker lookup API and reduce lazy dict propagation
- Update unit tests to reflect new lookup semantics
https://chatgpt.com/share/6984ccea-8290-800f-a6d7-ec0e94df95a0
Remove implicit docker_compose variable usage and resolve all docker paths
(files, directories, env) via lookup('docker', application_id).
This prevents scope leakage, centralizes path resolution, and makes docker
context deterministic across roles and handlers.
https://chatgpt.com/share/6984ccea-8290-800f-a6d7-ec0e94df95a0
- add sed_escape Jinja filter for sed replacement safety
- escapes \, &, and the active sed delimiter
- designed explicitly for sed s/// replacement context
- patch all sed-based config updates across roles
- espocrm, friendica, gitea, matomo, moodle, wordpress
- ensure every Jinja value in sed replacements uses sed_escape
- use correct delimiter-specific escaping (/, |, #)
- add robust integration test to enforce sed_escape usage
- scans all roles/*/tasks/**/*.yml
- detects sed substitutions with Jinja in replacement part
- ignores delimiters inside {{ ... }} blocks
- fails CI if sed_escape is missing
- harden repository against sed injection, delimiter breakage,
and credential corruption in config patching tasks
https://chatgpt.com/share/6974fb45-1ee8-800f-80bd-c71122e4ac02
- Introduce shared CAPTCHA settings and RECAPTCHA_ENABLED/HCAPTCHA_ENABLED flags in group_vars
- Wire reCAPTCHA/hCaptcha credentials into EspoCRM, Keycloak, Listmonk and Nextcloud
- Replace EspoCRM set_flags.php with generic seed_config.php and hook it into entrypoint/env
- Fix run_once handling in sys-ctl-cln-disc-space and minor CSS Jinja spacing issue
https://chatgpt.com/share/692a1d4f-1154-800f-a4ae-bb068aa24a53
• Build a custom image and replace upstream entrypoint with docker-entrypoint-custom.sh (strict fail on flag script).
• Introduce set_flags.php and wire via ESPOCRM_SET_FLAGS_SCRIPT; apply flags at container start; clear cache afterwards.
• Keep exactly one Docker volume (data:/var/www/html/); drop separate custom/extensions mounts.
• Compose: use custom image, add healthchecks & depends_on for daemon/websocket; keep service healthy gating.
• Ansible: deploy scripts, build & up via handlers; patch siteUrl as www-data; run upgrade non-fatal; always run flag setter.
• Vars/Env: add ESPO_INIT_* toggles and ESPOCRM_SET_FLAGS_SCRIPT; refactor variables for scripts & custom image paths.
Conversation context: https://chatgpt.com/share/68dd1992-020c-800f-bcf5-2db60cb4aab2
- Replace `ESPOCRM_NAME` with `ESPOCRM_CONTAINER` for clarity and consistency.
- Drop unused `ESPOCRM_CONFIG_FILE_PUBLIC`, rely only on `config-internal.php`.
- Make DB credential patching idempotent using `grep` + `sed` checks.
- Replace direct sed edits for maintenance/cron/cache with EspoCRM ConfigWriter.
- Add fallback execution as root if www-data user cannot write config.
- Clear EspoCRM cache only when config changes and in update mode.
- Remove obsolete OIDC scopes inline task (now handled via env/vars).
- Fix docker-compose template to use `ESPOCRM_CONTAINER`.
This refactor makes the EspoCRM role more robust, idempotent, and aligned
with EspoCRM’s official ConfigWriter mechanism.
See conversation: https://chatgpt.com/share/68a87820-12f8-800f-90d6-01ba97a1b279
Use EspoCRM's ConfigWriter API to patch the 'siteUrl' setting during updates.
This makes the process idempotent, avoids brittle regex replacements, and
ensures the running configuration stays in sync with the deployment domain.
https://chatgpt.com/share/689bb860-ba90-800f-adb5-4fa5a992b267
