Commit Graph
40 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 9bc0d834fe fix(espocrm): start the sidecars past a readiness gate that can never pass
espocrm_daemon and espocrm_websocket exited 0 every six seconds while the main
container sat healthy for thirty-nine minutes, so the deploy waited out its retries
on

    not converged: espocrm_daemon espocrm_websocket
      espocrm_daemon.1 desired=Shutdown current=Complete

The service log shows our own gate working -- "[entrypoint] Exec: docker-websocket.sh"
-- and upstream's failing right after it. docker-*.sh call exitIfNotReady, which runs
`bin/command app-check` and exits 0 when that fails. CommandManager resolves the name
to an AppCheck class, and espocrm/espocrm carries no AppCheck.php at 10.0.6, so the
gate can never pass. isLegacy does not spare us either: it returns true only when
/var/www/html is itself a mount point, and we mount three subdirectories.

Only the broken half is replaced. applyConfigEnv stays, because the websocket service
is the only carrier of ESPOCRM_CONFIG_WEB_SOCKET_*, and those reach the shared
config.php nowhere else. Readiness is the caller's job already: --wait-ready blocks on
the marker the main container writes once its install has finished, which is a
strictly better-informed signal than app-check.

The logic lives in files/shell/sidecar_boot.sh rather than a bash -c string, following
the route this role already uses for docker-entrypoint-custom.sh and seed_config.php:
copied into the build context, COPYed and chmod'ed in the Dockerfile, addressed
through ESPOCRM_SCRIPT_SIDECAR_DOCKER. That buys shellcheck and bash -n coverage the
embedded string never had, drops one layer of quoting, and turns a missing argument
into a named error instead of a php invocation with no script.

ESPOCRM_APP_DIR replaces the second hardcoded /var/www/html in vars.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 15:33:40 +02:00
kevinveenbirkenbachandClaude Opus 5 575a5a87ac refactor(layout): sort shipped scripts by language and split the unit suite
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).

This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.

Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:

  * 7 `src:` values built from a role variable holding a bare filename
    (svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
    The variable doubles as the destination name inside the container, so the
    language segment belongs at the call site, not in the variable.
  * `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
    through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
    18 roles ship exactly that and have no template fallback: the play would
    have died on each.
  * `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
    interface now reads `files/javascript/javascript.js`.
  * `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
    path through `path_absolute(... 'files', ...)`.
  * 4 `ansible.builtin.script:` invocations naming a bare command
    (pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
  * `dev-python/files/shell/install.sh` pointed at a sibling role's script.

Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 01:31:09 +02:00
kevinveenbirkenbachandClaude Opus 5 5137465c6f style: delete every comment the comments-valid policy rejects
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.

Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 03:22:02 +02:00
kevinveenbirkenbachandClaude Opus 5 642055a20a style(e2e): drop the timeout argument isVisible discards
Playwright declares the timeout option on locator.isVisible() and isHidden() as deprecated and ignored: both return immediately and never wait. Passing resolveTimeout(10_000) to one therefore changes nothing at runtime and only tells the reader a wait happened that never did. That reading is what let 7538b8a0b conclude two genuine call sites were fine, and what hid the false negative b97cf8888 had to fix.

Sixty call sites across fifty-eight role-local specs lose the argument and nothing else. Every added line in this commit is a bare isVisible(); a grep for added lines that are not one returns zero, so a behaviour change cannot have entered here. Files whose diff also alters waiting semantics are deliberately not part of this commit.

A repository lint added in b97cf8888 keeps the construct out. It matches isVisible and isHidden only, because isChecked, isDisabled, isEditable and isEnabled take a real timeout.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 13:49:18 +02:00
kevinveenbirkenbachandClaude Opus 5 a77bc07d4a fix(espocrm): take the original entrypoint path from the spot
The path to the image entrypoint the build patches was written out four times in the Dockerfile, and /usr/local/bin twice more in vars. It now comes from ESPOCRM_BIN_DIR through ESPOCRM_ENTRYPOINT_SCRIPT_ORIGINAL, declared as a build ARG after FROM and passed from the compose build args. The runtime side already reads the same value from the environment, because a build ARG does not exist at runtime.

The sed expressions themselves are byte-identical. Only the file operand changed, and the variable sits outside the single quotes, so the delimiter and ampersand escaping that e272ae2d0 had to correct is untouched.

Not exercised: the four sed and grep pairs are build gates and there is no cheap target that builds a role image. A missing ARG would expand to nothing and leave sed without a file operand, which exits 4 with no input files, so the failure is loud at that layer rather than a silently broken image.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 13:46:31 +02:00
kevinveenbirkenbachandClaude Opus 5 c2018354ab fix(espocrm): set the config keys the app actually reads
Two settings never reached EspoCRM. The image entrypoint maps a double underscore to a dot and a single one to camelCase, so ESPOCRM_CONFIG_LOGGER_PATH wrote a flat loggerPath that nothing reads, while the real key is nested under logger. The level therefore stayed at the WARNING default, which is why Internal server error - Check logs for details had no logs behind it. The value php://stdout was never in effect either, and that was luck: EspoFileHandler raises RuntimeException on it and EspoCRM installs no exception handler, so making the key live with that value would have turned a working file logger into a throwing one. The path and rotation are now written explicitly at the values that were already in effect.

LdapLogin builds its search as (&(objectClass=X)(userNameAttribute=user)(loginFilter)) - the filter is ANDed on top, it does not replace anything. ldapUserNameAttribute defaults to sAMAccountName, an Active Directory attribute, while this directory is OpenLDAP with uid, so the previous filter produced a term that could never match. Setting the attribute is the knob that decides matching; web-app-semaphore sets both for the same reason.

ldapCreateEspoUser decides whether a user who authenticated against LDAP but has no EspoCRM record gets one created. Without it such a login is refused with USER_NOT_FOUND. It is exposed as services.ldap.autocreate rather than hardcoded, because it shifts who may enter from an explicit EspoCRM account to directory membership; the created user gets no team and no role.

The log directory now has one spot in vars, and the entrypoint derives both the streamed filename and the init diagnostics from the same value. It streams data/logs/<stem>-<Y-m-d>.log to stdout so the rescue collector picks it up unchanged; the date is re-evaluated because EspoRotatingFileHandler renames at midnight and a single tail would go silent there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 13:34:20 +02:00
kevinveenbirkenbachandClaude Opus 5 6db7c4ad6f fix(espocrm): invoke the image entrypoint with a command it acts on
The wrapper calls the image entrypoint purely for its side effect --
install and migrate -- and then starts its own server, so it passed
/bin/true as the command to mean "do the init, start nothing". But
upstream gates that init on the command name:

    if [[ "${1-}" == apache2* ]] || [ "${1-}" = 'php-fpm' ]; then
        start
        applyConfigEnv
    fi
    exec "$@"

/bin/true matches neither arm. The whole block was skipped and
exec /bin/true returned 0, so a complete no-op looked like a successful
init. No install ran, data/config.php was never written, and the failure
surfaced two steps later as the misleading "seed_config.php execution
failed".

Pass 'apache2 -v' instead: it matches apache2*, so start runs, and the
trailing exec "$@" only prints the version banner and exits 0 -- hence
the added >/dev/null, which keeps that banner out of the wrapper's
stdout.

Fail loudly when data/config.php is still missing afterwards, so a
regression of this shape is reported at its source, and surface the
throwable from $writer->save() with class, message, file and line
instead of a bare fatal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 00:50:48 +02:00
kevinveenbirkenbachandClaude Fable 5 d88f272f9f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 15:31:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4c247b61ec test(e2e): give every standalone Playwright request an onion-aware timeout
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 04:45:03 +02:00
kevinveenbirkenbachandClaude Fable 5 e272ae2d04 fix(espocrm): correct sed delimiter and ampersand escape in entrypoint patch
The Dockerfile RUN that appends '|| true' to the upstream create-admin-user line used '|' as the sed delimiter, which collides with the '||' in the replacement (sed: unknown option to s), and left the '&' in '2>&1' unescaped (sed expands it to the whole match). Use '#' as the delimiter and escape the ampersand as '\&'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 01:12:29 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3a6c181dc8 fix(espocrm): tolerate an existing admin during upstream first-boot install
The upstream v10 entrypoint runs create-admin-user unguarded under
set -euo pipefail; on a persistent database that already holds the
admin (redeploy, DR recovery) it exits non-zero and aborts the install
before isInstalled is persisted, so every boot re-runs the install and
crash-loops. Append '|| true' the same way the adjacent set-password
line is already patched; the grep guard fails the build if upstream
moves the line.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 16:15:27 +02:00
kevinveenbirkenbachandClaude Opus 4.8 07c6ad8e7c test(e2e): scale every nav click + timeout literal for onion targets
Over Tor the redirect chain behind a login/logout/OIDC click routinely exceeds Playwright's fixed 30s action timeout, so the click throws before the flow's scaled waits run. Sweep all role Playwright suites and the shared persona helpers: give every navigation/auth-transition click an onion-scaled timeout (.click({ timeout: resolveTimeout(30_000) })) and route the remaining raw millisecond literals (timeout options, waitForTimeout, sleep-promise setTimeout, positional wait args) through resolveTimeout. Local UI clicks (menus, toggles, modal dismissals, DOM clicks inside evaluate) stay untouched.

Two new lints keep it that way: test_no_raw_timeout_literals forbids raw numeric timeout literals in Playwright specs (opt-out: nocheck: raw-timeout), test_persona_clicks_scaled requires every click in the shared persona auth flows to carry a resolveTimeout timeout (opt-out: nocheck: unscaled-click).

Verified: node --check on all 133 changed JS files, all 13 playwright lints green, make autoformat clean, make test green (2745+196+163+4 tests OK).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 11:35:53 +02:00
kevinveenbirkenbachandClaude Fable 5 aaa50ab02d fix(espocrm): satisfy shellcheck in the entrypoint log tail
Replace the ls -t parse with a find/sort pipeline (SC2012) and the
A && B || C construct with a real if-else (SC2015): a failing tail no
longer prints the misleading '(no app log found)', and '|| true' keeps
set -eu from swallowing the intended exit code in the failure path.
Verified the pipeline in the espocrm:10.0.2 image including the
empty-directory case; blocked the pre-push shellcheck gate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 03:15:53 +02:00
kevinveenbirkenbachandClaude Fable 5 842ad7e356 fix(espocrm): patch hardcoded set-password admin in the v10 entrypoint
Upstream actionInstall() runs create-admin-user "$ESPOCRM_ADMIN_USERNAME"
but then pipes the password to a hardcoded 'set-password admin' (line 203,
verified in the 10.0.2 image); with our admin named 'administrator' the
silenced failure aborts the install under set -euo pipefail and loops the
container. Patch the line at image build time; the grep guard fails the
build if a future image reshapes it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 03:15:53 +02:00
kevinveenbirkenbachandClaude Fable 5 e018e2bdf0 debug(espocrm): surface silenced image entrypoint init failures
The v10 image entrypoint silences several install commands
(create-admin-user, set-password run under set -e with all output
redirected), so a failing init crash-loops the swarm service with
nothing in docker logs. Trace the init to a file, mirror the upstream
info/warning/error lines, and dump the trace tail plus the newest app
log to stderr when init exits non-zero.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 03:15:52 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e3d33224eb fix(espocrm): gate daemon and websocket behind the boot ready marker
Swarm ignores depends_on, so the daemon and websocket replicas started
running PHP while the first-boot installer was still rebuilding the shared
NFS volume. Any PHP run mid-install leaves NFS silly-rename files in
data/cache that make the installer's rm/rebuild fail, killing the deploy.

Teach the custom entrypoint a --wait-ready <cmd> mode that blocks until the
existing boot ready marker appears (ESPOCRM_WAIT_READY_TIMEOUT_S, default
1800s, then a loud exit 1) and exec the daemon/websocket commands through
it. After the first boot the marker persists, so the gate is pass-through
on every later start in both compose and swarm.

Also drop an inline comment that leaked into the ESPOCRM_CONFIG_OIDC_FALLBACK
env value; env files have no inline comments, so the setting never read as
plain false.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 03:04:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 2944a7fc5d fix(espocrm): bake code node-local, share only mutable dirs over NFS
The stock image lays the app under /var/www/html from /usr/src at runtime, so the whole webroot landed on the shared NFS volume; on kernel-nfsd the www-data Apache workers on a non-populating swarm node get EPERM serving that root-owned code (403), and the stack never converges. Bake the code into the image at build time so /var/www/html is node-local on every replica, and share only data/, custom/ and client/custom over NFS. Move the boot-lock and ready-marker into the shared data/ so replicas still serialize the one-time install, keep the ready-marker check ahead of the lock (a serving replica must not re-enter the image entrypoint and rebuild the shared cache, which aborts 255), and chown the three shared dirs. Verified in compose (deploy + hard-restart + e2e); swarm eliminates cross-node code access by construction.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 00:03:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4c91df5de7 test(e2e): forbid raw page.goto, migrate all specs to Tor-resilient gotoOnion
New lint test_no_raw_goto.py forbids raw `page.goto(...)` in role playwright specs + persona flows (the gotoOnion wrapper in personas/utils is exempt) so navigation retries transient Tor-transport errors over onion. Migrates every raw call across ~75 roles (343 sites) to `gotoOnion(page, ...)` and teaches gotoOnion to detect relative onion URLs (`isOnionCanonical`). gotoOnion is a behaviour-identical drop-in on clearnet (single attempt, same return). Verified: no-raw-goto lint green, eslint 0 errors, make test green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:59:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 35dee215c8 test(playwright): route all timeouts through resolveTimeout for onion-compat
Add the global TIMEOUT_FACTOR and a shared timeouts.js helper: resolveTimeout(base) scales by TIMEOUT_FACTOR and, over a .onion target, an extra multiplier, so Tor's per-request latency no longer trips hard-coded timeouts. Migrate every raw timeout in the specs (timeout:/timeout=/setTimeout/waitForTimeout) to resolveTimeout, stage the helper in all three staging paths, and add test_no_raw_timeouts to forbid raw numeric timeouts. Clearnet behaviour is unchanged (factor 1: resolveTimeout(N) === N).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 10:38:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 52a7adadd8 fix(espocrm): create the session dir before the image init
The image installer itself calls session_start (install/cli.php); with the conf.d save_path pointing at data/.sessions, which the entrypoint only created after init, the installer lost its session state on every invocation and aborted with 'No database name specified in config'. Create and chown the session dir inside the lock before running the image entrypoint.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 02:37:27 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9ecb2f8f5e test(lint): cover Dockerfiles in the comments lint
Dockerfiles have no suffix, so the comments lint never scanned them. Match the Dockerfile basename in the changed-targets collector and dispatch it to the hash-comment parser; changed-files-only semantics stay, so the legacy tree is unaffected until a Dockerfile is touched. Drop the remaining prose comment from the espocrm Dockerfile it immediately flagged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 02:37:27 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e7ee192162 refactor(espocrm): full multi-replica conversion
Drop the replicas:1 pin and make the web tier genuinely scalable. The custom entrypoint now runs the image entrypoint's install/upgrade/config-seed exactly once cluster-wide: an atomic mkdir mutex on the shared volume root (flock is node-local on this NFS mount), the image init invoked with /bin/true so its final exec returns control, a ready marker for followers, stale-lock takeover past the orchestrator kill ceiling and TERM/INT release; every replica then starts apache itself. Previously the image entrypoint execed apache directly, so the seed/clear-cache block was dead code - it now runs as intended. PHP sessions move to the NFS-shared data/.sessions via a conf.d ini so OIDC/SAML handshakes survive replica hops. The cron daemon and the websocket service become explicitly pinned singletons (they only inherited the entity pin before). A missing image entrypoint after a version bump now fails loudly (upstream master restructured the entrypoint; 9.3.10 is verified). Converge retries 600, healthcheck start_period 15m, seeder debug env renamed out of the ESPOCRM_SEED_ config prefix scan.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 02:37:26 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e986fe5d9e refactor(test-e2e): route HSTS assertions through the shared TLS-gated helper
HSTS (Strict-Transport-Security) is an HTTPS-only header: a browser ignores it
over plain http (RFC 6797) and OpenResty only emits it on the TLS vhost. On an
onion node TLS is off and every domain is served over http, so the inline
`expect(headers["strict-transport-security"]).toBeTruthy()` in 33 specs failed
with `Received: undefined`.

Route all 33 through the shared expectHstsWhenTls(headers, baseUrl, label)
helper (personas/utils/tls.js), which requires HSTS over https and asserts its
absence over http. On a TLS node the base URL is https, so the assertion is
identical to before; on an onion node the same spec now passes. HSTS logic
lives in one place (SPOT) instead of 34 inline copies.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 03:26:03 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c2c532c8e6 refactor(playwright): centralize dashboard-tile mechanics in web-app-dashboard SPOT; move specs into files/playwright/ subdir
Architecture
============
- New SPOT helper `roles/web-app-dashboard/files/playwright/dashboard-card-flow.js` (`runDashboardCardScenario(page, context, target)`): per `roles_with_service('dashboard')` target, asserts tile visible → click loads in `#main iframe` (outer URL gets `?iframe=` + iframe src matches canonical_domain) → header "Tab" button pops the embed into a fresh browser tab.
- `web-app-dashboard/files/playwright/playwright.spec.js` parametrised loop calls the helper for every `DASHBOARD_TARGET_ROLES_JSON` entry — the single owner of dashboard-tile mechanics across the whole codebase.

Spec relocation
===============
- 71 `roles/<role>/files/playwright.spec.js` → `roles/<role>/files/playwright/playwright.spec.js` (`git mv`, history preserved). odoo + pixelfed show as D+A in `git status` because of substantial content rewrites; coverage retained.
- New SPOT constant `ROLE_FILE_PLAYWRIGHT_SPEC` in `utils/roles/mapping.py`, registered as optional on `ROLE_TYPE_APPLICATION` (disallowed elsewhere, mirrors `ROLE_FILE_META_VARIANTS` pattern).
- `roles/test-e2e-playwright/tasks/run_one.yml`: staging task replaced with `with_fileglob` over `files/playwright/*.js` — roles can ship companion helpers (like `dashboard-card-flow.js`) next to the spec without further wiring.

Spec migrations (no more dashboard-iframe path in app specs)
============================================================
13 specs rewritten so the persona/SSO journey starts at the app URL directly, never through the dashboard tile or iframe:
* fider, mailu, mattermost, nextcloud, odoo, taiga: iframe-wrap removed (no more `frameLocator("#main iframe")`)
* bigbluebutton, discourse, kix, matrix, openwebui, peertube: dashboard goto preamble removed
* pixelfed: ~270 LOC of iframe-aware helpers/login/logout collapsed to direct-URL flow

Title cleanup
=============
- 64 specs: `"<persona>: dashboard → app → universal logout"` / `"administrator: dashboard → prometheus → app → universal logout"` → `"<persona>: app → universal logout"`.
- 13 specs: `"<persona>: dashboard to <app> ..."` → `"<persona>: <app> ..."`.

Dead-code cleanup
=================
- `dashboardBaseUrl` consts + beforeEach assertions removed from 11 specs (no functional uses; personas helper handles env validation).
- `DASHBOARD_BASE_URL=` env-template lines removed from fider, mattermost, odoo, taiga, pixelfed, kix.
- `DASHBOARD_SERVICE_ENABLED=` env-template lines removed (pixelfed retained for the still-gated spec).
- Stale "loads inside the iframe" / "FrameLocator parameter" comments cleaned in fider, odoo, prometheus, nextcloud.

Tooling
=======
- 5 lint Python files + 1 integration test now use `from utils.roles.mapping import ROLE_FILE_PLAYWRIGHT_SPEC`.
- `eslint.config.js` glob pattern + `roles/test-e2e-playwright/meta/main.yml` description + `roles/test-e2e-playwright/README.md` updated to new path.
- 25 markdown/docstring references in `docs/`, `roles/**/README.md`, `roles/**/TODO.md`, persona-helper docstrings updated to new path.
- `docs/contributing/artefact/files/role/playwright.specs.js.md`: file-placement section now documents the `files/playwright/` convention + `ROLE_FILE_PLAYWRIGHT_SPEC` SPOT constant; dashboard SPOT delegation explicit at 3 places.

Net diff: 124 files, +1142/-1564 (−422 LOC).
Integration suite: 183/183 green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 23:24:55 +02:00
kevinveenbirkenbachandClaude Opus 4.7 70318a7172 fix(web-app-espocrm/patch_config): extract idempotent PHP-config patcher
The four `Update DB host/name/user/password (idempotent)` tasks shared
identical grep-then-sed logic, but their YAML folded-scalar layout
landed `&& echo CHANGED;` on a separate line in some cases. Inside the
container's dash, `&&` at the start of a line is a syntax error, so
`Update DB name` aborted the deploy whenever the seed value already
differed from the desired one (CI run 25634428082).

Extracted the loop body into roles/web-app-espocrm/files/patch_config_key.sh
(POSIX sh, env-driven contract: CONFIG_KEY/CONFIG_VALUE/CONFIG_FILE,
escapes \, &, |, / before the sed rewrite) and collapsed the four
tasks into a single Ansible loop that pipes the script via
`container exec -i ... sh < {{ role_path }}/files/patch_config_key.sh`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:06:32 +02:00
kevinveenbirkenbachandClaude Opus 4.7 0e651e4a22 chore(lint): tighten ESLint rules and clean up flagged offenders
Promote four cheap rules in `eslint.config.js`:
* `no-unused-vars`: warn → error (with `argsIgnorePattern: "^_"`)
* `no-var`: error
* `eqeqeq`: error (always)
* `playwright/no-wait-for-navigation`: off → error
* `linterOptions.reportUnusedDisableDirectives`: off → warn

Add a repo-root CommonJS block so `eslint.config.js` itself sees Node
globals (`require`, `module`).

Fix every flagged offender:
* 138 stale persona-helper imports across 70 spec files
  (`isServiceEnabled`, `skipUnlessServiceEnabled`, `runAdminFlow`,
  ... — leftovers from the SPOT consolidation).
* 4 unused catch bindings → modern `catch {}` syntax.
* 2 unused `dashboardBaseUrl` declarations (minio, moodle).
* 1 unused `label` arg → renamed to `_label` to opt out per the
  argsIgnorePattern.
* 1 intentionally-kept legacy helper in web-app-wordpress's spec
  receives a per-line `eslint-disable-next-line` with reason; the
  preceding docstring already explained why it stays.
* 1 `page.waitForNavigation` in web-app-fediwall's Mastodon OIDC flow
  receives a per-line disable with reason — the navigation target
  depends on the runtime OIDC issuer; `waitForURL` would need a
  runtime-built pattern.
* 10 stale `// eslint-disable-next-line no-console` directives now
  surfaced by `reportUnusedDisableDirectives`; removed.

ESLint after this pass: 0 errors, 154 warnings (148 pre-existing
`playwright/expect-expect` plus 6 `playwright/no-force-option` that
were already inherited from `playwright/flat/recommended`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 16:25:23 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c55ad69c41 refactor: drop requirement-file references from code, add doc rule + lint
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`,
`requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from
non-Markdown source files. Comments anchored to requirement numbers
rot when requirements are renumbered or rewritten and conflate
present-state behaviour (which the comment should describe) with the
audit trail of how the code got there (which belongs in commit
messages, PR descriptions, and the requirement file itself).

* Document the rule in `docs/contributing/documentation.md` under
  "Comments". Code MUST describe the *why* in present-state language;
  contract enforcement lives in tests, not folklore.
* Clean 254 source files across `roles/`, `tests/`, `plugins/`,
  `cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root
  `eslint.config.js`. Pure comment / docstring / log-message edits;
  no behavioural change.
* Add `tests/lint/repository/test_no_req_references_in_code.py` to
  enforce the rule going forward across `.js`, `.py`, `.yml`,
  `.yaml`, `.j2`, and `.sh` files. The legacy
  `test_no_stale_requirement_refs.py` is exempt as the only file
  whose purpose is to detect these very tokens.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 16:25:22 +02:00
kevinveenbirkenbachandClaude Opus 4.7 122c82910b refactor(playwright): share decodeDotenvQuotedValue + normalizeBaseUrl
Both helpers were inlined verbatim in 71 of the role-local
`playwright.spec.js` files (52 with the compact form, 19 with the
verbose form, plus one parameter-name variant). All bodies were
semantically identical.

Changes
-------
- New `roles/test-e2e-playwright/files/personas/utils/dotenv.js` holds
  the canonical pair (`decodeDotenvQuotedValue` and `normalizeBaseUrl`),
  re-exported through `personas/utils/index.js` so specs that
  `require("./personas")` get them via the existing aggregator.
- Every role spec drops its inline copies and pulls the helpers from
  `./personas` instead. Specs that only reference one of the two
  carry only that one in the destructuring (no unused-import noise).

Doc
---
`docs/contributing/artefact/files/role/playwright.specs.js.md` no
longer carries the "once a third spec needs them, MUST be promoted"
directive: the helpers live at the shared path and inline copies are
forbidden.

Verification
------------
- 419 passed / 1 skipped / 0 failed across `tests/lint/`,
  `tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors, 283 warnings (unchanged from
  baseline; no new unused-imports introduced).
- 0 inline `function decodeDotenvQuotedValue|normalizeBaseUrl`
  definitions remain in any role spec.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 14:55:05 +02:00
kevinveenbirkenbachandClaude Opus 4.7 32d7a62ad0 test(playwright): hard-fail on missing role spec; seed 27 minimal specs
Make `tests/lint/.../test_has_spec.py` raise instead of warn when a
roles/web-app-* role has no `files/playwright.spec.js`, then ship the
27 minimal specs needed to satisfy the new floor.

Each new spec follows `roles/web-app-mig` (TLS + canonical-domain
baseline, HTML content-type, three persona flows) with role-specific
admin-interaction labels seeded from app domain knowledge. The bodies
are intentionally static — `templates/playwright.env.j2` is not
shipped per role yet, so the specs do not run end-to-end on deploy;
detail tightening per role is follow-up work.

Also fix `_body_contains_stub_marker` in `test_no_stub_tests.py`: it
substring-matched `TODO` against `body.upper()`, which flagged
`masTODOn` inside the mastodon spec's body regex. Switched to
word-boundary matching so genuine `// TODO:` markers still trip the
lint while embedded substrings in app names or regex alternations do
not.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 12:06:50 +02:00
kevinveenbirkenbachandClaude Opus 4.7 5689f5b636 refactor(lint): forbid multi-line PHP in *.sh + transform 3 violators
New repository lint catches multi-line `php -r '...'` and `php <<EOF`
bodies inside *.sh / *.bash files under roles/, scripts/, .github/.
The escape-layering of bash + (template/YAML) + PHP makes such blocks
brittle and quote-injection-prone. The fix is to drop the PHP body
into a sibling files/<name>.php and pipe it via stdin (preferred from
Ansible: `shell:` with `stdin: "{{ lookup('file', '<name>.php') }}"`).

Three pre-existing violations transformed by replacing the .sh wrapper
with one or more .php sidecars and rewriting the caller task:

  - web-app-espocrm: set_siteurl.sh -> set_siteurl.php
    Single shell task pipes the .php via stdin; ESPOCRM_URL forwarded
    through `compose exec -e` so the URL never lands in the PHP literal.

  - web-app-snipe-it: ldap.sh (wrapper + 2 PHP blocks) -> 3 ansible
    tasks + wait_settings.php + apply_ldap_config.php
    The bash retry-loop becomes Ansible retries/delay/until; LDAP env
    vars are forwarded via `compose exec -e` (not via task-level
    environment: on a wrapper script). APP_KEY is preserved for
    Laravel Crypt::encrypt.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 09:26:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 311bbfc809 refactor(lint): catch FQCN shell blocks + extract 4 violators
Inline-script-size lint regex previously matched only bare keys
(shell:/command:/script:/raw:/run:). Extended to also catch FQCN
forms (ansible.builtin.*, ansible.legacy.*, ansible.windows.win_shell)
which are interchangeable in Ansible and must be linted identically.

Four pre-existing violations exposed and fixed by extracting the
shell bodies to files/ and calling them via ansible.builtin.script:

  - sys-cleanup/main.yml (21 logical lines)
    -> files/cleanup_caches.sh
  - svc-db-openldap/06_migrate_005.yml (30 logical lines)
    -> files/migrate_005_purge_stale.sh (4 args, no_log preserved)
  - dev-nix/02_non_arch_installer.yml (24 logical lines)
    -> files/aggressive_prune_nix.sh (1 arg)
  - web-app-espocrm/01_patch_config.yml (13 logical lines)
    -> files/set_siteurl.sh (URL via env, fixes quote injection
       risk in original Jinja-into-PHP-single-quoted body)

Removed cargo-culted args.executable: /bin/bash from the four
script: tasks; the script: module ignores it (shebang controls
interpreter).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 09:26:33 +02:00
Environment TestandClaude Sonnet 4.6 1db58af7fd fix(docker): replace ENV-based ENTRYPOINT with RUN symlink in espocrm and suitecrm
The ENV approach failed at runtime because the ca-trust-wrapper does not
propagate container ENV variables when executing the wrapped ENTRYPOINT,
causing $ESPOCRM_ENTRYPOINT_SCRIPT_DOCKER / $SUITECRM_ENTRYPOINT_SCRIPT_DOCKER
to remain unexpanded ("not found" error in CI).

Replace with a RUN-time symlink (ln -sf) so the dynamic ARG path is baked
into the image filesystem layer. ENTRYPOINT now uses the fixed /entrypoint.sh
path, which is independent of ENV propagation and cache state.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 21:48:18 +02:00
Environment TestandClaude Sonnet 4.6 1c026ce6ef fix(docker): remove quoted FROM and fix ENTRYPOINT ARG expansion in 4 roles
- Remove quotes from FROM instruction in confluence, jira, suitecrm, espocrm Dockerfiles
  (Docker does not strip quotes — quoted image references are invalid)
- Promote ENTRYPOINT_SCRIPT_DOCKER ARG to ENV in espocrm and suitecrm so the
  runtime shell can resolve the path (ARG is not expanded in ENTRYPOINT exec form)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 12:40:28 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 f5a81736cf refactor(roles): convert Dockerfile.j2 templates to plain files/Dockerfile
Replace Jinja2 Dockerfile templates with static Dockerfiles using Docker
ARG statements. Variables are now passed as build args in compose.yml.j2,
following the existing pattern from web-app-bridgy-fed. Affects 12 roles:

- svc-db-openldap
- web-app-bookwyrm
- web-app-confluence
- web-app-espocrm
- web-app-jira
- web-app-joomla
- web-app-mediawiki
- web-app-pretix
- web-app-shopware
- web-app-suitecrm
- web-app-xwiki
- web-svc-simpleicons

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 00:28:37 +02:00
kevinveenbirkenbach 5e097c02f1 fix(shellcheck): address lint findings across scripts
- openldap: use find instead of ls for module path detection
- docker-volumes health: robust whitelist parsing + quote vars
- btrfs health: quote path
- docker CLI installer: silence SC2031 false-positive for /etc/os-release vars
- test-dns: use loop counter
- chess: require APP_KEY_FILE env
- espocrm: log normalized seed flags to avoid unused vars
- shopware: disable SC2016 for php -r blocks (PHP vars in single quotes)
- fix script: add shebang + strict mode

https://chatgpt.com/share/69712026-7288-800f-b1d0-701e31754362
2026-01-21 19:51:12 +01:00
kevinveenbirkenbach bcb30ff164 feat(espocrm): run original entrypoint before custom logic and hard-guard bootstrap
Run the upstream EspoCRM docker-entrypoint.sh first to preserve the original
installation and webroot initialization logic. Afterwards apply custom
seed/patch logic with a strict bootstrap.php guard to fail fast on invalid
volume mounts.

Also harden daemon healthcheck to avoid fatal bootstrap crashes.

https://chatgpt.com/share/696e4abd-0478-800f-ba18-7e149502d5c5
2026-01-19 16:15:55 +01:00
kevinveenbirkenbach 654131ab89 Add global CAPTCHA config and EspoCRM seeder
- Introduce shared CAPTCHA settings and RECAPTCHA_ENABLED/HCAPTCHA_ENABLED flags in group_vars
- Wire reCAPTCHA/hCaptcha credentials into EspoCRM, Keycloak, Listmonk and Nextcloud
- Replace EspoCRM set_flags.php with generic seed_config.php and hook it into entrypoint/env
- Fix run_once handling in sys-ctl-cln-disc-space and minor CSS Jinja spacing issue

https://chatgpt.com/share/692a1d4f-1154-800f-a4ae-bb068aa24a53
2025-11-28 23:08:32 +01:00
kevinveenbirkenbach 2132356f02 Enable custom EspoCRM CSS with Infinito.Nexus color scheme
- Activate css feature flag for EspoCRM
- Extend CSP with frame-ancestors for PRIMARY_DOMAIN
- Add style.css mapping EspoCRM UI to the --color-01-** blue palette

See ChatGPT discussion: https://chatgpt.com/share/6929b521-3a90-800f-9ce7-29bfb1faaeb4
2025-11-28 15:44:15 +01:00
kevinveenbirkenbach 80e048a274 Fix: make EspoCRM entrypoint POSIX-compliant and remove illegal 'pipefail' usage
The previous entrypoint script used 'set -euo pipefail', which caused runtime errors
because /bin/sh (Dash/BusyBox) does not support 'pipefail'. This commit makes the
entrypoint fully POSIX-safe, adds robust fallbacks for missing scripts, and improves
logging. Also removes a trailing newline in the navigator Docker Compose template.

Related ChatGPT discussion: https://chatgpt.com/share/68eab0b7-7a64-800f-a8aa-e7d7262a262e
2025-10-11 21:33:07 +02:00
kevinveenbirkenbach e7702948b8 EspoCRM role: custom image + single data volume + runtime flag setter
• Build a custom image and replace upstream entrypoint with docker-entrypoint-custom.sh (strict fail on flag script).

• Introduce set_flags.php and wire via ESPOCRM_SET_FLAGS_SCRIPT; apply flags at container start; clear cache afterwards.

• Keep exactly one Docker volume (data:/var/www/html/); drop separate custom/extensions mounts.

• Compose: use custom image, add healthchecks & depends_on for daemon/websocket; keep service healthy gating.

• Ansible: deploy scripts, build & up via handlers; patch siteUrl as www-data; run upgrade non-fatal; always run flag setter.

• Vars/Env: add ESPO_INIT_* toggles and ESPOCRM_SET_FLAGS_SCRIPT; refactor variables for scripts & custom image paths.

Conversation context: https://chatgpt.com/share/68dd1992-020c-800f-bcf5-2db60cb4aab2
2025-10-01 14:08:09 +02:00