Commit Graph
52 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 a46348b6cf refactor(handlers): bind notify to a listen topic, not to a name
A handler name was doing two jobs at once: the line an operator reads in the
play output, and the identifier a `notify:` binds to. The two pull in opposite
directions -- the log wants "restart the SSH service, with diagnostics", the
binding wants a token that survives quoting, YAML flow style and a shell
round-trip -- and a name that changes for readability silently changes which
handler runs.

Every handler that a notify reached by name now carries a `listen:` with that
exact wording, so the binding is explicit before anything is renamed. The
topics themselves become identifiers: `compose up` reads `compose-up`,
`Restart NFS ganesha` reads `restart-nfs-ganesha`, 22 of them across both
sides of every notify. Where the topic is assembled at runtime, both branches
moved together: the conditional in sys-service carries two literals, and the
LDAP renderer interpolates `import-{{ folder }}-ldif-files`, which is what
makes those handlers reachable at all.

Freed from the binding, the names take the emoji prefix every other task line
already has.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 03:38:42 +02:00
kevinveenbirkenbachandClaude Opus 5 cd1225d2d9 fix(async): collect fire-and-forget results
The repo's async convention is fire and forget: ASYNC_POLL is 0 whenever
ASYNC_ENABLED is true, and the swarm test lane forces it true through
extra vars. A task dispatched that way returns a stub, so a changed_when
or failed_when reading its own register evaluates against nothing and can
never report changed and never fail. Several tasks carried a
"not ASYNC_ENABLED and" prefix in front of those conditions, added to
avoid touching undefined rc and stdout, which turned the guard off
entirely rather than adapting it.

That is how the nextcloud plugin configuration stayed broken unseen: the
batch reported ok after 0.8s while its delegated commands kept running,
the next task read the half written configuration, and no failure in any
of them could surface.

Every async in the repo was decided individually rather than swept. Where
nothing consumes the result, the async stays and a one line rationale now
records what does not read it. Where a later task, a handler or a browser
does consume it, either the async goes or the result is collected.

Collecting is what the new reaper does. It waits for a whole batch, then
reports every broken job at once instead of aborting on the first, so one
run shows whether a single item or the whole set is broken. Deciding what
counts as broken lives in a filter plugin, because the distinction is
subtle enough to deserve tests: a module reports failure in msg, a command
in rc and stdout, an idempotent API answers "already exists" and means
success, and a job that cannot be read at all comes back marked finished
with no outcome, which would otherwise read as green.

Timing cost is roughly one to two minutes on a deploy measured in hours.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 04:05:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6ed44cb9be fix(web-app-taiga): don't let the manager re-apply migrations in compose
taiga-back applies migrations from its own entrypoint on every boot. The
manager sidecar then ran 'manage.py migrate' too, and in compose there
is no converge wait, so the two applied the same migrations at once and
aborted with a duplicate-object error ("column ... already exists").
Swarm passed because its converge wait means taiga-back is already
migrated before the manager runs. Make the manager only wait for the
migrations to be applied ('migrate --check', read-only, retried) instead
of applying them, so taiga-back stays the sole migrator. No-op in swarm
where the check passes immediately.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 21:41:43 +02:00
kevinveenbirkenbachandClaude Fable 5 d7b1260aec fix(web-app-taiga): deploy the manager sidecar from the real compose file
The manager exec-sidecar (migrations/admin via container exec) was only
defined in compose-inits.yml.j2, which was rendered to disk but never
passed to docker: compose deploys failed with 'No such container:
taiga-manager' and swarm burned the full converge wait polling the
never-created taiga_manager service in all three variants.

Move the service block into compose.yml.j2 (joining the taiga overlay
like every sibling service, pinned to a single replica) and drop the
orphan template, its render task and the unused path var. The sidecar
is a sleep-infinity shell; ops still run only via the targeted ansible
exec, so nothing self-executes regardless of scheduling.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 15:34:28 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9789f22583 fix(web-app-taiga): resolve gateway upstreams by swarm-compatible service name
The gateway nginx hardcoded compose container_names (taiga-front/back/...) that
do not exist as DNS names in swarm, so it crash-looped on [emerg] host not found
in upstream. Point the upstreams at the service keys (which resolve in both
compose and swarm), render the role gateway config unconditionally (gating only
the OIDC location block), pin the protected service name to its SPOT var, and
add the back service to the async depends_on so it waits for the shared image.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 00:26:05 +02:00
kevinveenbirkenbachandClaude Opus 4.8 100c93e7bc fix(web-app-taiga): keep manager container alive and always render conf.json
The taiga-manager entrypoint 'python manage.py' exited immediately (no subcommand), crash-looping so migration exec hit 'No such container'. Give it a durable entrypoint and invoke django via 'python manage.py <cmd>'. Also render the frontend conf.json unconditionally since compose mounts it always.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 14:28:41 +02:00
kevinveenbirkenbachandClaude Opus 4.8 95f414d196 fix(swarm): run in-container exec/inspect on the task's hosting node
container exec/inspect ran on the swarm manager while the task may run on
a worker, yielding No such container or an empty container id. Use the
existing resolve_host_cid helper (HOST_NODE + LOCAL_CID + delegate_to)
for the ollama model pre-pull, the taiga admin migrations and the
peertube post-backfill health inspect, matching the fider/matomo/decidim
pattern. The helper resolves to the local host + compose cid in compose
mode, so that path is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 12:24:18 +02:00
kevinveenbirkenbachandClaude Opus 4.8 cddb0dc59b feat(lint): require emoji-prefixed task/step/job names + convert existing
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.

- emit emoji names at the source: cli/build/include generator now writes
  "🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
  tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
  task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
  🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 22:45:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 b21a7a4249 fix(web-app-taiga): deploy the stack before running migrations
The administrator/migration include execed in the taiga_manager container before the flush_handlers that deploys the stack, so the swarm play failed with 'resolve-container-id: no running task for service taiga_manager'. Move the flush_handlers ahead of the include so the manager service exists before migrate runs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 17:01:08 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a212ed9df2 refactor(roles): swarm-aware exec routing in 7 roles
Replace compose-mode-specific `{{ BIN_COMPOSE }} exec ... {{ SERVICE }}`
+ `args.chdir: lookup('container', ..., 'directories.instance')`
invocations with `container exec -i {{ ..._CONTAINER_ADDRESS | quote }}`
(plus `--workdir` flag where chdir was load-bearing). The container
exec wrapper resolves the actual runtime container name on both
compose (compose-suffixed name) and swarm (stack-task-id-suffixed
name) via the `container_address` lookup, so the task no longer
assumes the compose-mode service-name address.

Pure-syntax adaptation only — no behavioural change, same register /
when / failed_when / no_log semantics preserved on every task.

Roles touched (12 task files):
- web-app-espocrm (1)
- web-app-friendica (2)
- web-app-listmonk (1)
- web-app-openproject (2)
- web-app-snipe-it (3)
- web-app-taiga (1)
- web-app-wordpress (2)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 19:08:46 +02:00
kevinveenbirkenbachandClaude Opus 4.7 05f6a97bd8 refactor(stack_host_template): generalise IS_STACK_HOST template wrapper across the codebase
Renames the local action plugin from `conf_template` to
`stack_host_template` so the name carries the actual semantics
(IS_STACK_HOST gate, not nginx config), broadens the lint guard to
every bare `template:` task, and migrates every consumer found in the
audit.

Plugin / tests / lint:
  * plugins/action/conf_template.py -> plugins/action/stack_host_template.py
    (drop nginx-specific docstring; trust_as_template gate unchanged)
  * test_conf_template.py            -> test_stack_host_template.py
  * test_conf_template_migration.py  -> test_stack_host_template_migration.py
    Scope expanded from `.conf.j2` to every `template:` task; marker
    renamed to `nocheck: stack-host-template`.

Migrations (~50 tasks now using stack_host_template:):
  * sys-svc-compose/04_files.yml: compose.yml, .env, Dockerfile,
    compose.override (every per-app compose-tree file)
  * web-app-keycloak: sso_proxy upstream-config (fixes the friendica
    CI failure that prompted this refactor) + realm-import
  * web-app-taiga, web-app-matrix, web-app-opentalk, web-app-pgadmin,
    web-app-moodle, web-app-listmonk, web-app-littlejs, web-app-fediwall,
    web-app-wordpress, web-app-discourse, web-app-bigbluebutton,
    web-app-jenkins, web-app-jitsi, web-app-mobilizon, web-app-suitecrm,
    web-app-decidim, web-app-dashboard, web-app-nextcloud, web-app-xwiki,
    web-app-mailu, web-app-magento, web-app-odoo, web-app-prometheus,
    web-app-matrix, web-app-peertube, web-app-mailu (+ existing `conf_template:`
    callers updated to the new name)
  * sys-front-tls, svc-db-openldap, sys-svc-webserver-core,
    sys-svc-rdbms (dedicated DB env + initdb), sys-front-inj-css,
    sys-front-inj-dashboard, sys-front-inj-all, sys-stk-front-proxy,
    web-svc-cdn-css, web-svc-xmpp, web-svc-legal, web-svc-simpleicons,
    web-opt-rdr-domains

Nochecks (per-host or already-structurally-stack-host):
  * sys-svc-mail-smtp/postfix, sys-svc-sshd, sys-svc-container/daemon.json
  * sys-timer, sys-service, sys-svc-mail-msmtp, sys-svc-journalctl,
    sys-daemon, sys-svc-compose/utils/_render_one_template (delegate_to
    stack host with peer slurp dance)
  * dev-locales, dev-nix, desk-*, user, user-backup, user-workstation
  * svc-storage-nfs-server/exports, test-e2e-playwright stage dirs

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-06 15:45:38 +02:00
kevinveenbirkenbachandClaude Opus 4.7 d3555858ab fix(test-swarm): local dnsmasq on mgr-01 resolves mocked DOMAIN_PRIMARY
The MODE_DEBUG-gated DinD DNS handler in sys-svc-container spawned a
busybox container with `--dns 1.1.1.1` and expected the mocked
infinito.example to resolve to 127.0.0.1, which Cloudflare obviously
cannot do. Run an in-cluster dnsmasq inside mgr-01 instead of patching
the symptom out:

- Stop and mask systemd-resolved so port 53 is free.
- Bootstrap /etc/resolv.conf with 1.1.1.1 so apt-get can fetch dnsmasq.
- Install dnsmasq with bind-dynamic, no-resolv, address=/${INFINITO_DOMAIN}/127.0.0.1
  and 1.1.1.1/8.8.8.8 as upstream forwarders; bind-dynamic catches docker0
  when the Ansible-installed dockerd creates it later.
- Switch /etc/resolv.conf to 127.0.0.1 and verify the chain via
  `getent ahosts ${INFINITO_DOMAIN}` so the script fails fast on misconfig.
- Pin mgr-01's docker0 to 172.17.0.1/16 via daemon.json `bip`, then point
  the swarm.yml networks.internet.dns at that gateway so DinD-spawned
  containers reach dnsmasq via their default route.

Also re-stage three pure-formatting tightenings ansible-lint made to
existing `template:  # nocheck` lines (double space -> single space).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-06 11:40:15 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2921ab6f03 feat(ansible): conf_template action plugin enforces IS_STACK_HOST gate
Adds plugins/action/conf_template.py, a drop-in `template:` replacement
that returns `skipped: true` whenever IS_STACK_HOST evaluates to false.
Routes the rendered Jinja through ansible.module_utils.parsing.convert_bool
so both Python bool and string "True"/"False" templar outputs resolve
correctly. Inherits every standard template module argument.

Migrates seven nginx-tree consumers that previously aborted on workers
with "Destination directory does not exist": sys-front-tls,
svc-db-openldap, sys-svc-webserver-core (nginx.conf + default_server.conf),
web-app-matrix (synapse vhost), web-app-peertube (per-domain vhost),
web-opt-rdr-domains (redirect vhost), and web-app-prometheus.

Tags twelve legitimate bare `template:` callers with
`# nocheck: conf-template`: per-host system configs (locale, nix, systemd
manager defaults, journald, msmtprc) and per-app compose-dir configs
(magento x2, mailu x2, moodle, odoo, taiga) that intentionally bypass the
project's nginx tree.

Adds tests/lint/ansible/modules/test_conf_template_migration.py as the
drift guard: any future `template:` task whose `src:` ends in `.conf.j2`
must adopt conf_template or carry the nocheck marker. Adds seven unit
tests for the plugin covering skip semantics, dispatch, mode defaults,
mode overrides, full-arg passthrough, and the string-bool templar path.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 22:25:13 +02:00
kevinveenbirkenbach 1da3c2eda7 fix(web-app): gate manager-only role bodies behind IS_STACK_HOST
Across roles/web-app-*, the typical body shape is:
  - include_role: sys-stk-full (or sys-svc-compose)   cluster prereqs
  - <host-fs writes, container exec, compose exec>    stack-host only

Under swarm-mode delegation the second group only makes sense on the
host that holds the compose stack (manager), but the role plays on
every host in groups[<role>] so workers abort on missing paths.

Two workflow sweeps (gate-manager-ops + gate-manager-ops-01core, 58
agents total, discover/plan/adversarial-verify per role) plus three
manual fixes surfaced 27 roles to refactor. Each role keeps
sys-stk-full ungated so worker-side prereqs (sys-svc-container, NFS
client) still bootstrap; the trailing manager-only block either moves
into a new tasks/02_manager_ops.yml included with
when: IS_STACK_HOST | bool (SUB_FILE pattern) or gets the gate
inlined on the one or two trailing tasks (INLINE pattern). In
compose mode IS_STACK_HOST stays true so single-host deploys are
unchanged.

Manual catches the workflow agents missed:
- web-app-fediwall: utils/once/flag.yml lives AT THE END of 01_core,
  not at the start; preserved verbatim outside the gate so workers
  still flip the run_once_<role> fact.
- web-app-xwiki: a pre-existing block(...) wraps sys-stk-full plus
  _flush_config.yml; gated only the _flush_config.yml include so
  sys-stk-full keeps running on workers.
- web-app-dashboard: 02_javascript.yml (creates /opt/compose/dashboard/
  JS dirs) ran BEFORE sys-stk-full and was ungated; added the gate.

Roles touched (INLINE): akaunting, jenkins, magento, mobilizon
Roles touched (SUB_FILE): baserow, dashboard, decidim, erpnext,
  fediwall, friendica, gitea, jitsi, keycloak, listmonk, littlejs,
  mailu, matomo, moodle, odoo, openproject, opentalk, pixelfed,
  shopware, suitecrm, taiga, xwiki, zammad
2026-06-05 12:18:32 +02:00
kevinveenbirkenbachandClaude Opus 4.7 5ca53b2ea6 fix(web-app-taiga): admin PW env, native-logout assert, themed-routes CSS gate
- tasks/01_administrator.yml: pass DJANGO_SUPERUSER_PASSWORD to
  createsuperuser; without it Django sets an unusable password and
  every API/native login attempt 401s.
- files/playwright/_shared.js: logoutFromTaigaNative asserts at /login
  directly; Taiga redirects logged-out users to /discover where the
  form is gated behind a Login link.
- files/playwright/test-themed-routes.js + templates/playwright.env.j2:
  gate gradient assertions on CSS_SERVICE_ENABLED so the LDAP-only
  variant (stock Taiga UI, no web-svc-css) falls back to a readiness
  check instead of failing on missing injected theming.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 13:43:43 +02:00
kevinveenbirkenbachandClaude Opus 4.7 d7591cb44a lint(compose-repository): enforce Jinja + paired URL/ref; fix 3 violators
New lint
--------

``tests/lint/ansible/modules/test_compose_repository_url_ref.py`` walks
every dict in every project ``.yml``/``.yaml`` and asserts:

1. ``compose_repository_url`` MUST be a Jinja expression — keeps repo
   coordinates in ``meta/services.yml`` (looked up via vars) instead
   of pasted into task files.
2. ``compose_repository_ref`` MUST also be a Jinja expression for the
   same SPOT reason.
3. Every ``compose_repository_url`` MUST be paired with a
   ``compose_repository_ref`` in the same dict — pinning the clone
   source without pinning the ref leaves checkouts at upstream HEAD
   (non-reproducible deploys).

Violations fixed
----------------

The lint surfaced three offenders, all migrated to the established
``repository:``/``ref:`` SPOT pattern under the role's primary entity
in ``meta/services.yml`` (akaunting/bigbluebutton/roulette-wheel
already use this shape):

- **web-app-taiga**: added ``repository:`` + ``ref:`` to the ``taiga``
  service block; tasks now use ``TAIGA_REPOSITORY`` / ``TAIGA_REF``
  vars; restores the previously missing ref pinning.
- **web-app-openproject**: added ``repository:`` + ``ref:`` to the
  ``openproject`` service block (the ref keeps the
  ``stable/{{ OPENPROJECT_VERSION }}`` Jinja form, just SPOT-relocated).
- **web-app-littlejs**: added ``repository:`` + ``ref:`` to the
  ``littlejs`` service block; the existing
  ``LITTLEJS_REPOSITORY_BASE`` (used by ``nav_bottom.html.j2`` for the
  human-readable upstream link) is now derived from the new
  ``LITTLEJS_REPOSITORY`` via ``regex_replace('\\.git$', '')`` so the
  literal hostname is gone too.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 13:43:43 +02:00
kevinveenbirkenbachandClaude Opus 4.7 19254a23b4 refactor(sys-svc-compose): rename docker_compose_flush_handlers → compose_handlers_flush
Pure 1:1 rename (69 insertions / 69 deletions across 66 files) to
finish the ``docker_*`` → ``compose_*`` alignment started in
8373468e9. The flag configures whether the compose include flushes
handlers right after run; the ``docker_`` prefix was misleading
(handlers are Ansible-side, not docker-side) and the variable is part
of the same ``sys-svc-compose`` SPOT cluster as ``compose_repository_*``.

Touches the role default, the implementation
(``sys-svc-compose/tasks/utils/up.yml``), all 60+ consumer roles,
the network-utility wrapper + its README, and the
``test_network_create_via_util.py`` lint (both the docstring guidance
and the asserted diff).

Breaking change for any external override that still sets the old
name — no alias is provided.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 13:43:43 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3f6e2dd68d refactor(sys-svc-compose): rename docker_git_repository_* → compose_*
Pure 1:1 rename (60 insertions / 60 deletions across 30 files) to
align the variable names with the role that owns them and to drop the
misleading ``docker_git_`` prefix — these knobs configure the
compose-stack's git clone step, not the docker CLI:

- docker_git_repository_address → compose_repository_url
- docker_git_repository_version → compose_repository_ref
- docker_git_repository_pull    → compose_pull_repository
- docker_repository_path        → compose_repository_path

``ref`` better captures git refs (branches/tags/SHAs are not
"versions"); ``url`` is the idiomatic name for the clone source.

Touches every consumer (web-app-akaunting/bigbluebutton/dashboard/
fediwall/hugo/littlejs/mini-qr/minio/openproject/openwebui/
roulette-wheel/taiga + svc-db-openldap/svc-prx-openresty/sys-svc-rdbms
+ web-svc-collabora/coturn/logout/onlyoffice + sys-front-inj-all), the
role's own defaults/tasks, the bigbluebutton compose_mods filter and
its unit test, the declared-in-services lint, and the hugo
requirements doc.

Breaking change for any external override that still sets the old
names — no alias is provided.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 13:43:43 +02:00
kevinveenbirkenbachandClaude Opus 4.7 7779fea529 fix(dockerfiles): purge apt lists before update + lint + taiga rename
Move `rm -rf /var/lib/apt/lists/*` before `apt-get update` in every
Debian-based Dockerfile so a stale base-image-baked InRelease
(Valid-Until expired) cannot abort the build layer with exit 100. The
trailing rm at end-of-RUN never ran because `apt-get update` aborted
first; the leading rm forces a fresh fetch from the configured source.

Affected: moodle, bridgy-fed, bookwyrm (2 RUN), postmarks, taiga
backend, svc-db-openldap, svc-db-postgres, jenkins, joomla, mediawiki,
odoo, suitecrm (2 RUN), wordpress.

Add tests/lint/docker/dockerfile/test_apt_update_purge_lists.py to
enforce the contract repo-wide: every `RUN` containing `apt-get
update` MUST `rm -rf /var/lib/apt/lists/*` first in the same RUN, and
MUST NOT pass `Acquire::Check-Valid-Until=false` (which would silence
apt's replay-protection rather than refresh the list).

Rename roles/web-app-taiga/templates/Dockerfile.{back,front}.j2 to
Dockerfile.{backend,frontend}.j2 and update the matching tasks/
main.yml template refs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 13:43:42 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2b72d8b550 test(lint): forbid redundant default(...) on module IO in same-task conditionals
Adds tests/lint/ansible/test_no_redundant_default_on_module_io.py which
flags `<reg>.{stdout,stderr,rc} | default(...)` inside `changed_when:`,
`failed_when:`, or `until:` lines. Those conditionals only fire after
the module has actually executed, so the fallback can never apply --
keeping it just hides bugs and noise.

Cleans up the 21 existing offenders across 14 role files. The taiga
upsert-admin task additionally gains a `not ASYNC_ENABLED and ...`
short-circuit so its async-poll-0 path no longer leans on a silent
empty fallback for stdout/stderr; with the short-circuit, the
`default('')` is genuinely dead code.

Cross-task references (a later task reading a registered result of an
earlier task that may have been skipped via `when:`) stay out of scope
and remain a legitimate use of `| default(...)`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:06:32 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2a023ca6a1 chore(lint): autoformat strict-failure semantics + claude.sh rename
* scripts/lint/autoformat.sh:
  - rename run_sort_claude_settings worker to run_claude (matches the
    renamed scripts/lint/claude.sh).
  - drop the EXIT trap that wiped status_dir; cleanup happens explicitly
    after `wait` returns, so a parallel worker that interrupts the
    parent never races a vanished status dir.
  - each worker now captures its tool's exit code and writes one of
    OK / SKIP / FAIL. Tolerated `expected violations remain`-style exits
    (markdownlint-cli2 rc≤1, ansible-lint rc≤2) still count as OK; only
    real failures count as FAIL.
  - all workers run to completion regardless of individual tool failures
    (so the user always sees the full picture); after the summary, the
    script exits 1 if any worker reported FAIL.
* scripts/lint/sort_claude_settings.sh -> scripts/lint/claude.sh
  (history preserved via git mv); update test_claude_settings_sorted.py
  fix-hint accordingly.

Plus working-tree drift across roles/, group_vars/, inventories/,
requirements/, and .pre-commit-config.yaml accumulated from the
make-test setup phase regenerating generated artefacts during recent
test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 01:30:49 +02:00
kevinveenbirkenbachandClaude Opus 4.7 951f6d36d2 chore(lint): unify autoformat/galaxy parallelism into one PARALLEL env var
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL,
  declare it at file start with default `true`; still env-overridable.
  Reword the worker comment so the literal `# shellcheck` token no
  longer appears at start-of-line and trips SC1072/SC1073.
* scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE /
  GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure
  is always fatal. NAMESPACE is an internal constant. Add the same
  PARALLEL env var (default `true`, env-overridable); falsy demotes the
  xargs fan-out to one worker.
* docs/contributing/environment/variables.md: collapse the four
  AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both
  consumers.
* docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere;
  add the lint-galaxy parallelism note.

Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 00:59:24 +02:00
kevinveenbirkenbach 8d73974e88 fix taiga oidc login and logout 2026-03-28 17:14:29 +01:00
kevinveenbirkenbach 93ebc301e6 Fix Taiga LDAP and OIDC role integration 2026-03-25 17:54:36 +01:00
kevinveenbirkenbachandClaude Sonnet 4.6 8504c2ec87 feat(web-app-taiga): implement LDAP and OIDC auth with OAuth2 Proxy
- Add OAuth2 Proxy in front of Taiga (port 16491) as access guard
- Add OIDC login via Keycloak→LDAP using taiga-contrib-oidc-auth (default on)
  - Custom multi-stage Docker build compiles CoffeeScript/Gulp frontend plugin
  - Build context fixed to docker_repository_path (not directories.instance)
  - pip install via git+https:// (not PyPI, package not published there)
  - Settings rendered as Python literals in config.append.py (not os.getenv)
  - urls.py uses re_path() (Django 4+ compatible, url() was removed)
- Add LDAP direct login via taiga-contrib-ldap-auth-ext (default off)
  - LDAP_SERVER strips port via regex_replace (host-only required by plugin)
  - LDAP_MAP_USERNAME_TO_UID = None (prevents plugin crash)
- Both plugins append to config.py at container startup via cat >>
- Remove broken robrotheram image variants and dead TAIGA_SETTING_FILES loop
- Remove broken oidc/local.py.j2 (local.py is not imported by taiga-back)
- Add conf.json template for taiga-front (loginFormType / contribPlugins)
- Add oauth2_proxy_cookie_secret to schema

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 23:54:26 +01:00
kevinveenbirkenbach a55c99ef62 refactor(compose): replace docker lookup with container and unify compose paths
- Rename lookup plugin from `docker` to `container`
- Switch all lookups from `lookup('docker', ...)` to `lookup('container', ...)`
- Normalize compose file keys:
  - docker_compose → compose
  - docker_compose_override → compose_override
  - docker_compose_ca_override → compose_ca_override
- Centralize compose path resolution via docker_paths_utils
- Update compose_file_args to always derive paths from DIR_COMPOSITIONS
- Align all roles, templates, and vars with container/compose naming
- Update and rename unit tests accordingly

This prepares the codebase for engine-agnostic container backends
(e.g. Docker ↔ Podman) and removes legacy docker-specific assumptions.

https://chatgpt.com/share/6987a7cc-d374-800f-99ad-bb4c8f92c59e
2026-02-07 21:59:39 +01:00
kevinveenbirkenbach 53e817fa22 refactor(compose): migrate from docker-compose.yml to compose.yml everywhere
- Rename all docker-compose.yml templates to compose.yml
- Update override and CA override filenames accordingly
- Adjust compose wrapper, lookup plugins, filters and path helpers
- Update roles, tasks, handlers and scripts to use compose CLI
- Rewrite documentation and admin guides to reference compose
- Update unit and integration tests to expect compose.yml
- Keep internal variable keys for backward compatibility where needed

https://chatgpt.com/share/69870272-b168-800f-84c3-6417398e441f
2026-02-07 10:14:21 +01:00
kevinveenbirkenbach 9fa9692d7d refactor(compose): replace docker compose calls with compose wrapper
- Switch all docker compose invocations to the compose wrapper
- Update comments, task names, and log messages accordingly
- Simplify handlers and tasks to rely on wrapper-managed flags and env files
- Align tests and documentation with compose-based workflow

https://chatgpt.com/share/69866ede-be2c-800f-94b0-bffcf8fed260
2026-02-06 23:44:39 +01:00
kevinveenbirkenbach 0a7b4efa22 Replaced docker compose pull by compose pull 2026-02-06 23:26:20 +01:00
kevinveenbirkenbach 1db08dbb79 Renamed DOCKER_COMPOSE to COMPOSE 2026-02-06 10:33:50 +01:00
kevinveenbirkenbach 26a5b159e0 refactor(docker-lookup): replace want= syntax with path-based access
- Replace lookup('docker', ..., want='...') with lookup('docker', ..., 'path')
- Allow both top-level keys (directories/files) and dotted paths
- Simplify docker lookup API and reduce lazy dict propagation
- Update unit tests to reflect new lookup semantics

https://chatgpt.com/share/6984ccea-8290-800f-a6d7-ec0e94df95a0
2026-02-05 18:45:43 +01:00
kevinveenbirkenbach 56aea31667 refactor(docker): replace docker_compose vars with docker lookup across roles
Remove implicit docker_compose variable usage and resolve all docker paths
(files, directories, env) via lookup('docker', application_id).
This prevents scope leakage, centralizes path resolution, and makes docker
context deterministic across roles and handlers.

https://chatgpt.com/share/6984ccea-8290-800f-a6d7-ec0e94df95a0
2026-02-05 18:15:10 +01:00
kevinveenbirkenbach a0f791fc52 web-app-taiga: run migrations before admin creation
https://chatgpt.com/share/6980ee3c-2e8c-800f-92aa-63b19d971eb2
2026-02-02 19:34:02 +01:00
kevinveenbirkenbach 989c0c0fa2 refactor(stacks): merge stateful/stateless backend+full stacks into sys-stk-backend/sys-stk-full
- Replace sys-stk-back-stateful + sys-stk-back-stateless with unified sys-stk-backend role
- Replace sys-stk-full-stateful + sys-stk-full-stateless with unified sys-stk-full role
- Add reusable task helpers: _load_service.yml (optional deps) and 01_database.yml (DB bootstrap)
- Switch optional dependency loading (OpenLDAP, Ollama) to service_load_enabled (enabled && shared) + run_once guard
- Remove legacy OLLAMA_LOCAL_ENABLED flag; rely on docker.services.ollama.enabled/shared config
- Update all affected app/service roles and web-app template to use new stack role names

https://chatgpt.com/share/69685fb2-cb1c-800f-a9e8-2f5f765a1a53
2026-01-15 04:31:54 +01:00
kevinveenbirkenbach d508012653 fix(web-app-xwiki): add missing docker.services.ldap config to prevent get_app_conf assertion failure
The XWiki role expected docker.services.ldap.enabled for auth backend validation,
but the ldap service block was missing from the application config.

This caused get_app_conf to fail during the auth backend assertion
("Only one auth backend (OIDC or LDAP) may be enabled").

Add docker.services.ldap section with sane defaults:
- enabled: false
- shared: false
- trylocal: true

This restores schema consistency and prevents validation errors.

https://chatgpt.com/share/69680fef-507c-800f-9492-e0edd2880e92
2026-01-14 22:51:37 +01:00
kevinveenbirkenbach 1d0f26617a Optimized docker_git_repository_ variables 2025-12-03 14:23:53 +01:00
kevinveenbirkenbach 716ebef33b Refactor task includes and update variable handling for Ansible 2.20 migration
This commit updates multiple roles to ensure compatibility with Ansible 2.20.
Several include paths and task-loading mechanisms required adjustments,
as Ansible 2.20 applies stricter evaluation rules for complex Jinja expressions
and no longer resolves certain relative include paths the way Ansible 2.18 did.

Key changes:
- Replaced legacy once_finalize.yml and once_flag.yml with the new structure
  under tasks/utils/once/finalize.yml and tasks/utils/once/flag.yml.
- Updated all include_tasks statements to use 'path_join' with playbook_dir,
  ensuring deterministic and absolute file resolution across roles.
- Fixed all network helper includes by converting direct relative paths such as
  'roles/docker-compose/tasks/utils/network.yml' to proper Jinja-evaluated paths.
- Normalized MATOMO_* variable names for consistency with the updated variable
  scope behavior in Ansible 2.20.
- Removed deprecated patterns that were implicitly supported in Ansible 2.18
  but break under the more strict variable and path resolution model in 2.20.

These changes are part of the full migration step required to ensure the
infinito-nexus roles remain stable, deterministic, and forward-compatible with
Ansible 2.20.

Details of the discussion and reasoning can be found in this conversation:
https://chatgpt.com/share/69300a8d-24d4-800f-bec0-e895a695618a
2025-12-03 11:02:34 +01:00
kevinveenbirkenbach 27b89d8fb6 Taiga: refactor service naming & resource limits
Add CPU/memory/pids limits for taiga, async, front, gateway, events, async-rabbitmq, events-rabbitmq, manager, and protected. Align manager service usage (was taiga-manage) in admin tasks and inits compose. Switch to variable-driven service names (TAIGA_* vars), add container_name patterns, normalize volume mappings via TAIGA_VOLUME_STATIC/MEDIA, fix depends_on to use TAIGA_* vars, and set RabbitMQ hostnames from vars. Remove obsolete Development.md.

Conversation reference: https://chatgpt.com/share/68da83b7-0cb4-800f-9702-d8a2d4ebea71  (replace with this chat’s share link)
2025-09-29 15:04:12 +02:00
kevinveenbirkenbach 6da7f28370 Optimized whitespacing 2025-09-23 00:51:23 +02:00
kevinveenbirkenbach e6803e5614 refactor(ansible): normalize include_role syntax and unify host config paths via path_join
- Remove stray spaces after include_role: across many roles to ensure clean YAML and
  consistent linting/formatting.
- Listmonk:
  - Introduce LISTMONK_CONFIG_HOST = [ docker_compose.directories.config, 'config.toml' ] | path_join
  - Use that var in the template task (dest) and the docker-compose volume mount
- Matrix:
  - Build MATRIX_SYNAPSE_CONFIG_PATH_HOST, MATRIX_SYNAPSE_LOG_PATH_HOST, and
    MATRIX_ELEMENT_CONFIG_PATH_HOST via path_join
- Mobilizon:
  - Build mobilizon_host_conf_exs_file via path_join
  - Keep get_app_conf strictness unchanged (defaults to True in our filter), so behavior
    remains strict even though the explicit third arg was dropped
- Simpleicons:
  - Build server.js and package.json host paths via path_join
- Numerous web-app roles (Confluence, Discourse, EspoCRM, Friendica, Funkwhale, Gitea,
  GitLab, Jenkins, Joomla, Listmonk, Mailu, Mastodon, Matomo, Matrix, MediaWiki,
  Mobilizon, Moodle, Nextcloud, OpenProject, Peertube, Pixelfed, Pretix, Roulette Wheel,
  Snipe-IT, Syncope, Taiga, WordPress, XWiki, Yourls) and web-svc roles (coturn,
  libretranslate, simpleicons) updated for consistent include_role formatting

Why:
- path_join avoids double slashes and missing separators across different config roots
- Consistent include_role: formatting improves readability and prevents linter noise

Ref:
- Conversation: https://chatgpt.com/share/68d14711-727c-800f-b454-7dc4c3c1f4cb
2025-09-22 14:55:25 +02:00
kevinveenbirkenbach 6ea8301364 Refactor: migrate cmp/* and srv/* roles into sys-stk/* and sys-svc/* namespaces
- Removed obsolete 'cmp' category, introduced 'stk' category (fa-bars-staggered icon).
- Renamed roles:
  * cmp-db-docker → sys-stk-back-stateful
  * cmp-docker-oauth2 → sys-stk-back-stateless
  * srv-domain-provision → sys-stk-front
  * cmp-db-docker-proxy → sys-stk-full-stateful
  * cmp-docker-proxy → sys-stk-full-stateless
  * cmp-rdbms → sys-svc-rdbms
- Updated all include_role references, vars, templates and README.md files.
- Adjusted run_once comments and variable paths accordingly.
- Updated all web-app roles to use new sys-stk/* and sys-svc/* roles.

Conversation: https://chatgpt.com/share/68b0ba66-09f8-800f-86fc-76c47009d431
2025-08-28 22:23:09 +02:00
kevinveenbirkenbach 8baec17562 web-app-taiga: extract admin bootstrap into dedicated task; add robust upsert path
Add roles/web-app-taiga/tasks/01_administrator.yml to handle admin creation via 'createsuperuser' and, on failure, an upsert fallback using 'manage.py shell'. Ensures email, is_staff, is_superuser, is_active are set and password is updated when needed; emits CHANGED marker for idempotence.

Update roles/web-app-taiga/tasks/main.yml to include the new 01_administrator.yml task file, removing the inline admin logic for better separation of concerns.

Uses taiga-manage helper service and composes docker-compose.yml with docker-compose-inits.yml to inherit env/networks/volumes consistently.

Chat reference: https://chatgpt.com/share/68af7637-225c-800f-b670-2b948f5dea54
2025-08-27 23:58:37 +02:00
kevinveenbirkenbach 1401779a9d web-app-taiga: add manage/init flow and idempotent admin bootstrap; fix OIDC config and env quoting
config/main.yml: convert oidc from empty mapping to block; indent flavor under oidc; enable javascript feature.

tasks/main.yml: use path_join for taiga settings; create docker-compose-inits via TAIGA_DOCKER_COMPOSE_INIT_PATH; flush handlers; add idempotent createsuperuser via taiga-manage with async/poll and masked logs.

templates/docker-compose-inits.yml.j2: include compose/container base to inherit env and project settings.

templates/env.j2: quote WEB_PROTOCOL and WEBSOCKET_PROTOCOL.

templates/javascript.js.j2: add SSO warning include.

users/main.yml: add administrator email stub.

vars/main.yml: add js_application_name; restructure OIDC flavor flags; add compose PATH vars; expose TAIGA_SUPERUSER_* vars.

Chat reference: https://chatgpt.com/share/68af7637-225c-800f-b670-2b948f5dea54
2025-08-27 23:19:42 +02:00
kevinveenbirkenbach 73d5651eea web-app-taiga: refactor OIDC gating + defaults
- Introduced dedicated variables in vars/main.yml:
  * TAIGA_FLAVOR_TAIGAIO
  * TAIGA_TAIGAIO_ENABLED
- Replaced inline Jinja2 get_app_conf checks with TAIGA_TAIGAIO_ENABLED for
  consistency in tasks, docker-compose template and env file.
- Adjusted env.j2 to use TAIGA_TAIGAIO_ENABLED instead of direct flavor checks.
- Enabled css by default (true instead of false).
- Cleaned up spacing/indentation in config and env.

This improves readability, reduces duplicated logic, and makes it easier to
maintain both OIDC flavors (robrotheram, taigaio).

Conversation: https://chatgpt.com/share/68af65b3-27c0-800f-964f-ff4f2d96ff5d
2025-08-27 22:08:35 +02:00
kevinveenbirkenbach 12a267827d Refactor websocket and Taiga variables
- Introduce WEBSOCKET_PROTOCOL derived from WEB_PROTOCOL (wss if https, else ws).
- Replace hardcoded websocket URLs in EspoCRM, Nextcloud and Taiga with {{ WEBSOCKET_PROTOCOL }}.
- Fix mautrix-imessage to use ws:// for internal synapse:8008.
- Standardize Pixelfed OIDC env spacing.
- Refactor Taiga variables to TAIGA_* naming convention and clean up EMAIL_BACKEND definition.

See: https://chatgpt.com/share/68af62fa-4dcc-800f-9aaf-cff746daab1e
2025-08-27 21:57:04 +02:00
kevinveenbirkenbach 022800425d THE HUGE REFACTORING CALENDER WEEK 33; Optimized Matrix and during this updated variables, and implemented better reset and cleanup mode handling, also solved some initial setup bugs 2025-08-15 15:15:48 +02:00
kevinveenbirkenbach 756597668c Semi bsr for applications[] to prevent heavy to debug bugs in j2 - part 1 2025-07-13 15:11:38 +02:00
kevinveenbirkenbach 7a38241485 Made code more modular and refactored to cmp roles 2025-07-09 20:15:32 +02:00
kevinveenbirkenbach c9c73cbdb2 Decoupeld database, docker and proxy 2025-07-09 14:21:30 +02:00
kevinveenbirkenbach 575df76ec3 Shortened service- to svc- 2025-07-09 05:00:41 +02:00