Commit Graph
54 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 29015311b6 feat(lint): derive container paths from meta/volumes.yml everywhere
meta/volumes.yml says where a volume lands inside its container. Wherever a
second file spelled the same path out, the two drifted independently: the mount
moves, the consumer keeps pointing at the old place, and nothing fails until the
process looks for content that is not there. e031b7f61 fixed exactly this for
web-app-moodle; this generalises it and makes the rule hold.

The lookup gains what a caller needs. `targets` maps each mounting service to
its container path, and `target` is the one path an entry mounts at -- present
only when every mount agrees. An entry that lands at different paths per service
omits `target` rather than picking one, and the caller names the service:

    "{{ lookup('volume', application_id, 'prometheus_config').target }}"
    "{{ lookup('volume', 'web-app-prometheus', 'host_sys').targets['node-exporter'] }}"

66 literals across 48 role files now derive. Replacement only happened where the
path stood alone -- /srv/app inside /srv/app/data is a different path -- and
every rewritten file was re-parsed, Jinja and YAML alike. Three sites the tool
refused rather than guess were done by hand: zammad's path sat in a Jinja list
literal, seaweedfs' inside a quoted string feeding a filter, and openresty's
/var/run/docker.sock is not a mount target at all but the socket on the host,
which only happens to spell the same string. That one carries the single
nocheck.

tests/lint/repository/spot/test_volumes_path_single_spot.py keeps it that way.
roles/*/files/ is out of scope by construction: it is copied verbatim into an
image and executed there, so a Dockerfile or entrypoint has no Ansible to
evaluate a lookup with.

One defect this found, and it was in the tooling rather than the roles: deciding
ambiguity AFTER filtering short paths made web-app-prometheus' host_sys look
single-valued -- it mounts /sys for cadvisor and /host/sys for node-exporter,
and /sys is below the length filter. The template would have rendered an
undefined `.target`. Both the converter and the rule now decide ambiguity over
every target and filter afterwards, and that call reads targets['node-exporter'].

Verified by resolving rather than by reading: all 70 accesses in the diff were
run against the real registry and every one yields a non-empty path equal to the
literal it replaced. make test-lint: 328 passed.

Not verified: no Ansible render, so a call site whose surrounding context cannot
build the applications registry would still fail at deploy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 02:02:29 +02:00
kevinveenbirkenbachandClaude Opus 5 c247a6ce26 fix(openresty): probe the assert container via container_address
The readiness probe passed the bare service name to container inspect. In swarm that name does not belong to a container, because compose_only suppresses container_name there, but it does belong to the overlay network, which get_entity_name names identically. Docker resolved the network and failed with a missing State key on every swarm job once MODE_ASSERT armed the include.

Route the probe through OPENRESTY_CONTAINER_ADDRESS like the validation task two lines below, switch to shell so the resolver subshell is evaluated, and pin --type container so a name shared with a network can no longer select the wrong object.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 20:51:56 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4039313266 refactor(tasks): enforce and apply a single task-file numbering scheme
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.

Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:31 +02:00
kevinveenbirkenbachandClaude Opus 4.8 284f7c9669 fix(openresty): provision the full nginx config tree at bootstrap
The bootstrap task kept a hand-picked partial copy of the canonical nginx directory list that omitted conf.d/global and the servers/maps/streams/lua subtree. Roles that write http-includes before the front chain runs (prometheus's http-block in the bare variant 1, where only openresty itself is preloaded) hit 'Destination directory /etc/nginx/conf.d/global does not exist'; variant 0 masked the ordering because preloaded shared services pull sys-svc-webserver-core first. Loop over the canonical lookup('nginx','directories.ensure') SPOT plus the existing non-nginx extras, establishing the invariant that a provisioned openresty always owns its complete config tree.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 22:38:42 +02:00
kevinveenbirkenbachandClaude Fable 5 40e553069d refactor(structure): nest prefix-grouped sibling files into subfolders
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.

Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 20:00:03 +02:00
kevinveenbirkenbachandClaude Fable 5 c51c0de9ba refactor(diagnostics): recursive workflow-level rescue snapshot, no in-play rescue blocks
Replace scripts/system/diagnostics/rescue.sh with utils/diagnostics/
container.py: one workflow call on failure captures the current level
(containers with logs+inspect, swarm services, journal, host resources,
in-play role dumps) and recurses itself through every DiD boundary via
self-copy, from the runner down to the innermost runtime (RESCUE_MAX_DEPTH).
compose and swarm now share an identical failure step; the swarm per-node
collector wrapper is gone. All 17 script-only Ansible rescue: blocks are
deleted and their bare block: wrappers inlined (plus a wordpress leftover);
the three justified blocks (postgres pg_hba, xwiki extensions, keycloak
cross-node recovery) keep their evidence tasks and now end in a fail task.
Two lints enforce the state: rescue: is forbidden without a
'# nocheck: rescue-<reason>' first-task marker, and a block without
rescue/always/when or another scoping key must be inlined.
sys-service-terminator hoists become to the include apply. Also move
routine/08_cleanup.sh to utils/clean/teardown.sh and topology/ to
utils/topology/ so routine/ holds only the sequenced flow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 03:15:53 +02:00
kevinveenbirkenbachandClaude Opus 4.8 094c02a69d feat(diagnostics): single SPOT rescue script with CI-uploaded snapshots
Every Ansible rescue: block used to carry ad-hoc per-role failure dumps that
drifted apart and captured different subsets. Replace them all with one
diagnostics script (scripts/system/diagnostics/rescue.sh) that snapshots
containers (full logs plus inspect per container), swarm services (ps plus
logs per service), journal, host resources, runtime info and stats into a
per-run folder and always exits 1 so the deploy still fails. A new lint
enforces the shape: every rescue ends with the script, at most one
nocheck-justified extra task before it (postgres pg_hba dump, xwiki
extension-repo probe), and a nocheck rescue-recovery exemption for the one
block that must genuinely recover: keycloak object updates, where a
cross-node reschedule needs a swarm-wide node re-resolve plus a delegate_to
change that only a rescue can express; its recovery is kept and now also
captures diagnostics before the terminal failure.

The snapshot folder path is a group_vars SPOT: DIR_RESCUE_DIAGNOSTICS and
DIR_VAR_LIB live in group_vars/all/05_paths.yml as plain strings; new env
handlers derive INFINITO_RESCUE_DIAGNOSTICS_DIR and INFINITO_DIR_VAR_LIB
from it for the generated .env, replacing the default.env entries, and the
play-level environment forwards the value to every task host so rescue.sh
needs no second default. CI uploads the folder like the inventory and
playwright artifacts: compose copies it out of the deploy container in
dedicated.sh, swarm captures per node (manager and both workers) into
node-named subfolders.

Deleting the ad-hoc dumps orphaned helper files (postgres inspect_container,
terminator docker_diagnostics, peertube diagnostics, matomo bootstrap debug,
xwiki and openresty rescue includes) and 14 role vars; all removed. Generic
pieces of those dumps (runtime version and info, stats, dmesg oom evidence,
swapon) moved into the SPOT script instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 03:04:06 +02:00
kevinveenbirkenbach 9b2f870777 Merge branch 'main' into feature/swarm-nfs 2026-07-01 05:37:36 +02:00
kevinveenbirkenbachandClaude Opus 4.8 23d1447566 refactor(ansible): add path_absolute lookup, replace playbook_dir joins
Introduce plugins/lookup/path_absolute.py as the single source for the
repo-root anchor: lookup('path_absolute', 'roles/...') returns an absolute
path under playbook_dir, joining every term. Replace all hand-built
playbook_dir path constructions across .yml/.yaml and .j2 templates with it:

  [playbook_dir, '<rel>'] | path_join   -> lookup('path_absolute', '<rel>')
  playbook_dir ~ '/<rel>'               -> lookup('path_absolute', '<rel>')
  {{ playbook_dir }}/<rel>              -> lookup('path_absolute', '<rel>')

Unrelated repo_path symbols in cli/meta/roles/applications/resolution and the
single bare discover_playwright_roles(playbook_dir) call are left untouched.

Add tests/lint/ansible/test_playbook_dir_via_lookup.py: fails the build if any
of the three construction forms reappears in .yml/.yaml/.j2, pointing callers
at the lookup. Unit test covers the plugin.

Touching these files pulled their pre-existing narration comments into
test_comments_valid scope; clean them (delete restatement/banners, keep real
trip-wires as Workaround/Upstream/Invariant markers). Drop caution, important
and pitfall from the comment-lint marker vocabulary and remove the comments
that used them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 20:36:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 cddb0dc59b feat(lint): require emoji-prefixed task/step/job names + convert existing
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.

- emit emoji names at the source: cli/build/include generator now writes
  "🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
  tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
  task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
  🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 22:45:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 47729106e6 fix(swarm): unblock stack-deploy (openresty group_add) + CA-inject on verbatim env secrets
docker stack deploy rejects the compose group_add property ('Additional property group_add is not allowed'), so svc-prx-openresty drops it from the swarm compose and applies the docker.sock gid post-deploy via 'container service update --group-add' (swarm-native equivalent). The compose path (network_mode host) is unchanged; the new task is DEPLOYMENT_MODE == 'swarm' gated.

compose_ca's swarm --no-wrapper path no longer runs 'docker compose config', whose strict dotenv parser aborts on swarm env files carrying verbatim special-char secrets (e.g. a password beginning with a single quote). It reads the rendered compose files directly; wrap=False needs only service names. The compose (wrapper=True default) path keeps the original 'docker compose config' branch byte-identical.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 11:57:19 +02:00
kevinveenbirkenbachandClaude Opus 4.8 18e53dc4c6 fix(svc-prx-openresty): grant the swarm worker docker-socket access
In swarm the openresty worker runs as nobody and cannot read the host docker socket (root:<gid> 0660), so the init_worker docker-health Lua poll fails 'Permission denied' and the /metricz up metric stays 0. Stat the socket for its live gid and group_add it to the swarm service so the worker can read it. Swarm-gated; compose uses network_mode host and is untouched. Verified live: --group-add of the socket gid removes the EACCES.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 22:55:56 +02:00
Alejandro Roman 4663dc4fd8 clean long comments 2026-06-24 18:31:20 +02:00
Alejandro Roman e5ad7a67b1 fix open restry issues wit 2 level nested runner test 2026-06-24 18:31:17 +02:00
Alejandro Roman c34fbf89cb clean last comments 2026-06-24 18:30:53 +02:00
Alejandro Roman c038d147a3 fix runner issue in DinD + addeed docs for CLI runners 2026-06-24 18:30:46 +02:00
Alejandro Roman aff4a7d158 refactor(svc-runner): fix comments on PR 2026-06-24 18:30:43 +02:00
Alejandro Roman 0d28e2513b fix(svc-runners): fix dependencies issues 2026-06-24 18:30:36 +02:00
kevinveenbirkenbachandClaude Opus 4.8 baae3c377e fix(openresty-swarm): out-of-band overlay creation + stop-first host-mode redeploy
- Load openresty via the sys-svc-compose network/routine.yml helper (same
  pattern as the db/ollama roles) so the shared `openresty` overlay is created
  attachable + swarm-scoped before the stack deploys; emit the top-level
  networks block via compose_networks.
- Add an opt-in `order:` to the swarm update_config and set it to `stop-first`
  for openresty: it publishes 80/443 in host mode, so a default rolling update
  deadlocks (the new task cannot bind ports the old task still holds).
- Relax the network-include lint to accept a single compose_networks lookup in
  a host-mode template (openresty is mode-conditional: host in compose, overlay
  in swarm).
- Whitelist openresty's path_join network/routine.yml include in
  .ansible-lint-ignore (same dynamic-include pattern as the db/ollama roles).

Runtime-confirmed (swarm, attempt 32): openresty created its overlay, attached
consumers, reached backends, and redeployed cleanly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 19:49:18 +02:00
kevinveenbirkenbachandClaude Opus 4.7 366dc7b705 refactor(overlay): registry-driven shared-overlay networks with implicit defaults
Replace the hand-coded if-chains for db/ldap/ollama/proxy shared overlays
with a single registry-driven loop in sys-svc-{compose,container}/networks.yml.j2.
Each provider declares its overlay metadata under meta/server.yml.networks.overlay;
discover_role_services passes the block through verbatim into the service_registry.

Schema (modes/topology/consumer/aliases/proxy_resolvable) with implicit defaults:
- consumer.kind defaults to services_flags
- consumer.key defaults to <provides or entity_name>
- consumer.flags defaults to [enabled, shared]
- aliases defaults to [entity_name] (shared_net) or [] (default_net)

Openresty becomes a generic alias-collecting proxy: web-app-keycloak declares
a proxy_resolvable beacon with its public auth.<domain> alias; openresty sweeps
the registry at render time and harvests all proxy_resolvable aliases onto its
default-net overlay. Openresty no longer hardcodes keycloak's domain.

Lock the new behaviour with 9 render-snapshot tests covering provider self-render,
consumer attachment, beacon sweep, default-derivation, and compose-mode fallback.
Replace the Ansible-only `_entry.get('canonical') is not defined` filter with
`'canonical' not in _entry` so the templates also render correctly under vanilla
Jinja2 (test harness).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-16 18:42:03 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c61c68e903 test(lint): forbid >3-child blocks + ≥3-task same-when runs; refactor 5 roles
Adds two structural ansible lints under tests/lint/ansible/structure/:

- ansible-block-oversized: forbid `- block:` constructs with more than
  3 direct child tasks. Big block bodies hide intent, accumulate
  unrelated guards on the wrapper, and signal that the body wants to
  be its own include_tasks sub-file. Extraction keeps the parent
  scannable; the sub-file MAY carry the `# include-gated:
  when: …` header marker so swarm-compat lints exempt it.

- ansible-repeated-when-run: forbid 3+ consecutive sibling tasks
  carrying the SAME `when:` expression. Copy-pasted guards drift,
  silently disable each other over time, and read worse than a single
  `- name: …` + `when: <expr>` + `block:` wrapper. Per-line opt-out
  for runs where wrapping would alter semantics (cross-task register
  dependencies).

Both lints use the line-based scanner pattern from existing
tests/lint/ansible/jinja/* + `is_suppressed_at` from
utils.annotations.suppress for the per-line nocheck markers.

Refactors five roles into compliance:

- svc-prx-openresty/tasks/02_assert.yml: heavy rescue body extracted
  to 02a_assert_rescue_debug.yml.
- test-e2e-playwright/tasks/01_setup.yml: 3 install-needed siblings
  wrapped in a single when-block.
- web-app-erpnext/tasks/02_bench_bootstrap.yml: 3 site-probe-failed
  siblings wrapped likewise.
- web-app-odoo/tasks/02_install_modules.yml: 3 OIDC-enabled siblings
  wrapped. tasks/03_oidc.yml: dropped redundant single-task wrapper
  block (the original wrapper carried name only and would have
  required extraction for size — inlining the 23 tasks at file root
  keeps the file purpose-narrow and removes one layer of indirection).
- web-app-xwiki/tasks/04_extensions.yml: large block body + rescue
  body both extracted to 04a_extensions_rescue.yml +
  04b_extensions_execute.yml.

Mixed-scope offenders (web-app-keycloak/tasks/{02_manager_ops,05a,
assert/05_ldap,update/05_ldap_shared,02a} and
web-app-matomo/tasks/{03_manager_ops,05_bootstrap,05a}) deliberately
stay unstaged; they ship with their respective swarm-aware
restructure commits where they belong by topic, not bundled with the
structural-lint cleanup.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 19:08:46 +02:00
kevinveenbirkenbach e3645e2ea3 fix(swarm-nfs): pre-create openresty bind sources + SPOT NFS state path + reorder bootstrap
- roles/svc-prx-openresty/tasks/01_core.yml: extend the existing
  nginx-config pre-create into a loop that covers all seven host
  bind-mount sources openresty's compose template references (nginx
  config + base, www data, well-known, letsencrypt webroot + base,
  TLS self-signed base). docker swarm refuses to start containers
  whose bind-mount sources do not exist on the target node; compose
  silently created them, so the gap surfaced only under swarm.

- roles/sys-svc-compose/tasks/utils/nfs_prep.yml: switch the per-volume
  pre-create from `storage.nfs.export_base` to the SPOT
  `STORAGE_NFS_STATE_PATH`, matching the rest of the repo's NFS-path
  references.

- tasks/stages/01_constructor.yml: swap the bootstrap order so
  `svc-storage` runs before `svc-registry`. The registry's volume needs
  the NFS layer ready first; the prior order caused intermittent
  init races where the registry created its volume on local disk
  before NFS was mounted, hiding the actual cluster volume. Also move
  the sys-token-store include after the bootstrap loop so it can use
  the now-mounted shared state.
2026-06-14 19:08:43 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a4cb5b89e1 feat(swarm): swarm-aware container exec via container_address lookup
Adds a mode-aware `container_address` lookup plugin and a
`resolve-container-id` helper script that resolve swarm task IDs at
exec time (compose mode returns bare names unchanged). Migrates ~25
roles + ~28 task files from `command:` to `shell:` so the lookup's
`$(...)` subshell evaluates correctly, and extends `database` lookup
with an `address` key for cross-role DB exec.

Three new lint tests enforce the contract: every container exec
target goes through the resolver, lives behind an IS_STACK_HOST gate
(walks the include graph one hop deep), and uses `shell:` rather than
`command:` when the target is a Jinja reference. Existing nocheck
suppressions on bare-name DB exec sites (nextcloud redis, listmonk,
moodle OIDC, mailu) are replaced with the lookup-based form.

Fixes the systemd Environment= injection in sys-ctl-cln-disc-space
(it never expanded $(...)) by passing STACK / SERVICE_KEY parts and
resolving inside the script. Extracts shopware admin and mediawiki
extension install bodies into dedicated .sh files invoked via env.
Rewrites password-bearing shell tasks to either `| quote` directly or
the `-e VAR=... | quote` + `sh -c '... ${VAR} ...'` env-bridge pattern.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-10 03:46:28 +02:00
kevinveenbirkenbachandClaude Opus 4.7 1931241cc7 refactor(modes): introduce MODE_ASSERT_LOCAL SPOT and migrate preload asserts
Adds MODE_ASSERT_LOCAL to group_vars/all/01_modes.yml:
    MODE_ASSERT_LOCAL: "{{ (MODE_ASSERT | bool) and (IS_STACK_HOST | bool) }}"

The mailu preload-assertion in sys-svc-mail/tasks/01_core.yml previously
fired on every host in `'web-app-mailu' in group_names` with no
MODE_ASSERT gate at all, so swarm workers ran into the assertion every
single CI run even though the preload state lives only on the stack
host. Same shape applies to a dozen other assertions: sys-svc-rdbms
(already double-gated, now collapsed), keycloak's kcadm-driven LDAP /
RBAC / reCAPTCHA / groups-path asserts, the sys-svc-compose handler
validators for compose/swarm config, svc-prx-openresty's 02_assert,
sys-svc-webserver-core's 05_assert, web-app-matomo's bootstrap token
check, web-app-joomla's 05_assert, and the sys-ctl-alm-compose notifier
self-start.

Each asserted state (run_once_<role> flags, kcadm output, openresty
container probes, compose-tree files) lives only on IS_STACK_HOST, so
the new SPOT keeps the assertion semantics intact while skipping the
spurious worker-side failures.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-06 14:18:33 +02:00
kevinveenbirkenbachandClaude Opus 4.7 19254a23b4 refactor(sys-svc-compose): rename docker_compose_flush_handlers → compose_handlers_flush
Pure 1:1 rename (69 insertions / 69 deletions across 66 files) to
finish the ``docker_*`` → ``compose_*`` alignment started in
8373468e9. The flag configures whether the compose include flushes
handlers right after run; the ``docker_`` prefix was misleading
(handlers are Ansible-side, not docker-side) and the variable is part
of the same ``sys-svc-compose`` SPOT cluster as ``compose_repository_*``.

Touches the role default, the implementation
(``sys-svc-compose/tasks/utils/up.yml``), all 60+ consumer roles,
the network-utility wrapper + its README, and the
``test_network_create_via_util.py`` lint (both the docstring guidance
and the asserted diff).

Breaking change for any external override that still sets the old
name — no alias is provided.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 13:43:43 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3f6e2dd68d refactor(sys-svc-compose): rename docker_git_repository_* → compose_*
Pure 1:1 rename (60 insertions / 60 deletions across 30 files) to
align the variable names with the role that owns them and to drop the
misleading ``docker_git_`` prefix — these knobs configure the
compose-stack's git clone step, not the docker CLI:

- docker_git_repository_address → compose_repository_url
- docker_git_repository_version → compose_repository_ref
- docker_git_repository_pull    → compose_pull_repository
- docker_repository_path        → compose_repository_path

``ref`` better captures git refs (branches/tags/SHAs are not
"versions"); ``url`` is the idiomatic name for the clone source.

Touches every consumer (web-app-akaunting/bigbluebutton/dashboard/
fediwall/hugo/littlejs/mini-qr/minio/openproject/openwebui/
roulette-wheel/taiga + svc-db-openldap/svc-prx-openresty/sys-svc-rdbms
+ web-svc-collabora/coturn/logout/onlyoffice + sys-front-inj-all), the
role's own defaults/tasks, the bigbluebutton compose_mods filter and
its unit test, the declared-in-services lint, and the hugo
requirements doc.

Breaking change for any external override that still sets the old
names — no alias is provided.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 13:43:43 +02:00
kevinveenbirkenbachandClaude Opus 4.7 951f6d36d2 chore(lint): unify autoformat/galaxy parallelism into one PARALLEL env var
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL,
  declare it at file start with default `true`; still env-overridable.
  Reword the worker comment so the literal `# shellcheck` token no
  longer appears at start-of-line and trips SC1072/SC1073.
* scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE /
  GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure
  is always fatal. NAMESPACE is an internal constant. Add the same
  PARALLEL env var (default `true`, env-overridable); falsy demotes the
  xargs fan-out to one worker.
* docs/contributing/environment/variables.md: collapse the four
  AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both
  consumers.
* docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere;
  add the lint-galaxy parallelism note.

Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 00:59:24 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c52e4269bf chore(tooling): integrate mbake (Makefile linter/formatter) + working-tree drift sync
Makefile lint integration:
* scripts/install/lint.sh: replace the prior checkmake binary install with
  ensure_mbake (mbake is pip-installable, parses our Makefile correctly
  whereas checkmake bailed on .ONESHELL: / .SHELLFLAGS: / ifneq).
* scripts/lint/makefile.sh: new runner — `mbake format --check Makefile`
  reports formatting drift and missing .PHONY declarations.
* scripts/lint/autoformat.sh: add a two-pass `mbake format Makefile`
  (mbake's formatter is not strictly idempotent on the first pass).
* Makefile: split the multi-line `.PHONY:` block into 18 single-line
  `.PHONY:` declarations so mbake's parser can validate it; add the
  previously-missing `install-python-dev` and `build-dependency` PHONY
  targets; add `lint-makefile` aggregate-and-direct target.
* .pre-commit-config.yaml: new make-lint-makefile hook scoped to
  `^Makefile$`.
* .github/workflows/lint-makefile.yml: new CI workflow (pip-install mbake
  + make lint-makefile).
* .github/workflows/ci-orchestrator.yml: wire lint-makefile into
  lint-gate.needs.

Misc:
* .gitignore: add roles/importer_result.json (tooling artefact written by
  galaxy-importer when it runs in roles/).

Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 00:32:44 +02:00
kevinveenbirkenbach c1d50d78da refactor(container): replace BIN_CONTAINER usages with system-wide container wrapper
https://chatgpt.com/share/698660b7-4c78-800f-b25f-af5408cff034
2026-02-06 23:00:21 +01:00
kevinveenbirkenbach ba35d7c5e0 refactor(container): replace docker ps usages with container wrapper and move binaries to /usr/bin
https://chatgpt.com/share/698660b7-4c78-800f-b25f-af5408cff034
2026-02-06 22:44:14 +01:00
kevinveenbirkenbach 2147690da0 Use container wrapper for docker logs across roles
https://chatgpt.com/share/698659d9-62e4-800f-89a1-5d44f309bde4
2026-02-06 22:28:38 +01:00
kevinveenbirkenbach 46238d4c89 refactor: use BIN_CONTAINER wrapper for container exec commands
https://chatgpt.com/share/698657ee-e708-800f-ba5d-4c7c5198a5fe
2026-02-06 22:06:42 +01:00
kevinveenbirkenbach 63926b23b9 chore(container): replace direct docker inspect/run calls with container wrapper
https://chatgpt.com/share/698647e2-7b94-800f-814d-5eb63fe4e537
2026-02-06 20:58:17 +01:00
kevinveenbirkenbach 48237d758c Rename nginx_paths lookup to nginx and update all references
* Rename lookup plugin: lookup_plugins/nginx_paths.py -> lookup_plugins/nginx.py
* Update lookup usage from lookup('nginx_paths', ...) to lookup('nginx', ...) across roles/templates/vars
* Update unit tests accordingly (imports + patch targets)

https://chatgpt.com/share/6985e705-4040-800f-a779-347f06e46f82
2026-02-06 14:05:04 +01:00
kevinveenbirkenbach 8fe6b92e63 refactor(lookups): migrate nginx_paths, tls_resolve and cert_plan to strict positional want-path API
- Replace legacy want= keyword usage with mandatory positional want-path
- Align all nginx_paths, tls_resolve and cert_plan lookups to strict API
- Keep behavior unchanged while enforcing consistent lookup semantics

https://chatgpt.com/share/6985d642-b40c-800f-9cdf-ed54fc484a92
2026-02-06 13:06:09 +01:00
kevinveenbirkenbach 51239968fd refactor: rename docker-compose roles to sys-svc-compose and update paths
Renamed docker-compose and docker-compose-ca roles to sys-svc-compose and sys-svc-compose-ca, updated template includes, handlers, lookups, tests, and variables, and moved compose instances to /opt/compose.

https://chatgpt.com/share/6985ab2c-8344-800f-9e9a-768f562d138d
2026-02-06 09:49:42 +01:00
kevinveenbirkenbach b5bcc4861a Fix: pin application_id when including docker-compose to prevent handler/templating bleed
https://chatgpt.com/share/6984e9b7-400c-800f-84e5-f03a6e9ee18f
2026-02-05 20:04:00 +01:00
kevinveenbirkenbach 8a81630391 Added newline at file and POSIX compatible 2026-02-04 19:49:56 +01:00
kevinveenbirkenbach 82e2cde747 refactor(nginx): replace NGINX vars with nginx_paths lookup and add http/https support
- Drop legacy NGINX path definitions from group_vars
- Add nginx_paths lookup plugin (paths + ensure dirs + http_includes)
- Update roles/templates to use nginx_paths for config/data/cache/user paths
- Add per-domain HTTP->HTTPS redirect + ACME challenge include
- Add unit tests for nginx_paths lookup
- Clean up obsolete sys-svc-letsencrypt role files

https://chatgpt.com/share/697d4466-af54-800f-a243-b28367638c85
2026-01-31 00:52:42 +01:00
kevinveenbirkenbach 3c567ec35b fix(openresty): ensure nginx config directory exists before bootstrap copy
Create the nginx configuration directory before writing the base nginx.conf
to avoid failures when the path does not yet exist.

https://chatgpt.com/share/69675c37-07c8-800f-92bd-393ec344f35c
2026-01-14 10:04:50 +01:00
kevinveenbirkenbach 4f0e45159d Optimized database_type loading 2026-01-10 16:51:18 +01:00
kevinveenbirkenbach 1dac0fe82e fix(openresty): bootstrap minimal nginx.conf before container startup
Ensure OpenResty can start even when no configuration is present yet by
providing a minimal, valid nginx.conf bootstrap.

This decouples container startup from configuration deployment and prevents
handler execution from failing when Docker/OpenResty is initialized before
NGINX config templates are rendered.

https://chatgpt.com/share/695ff412-4324-800f-a76d-27ff2bf2dff7
2026-01-08 19:13:58 +01:00
kevinveenbirkenbach 9251c2ad8a svc-prx-openresty: simplify handler and move validation to assert tasks
- Simplify OpenResty restart handler to only perform pre-flight config check
  and container restart
- Move runtime validation (wait + post-flight check) into dedicated assert tasks
- Prepare assert workflow for unconditional log output and clearer diagnostics
- Remove complex block/rescue logic from handler context

https://chatgpt.com/share/6949582b-28a8-800f-9f7c-2949a8258c7a
2025-12-22 15:39:24 +01:00
kevinveenbirkenbach 73ec220847 Added utils/load_handlers.yml required due to update 2.18 -> 2.20 2025-12-04 11:04:00 +01:00
kevinveenbirkenbach 5c67c42911 Restructured Handler Logic 2025-12-04 02:37:12 +01:00
kevinveenbirkenbach 031775c096 Removed redundancies and simplified run_once flag 2025-12-03 18:12:09 +01:00
kevinveenbirkenbach 1d0f26617a Optimized docker_git_repository_ variables 2025-12-03 14:23:53 +01:00
kevinveenbirkenbach 716ebef33b Refactor task includes and update variable handling for Ansible 2.20 migration
This commit updates multiple roles to ensure compatibility with Ansible 2.20.
Several include paths and task-loading mechanisms required adjustments,
as Ansible 2.20 applies stricter evaluation rules for complex Jinja expressions
and no longer resolves certain relative include paths the way Ansible 2.18 did.

Key changes:
- Replaced legacy once_finalize.yml and once_flag.yml with the new structure
  under tasks/utils/once/finalize.yml and tasks/utils/once/flag.yml.
- Updated all include_tasks statements to use 'path_join' with playbook_dir,
  ensuring deterministic and absolute file resolution across roles.
- Fixed all network helper includes by converting direct relative paths such as
  'roles/docker-compose/tasks/utils/network.yml' to proper Jinja-evaluated paths.
- Normalized MATOMO_* variable names for consistency with the updated variable
  scope behavior in Ansible 2.20.
- Removed deprecated patterns that were implicitly supported in Ansible 2.18
  but break under the more strict variable and path resolution model in 2.20.

These changes are part of the full migration step required to ensure the
infinito-nexus roles remain stable, deterministic, and forward-compatible with
Ansible 2.20.

Details of the discussion and reasoning can be found in this conversation:
https://chatgpt.com/share/69300a8d-24d4-800f-bec0-e895a695618a
2025-12-03 11:02:34 +01:00
kevinveenbirkenbach e09f561f0b Refactor run-once orchestration and bootstrap Mailu/Mastodon in a single deploy
- Replace legacy utils/run_once.yml with the new helpers utils/once_flag.yml and utils/once_finalize.yml
- Introduce utils/compose_up.yml to ensure docker-compose stacks are up and to flush handlers safely without coupling to run-once flags
- Migrate all affected roles (desk-*, dev-*, sys-ctl-*, sys-svc-*, web-app-*, web-svc-*, util-*) to the new run-once helpers
- Rework sys-svc-msmtp to auto-load Mailu once per deploy, check reachability, and reuse the running stack instead of requiring multiple playbook passes
- Adjust web-app-mailu to integrate cert deployment, handler flushing, and run-once handling so Mailu is fully initialized in a single deploy
- Improve Matomo, CDN, logout and CSP/health-check related roles to cooperate with the new compose_up / once_* pattern
- Simplify alarm/backup/timer/service orchestration (sys-ctl-alm-*, sys-bkp-provider, sys-timer-cln-bkps, etc.) by moving run-once logic into dedicated 01_core.yml files
- Update integration tests so utils/once_flag.yml and utils/once_finalize.yml are recognised as valid run-once providers, keeping the global run_once_* guarantees consistent
- Align frontend injection and service dependencies so Mastodon- and Mailu-related services can be brought up coherently within a single deployment cycle rather than several iterations
2025-12-01 13:30:50 +01:00
kevinveenbirkenbach e6803e5614 refactor(ansible): normalize include_role syntax and unify host config paths via path_join
- Remove stray spaces after include_role: across many roles to ensure clean YAML and
  consistent linting/formatting.
- Listmonk:
  - Introduce LISTMONK_CONFIG_HOST = [ docker_compose.directories.config, 'config.toml' ] | path_join
  - Use that var in the template task (dest) and the docker-compose volume mount
- Matrix:
  - Build MATRIX_SYNAPSE_CONFIG_PATH_HOST, MATRIX_SYNAPSE_LOG_PATH_HOST, and
    MATRIX_ELEMENT_CONFIG_PATH_HOST via path_join
- Mobilizon:
  - Build mobilizon_host_conf_exs_file via path_join
  - Keep get_app_conf strictness unchanged (defaults to True in our filter), so behavior
    remains strict even though the explicit third arg was dropped
- Simpleicons:
  - Build server.js and package.json host paths via path_join
- Numerous web-app roles (Confluence, Discourse, EspoCRM, Friendica, Funkwhale, Gitea,
  GitLab, Jenkins, Joomla, Listmonk, Mailu, Mastodon, Matomo, Matrix, MediaWiki,
  Mobilizon, Moodle, Nextcloud, OpenProject, Peertube, Pixelfed, Pretix, Roulette Wheel,
  Snipe-IT, Syncope, Taiga, WordPress, XWiki, Yourls) and web-svc roles (coturn,
  libretranslate, simpleicons) updated for consistent include_role formatting

Why:
- path_join avoids double slashes and missing separators across different config roots
- Consistent include_role: formatting improves readability and prevents linter noise

Ref:
- Conversation: https://chatgpt.com/share/68d14711-727c-800f-b454-7dc4c3c1f4cb
2025-09-22 14:55:25 +02:00