Commit Graph
43 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 06f5905d64 fix(sys-svc-rdbms): patch the persisted password before the health gate again
1f870b1e5 gave the patch one call site and put it in post_deploy.yml, which runs
after the deploy has waited for the stack. 3831c8f804 had named the constraint in
its own subject line: before the health gate. An app whose health check
authenticates against its own database fails that check with the password it
persisted, compose gives up with "dependency failed to start: container matomo is
unhealthy", and nothing behind the gate ever runs -- including the patch that
would have fixed it.

Measured, not argued: with the call in post_deploy.yml a local compose deploy of
five roles passed its first pass at ok=4571 failed=0 and died in the second, the
one that carries the rotated credential. With the call back in front, all three
passes report failed=0 and the Playwright specs pass in both.

sys-svc-rdbms/tasks/main.yml is where it goes: the stack include reaches it
before anything comes up, it is per-consumer rather than per-role-once, and it
still resolves the declaration through the same path, so there is still exactly
one call site. The patch reads the volume from the host and needs no running
container, which is what makes the earlier position possible at all; before the
first deploy has created the volume it exits without doing anything.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 11:04:04 +02:00
kevinveenbirkenbachandClaude Opus 5 ea2f13c87c fix(sys-svc-rdbms): stat the compose template where its path is meaningful
path_absolute builds from playbook_dir, a controller-side value, but the task
carried no delegate_to and therefore ran on the managed node. Anywhere the
controller is not the target - every swarm node, every remote deploy - the
path does not exist, stat.exists is false and the dedicated branch is skipped
for every role, taking the env file, the initdb setup and the backup seed with
it. Silently, while the rendered service still points env_file at a file
nobody wrote.

A single-host compose run cannot see this: there the controller and the target
are the same machine, which is why 47c01259e looked green.

Delegating alone is enough. run_once would additionally hand one host's answer
to every other host of a `hosts: all` play whose include is gated per host,
making the registered result depend on host ordering rather than on the host
being configured - for a filesystem check that is identical for all of them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 23:28:13 +02:00
kevinveenbirkenbachandClaude Opus 5 f13ba5dc15 style(sys-svc-rdbms): fit the compose-template lookup's name in one line
47c01259e named the task 87 characters wide; the convention allows 80 so the
name stays readable in a playbook run. The lint that caught it also says where
a longer explanation belongs, and it is not the task name.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 23:28:13 +02:00
kevinveenbirkenbachandClaude Opus 5 c158c15aaf fix(sys-svc-rdbms): set up only the engine this role actually renders
A dedicated database is not the same as one this role provides.
web-app-bigbluebutton declares postgres enabled and unshared, so the lookup
reports it local and the dedicated branch ran - rendering an env file, an
initdb setup and a backup row for an engine bbb-docker brings itself. Nothing
of that output is read: bigbluebutton never calls the database lookup, ships
its own env.j2 and vendors its own initdb hook.

The backup row was the one that hurt. 7d044c265 taught the dedicated branch to
seed, and it seeded credentials derived from the entity name into a postgres
that has no such role:

    pg_dump -U bigbluebutton -d bigbluebutton
    FATAL: role "bigbluebutton" does not exist

bigbluebutton seeds its own row correctly - the '*' cluster row with the
superuser its image actually creates - so the drill now has one working row
and one impossible one for the same instance.

Whether this role provides the engine is already written down: its service
template is reached through <role>/templates/compose.yml.j2 -> base.yml.j2, so
a role without that template never gets one. Across the 53 roles with a
database service exactly one lacks it, and none has it without including
base.yml.j2, which makes the condition exact rather than a stand-in for "is
this bigbluebutton". Reading it beats declaring it: a flag would be a second
copy of a fact the tree already states, free to drift from it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 23:28:12 +02:00
kevinveenbirkenbachandClaude Opus 5 7d044c2652 fix(sys-svc-rdbms): seed dedicated databases into the backup, not just central ones
baudolo dumps only what databases.csv names, and the entry was seeded from the central branch alone. An application running its own engine - Discourse with svc-db-postgres absent from the group - therefore never reached the seed: databases.csv stayed empty, baudolo warned that it was continuing without dumps, and the volume was stored as a raw PGDATA tree that restores a torn cluster.

Whether a database must be backed up is a property of there being one, not of it being central, so both branches now include the same helper with their own gate: centrally the host from the provider group seeds, dedicated the stack host itself. The coordinates come from the same lookup either way.

Verified without a deploy that the row this writes is the row baudolo looks for: the seed records the consumer entity as the instance, and baudolo derives the same key from the container name (discourse-database -> discourse). A mismatch there would have filled databases.csv and still produced no dump.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 03:22:04 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4039313266 refactor(tasks): enforce and apply a single task-file numbering scheme
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.

Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:31 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f76bfcf531 refactor(svc-bkp-volume-2-local): rename role from svc-bkp-container-2-local
Rename the role, its test tree, the primary entity key
(volume-2-local), the derived vars (BKP_VOLUME_2_LOCAL_*,
SYS_SERVICE_BACKUP_VOLUME_2_LOCAL), the run-once fact, and every
group_names consumer flag, include path, doc, and lint reference. The
container_backup service key and the on-disk backup-docker-to-local
tree keep their names: both are contracts with existing consumer
entries, snapshot trees, and the baudolo pull tooling.

Operators must rename the svc-bkp-container-2-local inventory group and
disable the previous versioned units on already-provisioned hosts; the
pre-deploy snapshot glob derives its base from the new role name and
will not find them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 18:52:02 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3cf83bfab1 fix(rdbms): reset compose_mode_force when initing the shared DB provider
A consumer that forces a per-role deploy mode (web-app-matrix ansible-flavor and web-app-bigbluebutton set compose_mode_force=compose) leaked that value through the include_role chain into the shared database provider (svc-db-postgres/mariadb). In a swarm cluster the provider then ran via 'docker compose up' (bridge) instead of 'docker stack deploy' (overlay), breaking the shared DB. Reset compose_mode_force to empty on the provider include, alongside the existing compose_repository_pull override, so the shared provider always follows the cluster DEPLOYMENT_MODE. Covers every consumer, not just matrix/bbb.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 00:31:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 23d1447566 refactor(ansible): add path_absolute lookup, replace playbook_dir joins
Introduce plugins/lookup/path_absolute.py as the single source for the
repo-root anchor: lookup('path_absolute', 'roles/...') returns an absolute
path under playbook_dir, joining every term. Replace all hand-built
playbook_dir path constructions across .yml/.yaml and .j2 templates with it:

  [playbook_dir, '<rel>'] | path_join   -> lookup('path_absolute', '<rel>')
  playbook_dir ~ '/<rel>'               -> lookup('path_absolute', '<rel>')
  {{ playbook_dir }}/<rel>              -> lookup('path_absolute', '<rel>')

Unrelated repo_path symbols in cli/meta/roles/applications/resolution and the
single bare discover_playwright_roles(playbook_dir) call are left untouched.

Add tests/lint/ansible/test_playbook_dir_via_lookup.py: fails the build if any
of the three construction forms reappears in .yml/.yaml/.j2, pointing callers
at the lookup. Unit test covers the plugin.

Touching these files pulled their pre-existing narration comments into
test_comments_valid scope; clean them (delete restatement/banners, keep real
trip-wires as Workaround/Upstream/Invariant markers). Drop caution, important
and pitfall from the comment-lint marker vocabulary and remove the comments
that used them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 20:36:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 cddb0dc59b feat(lint): require emoji-prefixed task/step/job names + convert existing
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.

- emit emoji names at the source: cli/build/include generator now writes
  "🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
  tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
  task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
  🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 22:45:06 +02:00
kevinveenbirkenbachandClaude Opus 4.7 59c221521c fix(cluster-a): NFS-everywhere for cluster-shared /var/lib/infinito state
Resolves CI swarm rc=69 ("msmtp: needs a password"): controller-side
Ansible lookups (lookup('email').password etc.) couldn't see the
tokens.yml that Mailu writes target-side, because the controller has
its own filesystem in swarm mode.

Mount NFS-shared /var/lib/infinito on every swarm host AND on the
controller during deploy (in-play delegate_to: localhost task at end
of svc-storage-nfs-server role). State subdir exported separately
with no_root_squash + wildcard CIDRs so root-owned writes succeed
and the controller can mount from outside the svc-swarm IP set.

Single-writer guarantee on FILE_TOKENS via delegate_to: STACK_HOST +
run_once on sys-token-store/{01_core,write}.yml. Backup roles
(svc-bkp-container-2-local, sys-ctl-cln-bkps, sys-ctl-cln-faild-bkps,
user-backup/03_permissions_folders, three direct callers of
04_seed-database-to-backup.yml) gated on IS_STACK_HOST to prevent
N-host write races on the shared NFS backup tree.

Fail-fast assertion in svc-storage-nfs-client + 05_controller_mount
when pre-existing local /var/lib/infinito would be hidden by mount-
over. Operator must migrate or wipe before re-deploying.

SPOTs: SWARM_LAB_NETWORK in workflow env block, storage.nfs.state_subdir
in group_vars/all/15_storage.yml, DIR_VAR_LIB unchanged. UPPER_CASE
constants in NFS roles. Cleanup script umounts controller view before
container teardown to prevent stale-mount leak between matrix jobs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-09 19:02:57 +02:00
kevinveenbirkenbachandClaude Opus 4.7 1931241cc7 refactor(modes): introduce MODE_ASSERT_LOCAL SPOT and migrate preload asserts
Adds MODE_ASSERT_LOCAL to group_vars/all/01_modes.yml:
    MODE_ASSERT_LOCAL: "{{ (MODE_ASSERT | bool) and (IS_STACK_HOST | bool) }}"

The mailu preload-assertion in sys-svc-mail/tasks/01_core.yml previously
fired on every host in `'web-app-mailu' in group_names` with no
MODE_ASSERT gate at all, so swarm workers ran into the assertion every
single CI run even though the preload state lives only on the stack
host. Same shape applies to a dozen other assertions: sys-svc-rdbms
(already double-gated, now collapsed), keycloak's kcadm-driven LDAP /
RBAC / reCAPTCHA / groups-path asserts, the sys-svc-compose handler
validators for compose/swarm config, svc-prx-openresty's 02_assert,
sys-svc-webserver-core's 05_assert, web-app-matomo's bootstrap token
check, web-app-joomla's 05_assert, and the sys-ctl-alm-compose notifier
self-start.

Each asserted state (run_once_<role> flags, kcadm output, openresty
container probes, compose-tree files) lives only on IS_STACK_HOST, so
the new SPOT keeps the assertion semantics intact while skipping the
spurious worker-side failures.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-06 14:18:33 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9cd10057c0 fix(sys-svc-rdbms): gate preload assertion behind MODE_ASSERT + IS_STACK_HOST
The unconditional run_once_<provider> assertion fires on every swarm
deploy where sys-service-loader's loop iteration order does not strictly
honour bucket order (a known race when include_tasks runs across mixed
host scopes). MODE_ASSERT keeps the check available in debug runs to
surface the underlying loader race, while IS_STACK_HOST scopes it to
the host where the provider would actually have been preloaded. In
production the immediately following Initialize task lets svc-db-<x>
self-heal via its own run_once gate, so the assertion is no longer
load-bearing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 20:57:58 +02:00
kevinveenbirkenbachandClaude Opus 4.7 411bcfe4fe refactor: cluster-wide shared-DB detection + openresty-host gating + load_app guard
sys-svc-rdbms now decides central-DB vs dedicated-DB mode from the provider
group's presence in the inventory (groups[provider_id] | length > 0)
instead of the per-host lookup('database', _, 'shared'). The old form
returned different values on different hosts (e.g. mariadb.shared =
'svc-db-mariadb' in group_names), so workers would silently disagree with
the manager about which mode is active and trip the central-init handlers
on the wrong host. The central-init block now also gates on the host
actually being in the provider group.

sys-stk-full guards the sys-stk-front-proxy include behind
"inventory_hostname in groups['svc-prx-openresty']", so under swarm only
the openresty host renders the per-app vhost. Combined with the
compose-mode auto-place from the previous commit, single-host compose and
swarm both pick exactly one host.

tasks/utils/load_app.yml skips loading a role for hosts that are not in
its target group when a group exists, matching the new gating semantic
upstream.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-04 17:11:07 +02:00
kevinveenbirkenbachandClaude Opus 4.7 bfe1d6fd79 refactor(sys-service-loader): rename + post-load queue mechanism
Renames the shared-services loader role from sys-utils-service-loader to
sys-service-loader (shorter, no orphaned "utils" prefix; entity name in
meta/services.yml realigns to service-loader per get_entity_name).

Introduces an explicit two-phase contract in the loader: after the
ordered preload pass, flush a `_service_loader_post_run` queue that
service roles register into during preload via the new helper
`list_or_shoot.yml` (list during preload, shoot directly outside it).

Replaces the `when: not _service_loader_preloading` guards in
sys-svc-compose and sys-svc-webserver-core with helper calls. Closes
the gap where non-web deploys would never load these post-load
dependencies; in normal web deploys behavior stays the same thanks to
each sub-role's `run_once_<role>` gate.

Docs, comments, test docstring, and constructor-stage invocation
follow the rename. Historical changelogs (CHANGELOG.md, packaging/*)
intentionally left at the original name.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 06:48:14 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a08db7fa11 refactor(roles/svc-bkp-*): rename backup roles + make container/local-2-remote invokable
Consolidates the backup-role topology around four directional roles
under the existing svc.bkp category:

  sys-ctl-bkp-docker-2-loc  →  svc-bkp-container-2-local
  svc-bkp-loc-2-usb         →  svc-bkp-local-2-device
  svc-bkp-rmt-2-loc         →  svc-bkp-remote-2-local
  sys-bkp-provider          →  deleted (purely a bundler with no own logic;
                                the two pieces it pulled in — user-backup
                                and sys-timer-cln-bkps — are now declared
                                directly by callers that need them)

Every renamed role moves into svc.bkp (already invokable per
roles/categories.yml), gets an `application_id` in vars/main.yml, and
declares the per-app primary-entity service flags + resource limits the
invokable-role lint requires.

Companion changes:

  - svc-bkp-container-2-local: meta/variants.yml declares three matrix
    rounds (gitea+keycloak, postgres+mariadb, openldap) that pin the
    companion topology via service-key flags the CombinedResolver
    translates into provider-role includes.

  - sys-svc-compose: container-2-local is no longer in the docker-mgmt
    include loop; it joins via a separate `'svc-bkp-container-2-local'
    in group_names` gate so the 3 backup roles are deployable
    independently (each addresses its own failure mode).

  - The three `04_seed-database-to-backup.yml` includes (web-app-matrix,
    web-app-bigbluebutton, sys-svc-rdbms) are gated by the same
    group_names check so consumer apps only seed when the container
    backup is actually in scope.

  - web-app-gitea picks up `enabled: true, shared: true, provides:
    gitea` on its primary entity so the service registry can pull it in
    via a `services.gitea.enabled+shared` flag (needed for the
    container-backup variant 0).

  - group_vars constants and the pyproject ruff per-file-ignore + the
    .ansible-lint-ignore paths follow the role-id rename.

  - The remote-2-local script picks up the snake_case rename
    (`pull-specific-host.py` → `pull_specific_host.py`) and the
    `os.makedirs` → `Path.mkdir` conversion so the role no longer needs
    the PTH/N999 ruff escape hatch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 04:23:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 300ea0a054 feat(sys-svc-rdbms): build per-consumer postgres image with required libraries for shared:false
Generalises the postgres image build to be driven by a declarative
extension→library registry instead of a hardcoded POSTGRES_VECTOR_ENABLED
flag. Same Dockerfile.j2 now serves both the central svc-db-postgres
image (libraries aggregated across all deployed consumers) and per-
consumer images for shared:false postgres consumers whose extensions
need libraries not in the postgis/postgis base (e.g. pgvector).
postgres.yml.j2 switches between image: and build: based on whether
custom libraries are needed. New postgres_libraries lookup filters
base-image extensions out and returns recipes for the rest. Dedicated
task path extracted into dedicated.yml + dedicated_postgres.yml so the
block-with-when size limit stays satisfied and the postgres-type check
happens once.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 16:55:01 +02:00
kevinveenbirkenbachandClaude Opus 4.7 6ad3814e15 feat(sys-svc-rdbms): apply postgres extensions for shared:false consumers via initdb.d mount
sys-svc-rdbms now renders an extensions.sql into a per-consumer
.initdb.d/ directory and bind-mounts it into the dedicated postgres
container at /docker-entrypoint-initdb.d/. CREATE EXTENSION runs as
POSTGRES_USER (= bootstrap superuser) on first initdb so non-trusted
extensions succeed. Same declarative source (service_extensions
lookup, services.yml extensions field) drives both shared:true
(02_init.yml loop) and shared:false (initdb.d) paths. database lookup
exposes a new initdb_dir field; service_extensions lookup loses its
shared:true guard (shared:false is now a real path). Refactored
repeated lookups in sys-svc-rdbms tasks into vars/main.yml.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 16:55:00 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3f6e2dd68d refactor(sys-svc-compose): rename docker_git_repository_* → compose_*
Pure 1:1 rename (60 insertions / 60 deletions across 30 files) to
align the variable names with the role that owns them and to drop the
misleading ``docker_git_`` prefix — these knobs configure the
compose-stack's git clone step, not the docker CLI:

- docker_git_repository_address → compose_repository_url
- docker_git_repository_version → compose_repository_ref
- docker_git_repository_pull    → compose_pull_repository
- docker_repository_path        → compose_repository_path

``ref`` better captures git refs (branches/tags/SHAs are not
"versions"); ``url`` is the idiomatic name for the clone source.

Touches every consumer (web-app-akaunting/bigbluebutton/dashboard/
fediwall/hugo/littlejs/mini-qr/minio/openproject/openwebui/
roulette-wheel/taiga + svc-db-openldap/svc-prx-openresty/sys-svc-rdbms
+ web-svc-collabora/coturn/logout/onlyoffice + sys-front-inj-all), the
role's own defaults/tasks, the bigbluebutton compose_mods filter and
its unit test, the declared-in-services lint, and the hugo
requirements doc.

Breaking change for any external override that still sets the old
names — no alias is provided.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 13:43:43 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2a023ca6a1 chore(lint): autoformat strict-failure semantics + claude.sh rename
* scripts/lint/autoformat.sh:
  - rename run_sort_claude_settings worker to run_claude (matches the
    renamed scripts/lint/claude.sh).
  - drop the EXIT trap that wiped status_dir; cleanup happens explicitly
    after `wait` returns, so a parallel worker that interrupts the
    parent never races a vanished status dir.
  - each worker now captures its tool's exit code and writes one of
    OK / SKIP / FAIL. Tolerated `expected violations remain`-style exits
    (markdownlint-cli2 rc≤1, ansible-lint rc≤2) still count as OK; only
    real failures count as FAIL.
  - all workers run to completion regardless of individual tool failures
    (so the user always sees the full picture); after the summary, the
    script exits 1 if any worker reported FAIL.
* scripts/lint/sort_claude_settings.sh -> scripts/lint/claude.sh
  (history preserved via git mv); update test_claude_settings_sorted.py
  fix-hint accordingly.

Plus working-tree drift across roles/, group_vars/, inventories/,
requirements/, and .pre-commit-config.yaml accumulated from the
make-test setup phase regenerating generated artefacts during recent
test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 01:30:49 +02:00
kevinveenbirkenbach 837d0c5335 test(suppression): unify # noqa / # nocheck marker grammar
Introduce a single grammar for per-line, per-block, and per-file
suppression markers across the test suite, replacing seven ad-hoc
syntaxes that had drifted apart.

Helper
  utils/annotations/suppress.py exposes is_suppressed_at,
  is_suppressed_in_head, is_suppressed_anywhere, line_has_rule, and
  suppressed_line_numbers. Markers may use either `noqa` or `nocheck`
  (case-insensitive synonyms), live inside `#`, `//`, `{# ... #}`, or
  `<\!-- ... -->` comments, and combine multiple kebab-case rules
  comma-separated. 17 unit tests cover the grammar in isolation.

Refactored consumers
  - tests/external/repository/test_urls_reachable.py (rule: url)
  - tests/external/docker/test_image_versions.py via
    utils/docker/version_updater.py (rule: docker-version)
  - tests/lint/repository/test_no_direct_yaml_calls.py (rule:
    direct-yaml)
  - tests/lint/ansible/test_service_shared_consistency.py (rule:
    shared)
  - tests/lint/ansible/roles/test_web_app_email_integration.py (rule:
    email)
  - tests/lint/repository/test_python_file_size.py (rule: file-size)
  - tests/lint/ansible/test_run_once_tags.py +
    tests/integration/roles/run_once/test_schema.py (rules: run-once,
    run-once-suffix)

Marker migration
  12 role tasks/main.yml files: `# run_once_<key>: deactivated`
  -> `# nocheck: run-once`. roles/web-app-suitecrm/meta/services.yml
  drops a malformed inline `noqa: intentional legacy pin` from a TODO
  comment so the new parser does not pick it up as a stray rule.

Caching wins
  Test files that previously loaded YAML / read files via raw
  yaml.safe_load + Path.read_text now route through utils.cache.yaml
  (load_yaml_any / load_yaml_str) and utils.cache.files.read_text:
    - test_service_shared_consistency.py
    - test_web_app_email_integration.py
    - tests/integration/roles/run_once/test_schema.py
    - utils/docker/version_updater.py
  This both speeds up repeated reads and stays consistent with the
  direct-yaml lint policy.

Documentation
  New SPOT page at docs/contributing/actions/testing/suppression.md
  with grammar, position semantics, the catalog of rules, examples,
  and an "adding a new rule" checklist. lint.md and external.md drop
  their per-rule tables and link to the new page. All touched docs
  conform to docs/contributing/documentation.md (no em-dash thought
  breaks, no SPOT self-labels, RFC-2119 keywords used consistently).

Settings sort tooling
  scripts/lint/sort_claude_settings.sh wraps utils/claude/settings_sort.py
  and is wired into scripts/lint/autoformat.sh. The tooling fixes the
  ASCII-ascending requirement enforced by
  tests/lint/repository/test_claude_settings_sorted.py; the test's
  failure message now points operators at the script. .claude/settings.json
  is committed in sorted form.
2026-04-30 11:54:46 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 be6a310d59 refactor(services): implement role-local service registry (req-001)
- Delete group_vars/all/20_services.yml; all service metadata now lives
  in each role's config/main.yml under compose.services.<name>
- Rename generic 'database' service key to explicit 'mariadb'/'postgres'
  across all app roles; remove legacy type: field
- Add utils/service_registry.py and plugins/lookup/service_registry.py
  as the single source of truth for service discovery
- Add utils/database_service.py to centralize DB service resolution
- Delete service_load_enabled filter and service_should_load lookup;
  replaced by service lookup plugin
- Update sys-utils-service-loader to run in constructor stage and use
  generic load_service.yml; remove load_frontend_service.yml
- Add _service_loader_preloading guard to sys-svc-compose,
  sys-svc-webserver-core and sys-ctl-alm-compose to prevent
  health-checks/alarms from firing during preload phase
- Replace manual load_app.yml calls for shared DB/Mailu with assert-
  based preload validation in sys-svc-rdbms and sys-svc-mail
- Update networks.yml.j2 templates to resolve LDAP/AI networks via
  service_registry instead of hardcoded role names
- Migrate all tests to new service discovery API; delete obsolete tests
- Update docs/contributing/code/services.md to reflect role-local model

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 19:33:20 +02:00
kevinveenbirkenbach a55c99ef62 refactor(compose): replace docker lookup with container and unify compose paths
- Rename lookup plugin from `docker` to `container`
- Switch all lookups from `lookup('docker', ...)` to `lookup('container', ...)`
- Normalize compose file keys:
  - docker_compose → compose
  - docker_compose_override → compose_override
  - docker_compose_ca_override → compose_ca_override
- Centralize compose path resolution via docker_paths_utils
- Update compose_file_args to always derive paths from DIR_COMPOSITIONS
- Align all roles, templates, and vars with container/compose naming
- Update and rename unit tests accordingly

This prepares the codebase for engine-agnostic container backends
(e.g. Docker ↔ Podman) and removes legacy docker-specific assumptions.

https://chatgpt.com/share/6987a7cc-d374-800f-99ad-bb4c8f92c59e
2026-02-07 21:59:39 +01:00
kevinveenbirkenbach 0a7b4efa22 Replaced docker compose pull by compose pull 2026-02-06 23:26:20 +01:00
kevinveenbirkenbach 5e33e70892 refactor(database lookup usage): switch conditionals to enabled flag
Replace indirect type!='' checks with explicit database.enabled lookups
to improve semantics, readability, and alignment with strict lookup API.

https://chatgpt.com/share/6985d642-b40c-800f-9cdf-ed54fc484a92
2026-02-06 13:25:37 +01:00
kevinveenbirkenbach 26a5b159e0 refactor(docker-lookup): replace want= syntax with path-based access
- Replace lookup('docker', ..., want='...') with lookup('docker', ..., 'path')
- Allow both top-level keys (directories/files) and dotted paths
- Simplify docker lookup API and reduce lazy dict propagation
- Update unit tests to reflect new lookup semantics

https://chatgpt.com/share/6984ccea-8290-800f-a6d7-ec0e94df95a0
2026-02-05 18:45:43 +01:00
kevinveenbirkenbach efc160d199 Deleted DOCKER_VARS_FILE orphans 2026-02-05 18:26:19 +01:00
kevinveenbirkenbach 56aea31667 refactor(docker): replace docker_compose vars with docker lookup across roles
Remove implicit docker_compose variable usage and resolve all docker paths
(files, directories, env) via lookup('docker', application_id).
This prevents scope leakage, centralizes path resolution, and makes docker
context deterministic across roles and handlers.

https://chatgpt.com/share/6984ccea-8290-800f-a6d7-ec0e94df95a0
2026-02-05 18:15:10 +01:00
kevinveenbirkenbach d9d2af4a39 Solved templating error 2026-02-05 11:23:14 +01:00
kevinveenbirkenbach 135f366369 refactor(sys-svc-rdbms): switch central DB logic to database lookup plugin
- group central database creation and backup seeding into a single block
- resolve database metadata (type, instance, host, etc.) via lookup('database')
- remove legacy per-task central DB handling
- prepare sys-svc-rdbms for unified database plugin usage

https://chatgpt.com/share/6983cd01-a430-800f-9fe7-36551b14d30d
2026-02-04 23:49:23 +01:00
kevinveenbirkenbach e194981535 refactor(backup): make database seeding strict and explicit
- enforce explicit database parameters for baudolo seeding
- replace implicit checks with assert-based fail-fast logic
- update rdbms, matrix and bigbluebutton roles to pass all db fields
- keep backup behavior unchanged while hardening configuration

https://chatgpt.com/share/6983c9e4-860c-800f-b0f7-28462c08e249
2026-02-04 23:36:08 +01:00
kevinveenbirkenbach 4b7200b451 lookup(database): add stable db role id and switch consumers to use it
- Extend database lookup with `id` (e.g. svc-db-postgres, svc-db-mariadb)
- Return empty id when no database type is configured
- Replace dynamic 'svc-db-{{ type }}' constructions with lookup-provided id
- Update docker network templates to rely on lookup database id
- Adjust sys-svc-rdbms to load DB role via lookup id
- Extend unit tests to assert correct id behavior for all cases

This makes DB role resolution explicit, safer, and less error-prone.

https://chatgpt.com/share/6983c6de-c5f8-800f-a5e3-786f90e0ebc1
2026-02-04 23:23:03 +01:00
kevinveenbirkenbach f6f96a11ac Refactor: use database lookup helper everywhere
https://chatgpt.com/share/6983bcf6-85bc-800f-a0fc-5684b9078469
2026-02-04 22:52:08 +01:00
kevinveenbirkenbach 3e8abe3cb4 fix(rdbms): preserve database_consumer_id across role boundaries
- Introduce database_consumer_id_passed to avoid fact/vars collisions
- Explicitly pass consumer context into svc-db-postgres via include_role
- Strengthen context assertion in postgres init tasks
- Move once-flag include to the beginning of postgres core tasks
- Make POSTGRES_EXEC_SHELL non-interactive by injecting PGPASSWORD
- Improve test coverage to detect facts overridden by include-time vars

https://chatgpt.com/share/69833e56-93a0-800f-8e3d-d970cf87daef
2026-02-04 13:40:44 +01:00
kevinveenbirkenbach 77a0bc5508 svc-db-postgres: unify docker exec shell + tidy yaml formatting
- Align database config key spacing and add missing newline in config/main.yml
- Run socket readiness check as OS user postgres
- Introduce POSTGRES_EXEC_SHELL helper and use it for TCP/auth debug psql calls
- Update sys-svc-rdbms comment (behavior note seems outdated)

https://chatgpt.com/share/6982a0b9-be28-800f-b5dc-d489b1e4eed8
2026-02-04 02:28:01 +01:00
kevinveenbirkenbach 4f0e45159d Optimized database_type loading 2026-01-10 16:51:18 +01:00
kevinveenbirkenbach 73ec220847 Added utils/load_handlers.yml required due to update 2.18 -> 2.20 2025-12-04 11:04:00 +01:00
kevinveenbirkenbach 1d0f26617a Optimized docker_git_repository_ variables 2025-12-03 14:23:53 +01:00
kevinveenbirkenbach 716ebef33b Refactor task includes and update variable handling for Ansible 2.20 migration
This commit updates multiple roles to ensure compatibility with Ansible 2.20.
Several include paths and task-loading mechanisms required adjustments,
as Ansible 2.20 applies stricter evaluation rules for complex Jinja expressions
and no longer resolves certain relative include paths the way Ansible 2.18 did.

Key changes:
- Replaced legacy once_finalize.yml and once_flag.yml with the new structure
  under tasks/utils/once/finalize.yml and tasks/utils/once/flag.yml.
- Updated all include_tasks statements to use 'path_join' with playbook_dir,
  ensuring deterministic and absolute file resolution across roles.
- Fixed all network helper includes by converting direct relative paths such as
  'roles/docker-compose/tasks/utils/network.yml' to proper Jinja-evaluated paths.
- Normalized MATOMO_* variable names for consistency with the updated variable
  scope behavior in Ansible 2.20.
- Removed deprecated patterns that were implicitly supported in Ansible 2.18
  but break under the more strict variable and path resolution model in 2.20.

These changes are part of the full migration step required to ensure the
infinito-nexus roles remain stable, deterministic, and forward-compatible with
Ansible 2.20.

Details of the discussion and reasoning can be found in this conversation:
https://chatgpt.com/share/69300a8d-24d4-800f-bec0-e895a695618a
2025-12-03 11:02:34 +01:00
kevinveenbirkenbach 0756e9d06e Optimized database variables 2025-12-02 11:59:55 +01:00
kevinveenbirkenbach fa6bb67a66 Removed whitespaces in templates: 2025-09-22 16:28:57 +02:00
kevinveenbirkenbach 577767bed6 sys-svc-rdbms: Refactor database service templates and add version support for Magento
- Unified Jinja2 variable spacing in tasks and templates
- Introduced database_image and database_version variables in vars/database.yml
- Updated mariadb.yml.j2 and postgres.yml.j2 to use {{ database_image }}:{{ database_version }}
- Ensured env file paths and includes are consistent
- Prepared support for versioned database images (needed for Magento deployment)

Ref: https://chatgpt.com/share/68b96a9d-c100-800f-856f-cd23d1eda2ed
2025-09-04 12:32:34 +02:00
kevinveenbirkenbach 6ea8301364 Refactor: migrate cmp/* and srv/* roles into sys-stk/* and sys-svc/* namespaces
- Removed obsolete 'cmp' category, introduced 'stk' category (fa-bars-staggered icon).
- Renamed roles:
  * cmp-db-docker → sys-stk-back-stateful
  * cmp-docker-oauth2 → sys-stk-back-stateless
  * srv-domain-provision → sys-stk-front
  * cmp-db-docker-proxy → sys-stk-full-stateful
  * cmp-docker-proxy → sys-stk-full-stateless
  * cmp-rdbms → sys-svc-rdbms
- Updated all include_role references, vars, templates and README.md files.
- Adjusted run_once comments and variable paths accordingly.
- Updated all web-app roles to use new sys-stk/* and sys-svc/* roles.

Conversation: https://chatgpt.com/share/68b0ba66-09f8-800f-86fc-76c47009d431
2025-08-28 22:23:09 +02:00