Commit Graph
32 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.8 4039313266 refactor(tasks): enforce and apply a single task-file numbering scheme
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.

Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:31 +02:00
kevinveenbirkenbachandClaude Fable 5 40e553069d refactor(structure): nest prefix-grouped sibling files into subfolders
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.

Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 20:00:03 +02:00
kevinveenbirkenbachandClaude Opus 4.8 cddb0dc59b feat(lint): require emoji-prefixed task/step/job names + convert existing
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.

- emit emoji names at the source: cli/build/include generator now writes
  "🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
  tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
  task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
  🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 22:45:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0bbd6c43ff fix(postgres): pin provider context on 02_init for nested/looped consumers
svc-db-postgres 02_init.yml asserts application_id == 'svc-db-postgres' AND database_consumer_id \!= 'svc-db-postgres'. When 01_core is skipped (run-once), the include_role application_id var does not reliably reach 02_init through a consumer's nested looped include (web-app-matrix's per-bridge DB seeding), so the assert fails. Pin application_id on the 02_init include so it holds for any caller. On the matrix side, freeze the consumer id eagerly before the provider include so database_consumer_id stays the matrix consumer instead of lazily resolving to svc-db-postgres once application_id is pinned.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 11:17:59 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c52e4269bf chore(tooling): integrate mbake (Makefile linter/formatter) + working-tree drift sync
Makefile lint integration:
* scripts/install/lint.sh: replace the prior checkmake binary install with
  ensure_mbake (mbake is pip-installable, parses our Makefile correctly
  whereas checkmake bailed on .ONESHELL: / .SHELLFLAGS: / ifneq).
* scripts/lint/makefile.sh: new runner — `mbake format --check Makefile`
  reports formatting drift and missing .PHONY declarations.
* scripts/lint/autoformat.sh: add a two-pass `mbake format Makefile`
  (mbake's formatter is not strictly idempotent on the first pass).
* Makefile: split the multi-line `.PHONY:` block into 18 single-line
  `.PHONY:` declarations so mbake's parser can validate it; add the
  previously-missing `install-python-dev` and `build-dependency` PHONY
  targets; add `lint-makefile` aggregate-and-direct target.
* .pre-commit-config.yaml: new make-lint-makefile hook scoped to
  `^Makefile$`.
* .github/workflows/lint-makefile.yml: new CI workflow (pip-install mbake
  + make lint-makefile).
* .github/workflows/ci-orchestrator.yml: wire lint-makefile into
  lint-gate.needs.

Misc:
* .gitignore: add roles/importer_result.json (tooling artefact written by
  galaxy-importer when it runs in roles/).

Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 00:32:44 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 be6a310d59 refactor(services): implement role-local service registry (req-001)
- Delete group_vars/all/20_services.yml; all service metadata now lives
  in each role's config/main.yml under compose.services.<name>
- Rename generic 'database' service key to explicit 'mariadb'/'postgres'
  across all app roles; remove legacy type: field
- Add utils/service_registry.py and plugins/lookup/service_registry.py
  as the single source of truth for service discovery
- Add utils/database_service.py to centralize DB service resolution
- Delete service_load_enabled filter and service_should_load lookup;
  replaced by service lookup plugin
- Update sys-utils-service-loader to run in constructor stage and use
  generic load_service.yml; remove load_frontend_service.yml
- Add _service_loader_preloading guard to sys-svc-compose,
  sys-svc-webserver-core and sys-ctl-alm-compose to prevent
  health-checks/alarms from firing during preload phase
- Replace manual load_app.yml calls for shared DB/Mailu with assert-
  based preload validation in sys-svc-rdbms and sys-svc-mail
- Update networks.yml.j2 templates to resolve LDAP/AI networks via
  service_registry instead of hardcoded role names
- Migrate all tests to new service discovery API; delete obsolete tests
- Update docs/contributing/code/services.md to reflect role-local model

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 19:33:20 +02:00
kevinveenbirkenbach 51239968fd refactor: rename docker-compose roles to sys-svc-compose and update paths
Renamed docker-compose and docker-compose-ca roles to sys-svc-compose and sys-svc-compose-ca, updated template includes, handlers, lookups, tests, and variables, and moved compose instances to /opt/compose.

https://chatgpt.com/share/6985ab2c-8344-800f-9e9a-768f562d138d
2026-02-06 09:49:42 +01:00
kevinveenbirkenbach 2d2e218fe5 Solved templating warnings 2026-02-05 11:00:34 +01:00
kevinveenbirkenbach 34b94b7059 More explicit overwritting to harden postgres 2026-01-18 20:01:58 +01:00
kevinveenbirkenbach db68f16e19 Added vars wrapper for RDMBS 2025-12-04 10:58:05 +01:00
kevinveenbirkenbach 27c399123b Optimized handler loading. Requirerd due to 2.18->2.20 update 2025-12-04 00:53:27 +01:00
kevinveenbirkenbach 379b1d420e Refactor MariaDB and PostgreSQL roles for Ansible 2.20 compatibility and Infinito.Nexus conventions
- Replace legacy docker_container-based MariaDB deployment with docker-compose based workflow
- Add custom Dockerfile and docker-compose templates for MariaDB
- Split MariaDB command into separate arguments to avoid entrypoint parsing errors
- Introduce MARIADB_CUSTOM_IMAGE and MARIADB_EXPOSE_LOCAL variables
- Add docker_compose_flush_handlers to ensure correct handler execution on first run
- Replace utils/once/finalize.yml with utils/once/flag.yml for new run-once semantics
- Align variable naming with Infinito.Nexus UPPERCASE conventions
- Fix PostgreSQL custom image variable name (POSTGRES_CUSTOM_IMAGE_NAME → POSTGRES_CUSTOM_IMAGE)
- Remove obsolete flush_handlers var injection in svc-db-postgres/tasks/main.yml
- General cleanup after migration from Ansible 2.18 → 2.20

Conversation reference:
https://chatgpt.com/share/69306c81-9934-800f-b317-f53a8f246a73
2025-12-03 18:00:30 +01:00
kevinveenbirkenbach 8314d7e6a6 Refactor DB initialization handling: externalize database_init flag for svc-db-mariadb and svc-db-postgres, remove internal auto-detection, and pass explicit init signal from sys-stk-back-stateful. Improves clarity, avoids incorrect credential lookups, and stabilizes central DB setup.
Reference: ChatGPT conversation https://chatgpt.com/share/692def1c-a79c-800f-b7dd-35e1845424bf
2025-12-01 20:40:30 +01:00
kevinveenbirkenbach 81304934d5 Removed another block 2025-12-01 05:42:26 +01:00
kevinveenbirkenbach 5d1210d651 feat(ai): introduce dedicated AI roles and wiring; clean up legacy AI stack
• Add svc-ai category under roles and load it in constructor stage

• Create new 'svc-ai-ollama' role (vars, tasks, compose, meta, README) and dedicated network

• Refactor former AI stack into separate app roles: web-app-flowise and web-app-openwebui

• Add web-app-minio role; adjust config (no central DB), meta (fa-database, run_after), compose networks include, volume key

• Provide user-focused READMEs for Flowise, OpenWebUI, MinIO, Ollama

• Networks: add subnets for web-app-openwebui, web-app-flowise, web-app-minio; rename web-app-ai → svc-ai-ollama

• Ports: rename ai_* keys to web-app-openwebui / web-app-flowise; keep minio_api/minio_console

• Add group_vars/all/17_ai.yml (OLLAMA_BASE_LOCAL_URL, OLLAMA_LOCAL_ENABLED)

• Replace hardcoded include paths with path_join in multiple roles (svc-db-postgres, sys-service, sys-stk-front-proxy, sys-stk-full-stateful, sys-svc-webserver, web-svc-cdn, web-app-keycloak)

• Remove obsolete web-app-ai templates/vars/env; split Flowise into its own role

• Minor config cleanups (CSP flags to {}, central_database=false)

https://chatgpt.com/share/68d15cb8-cf18-800f-b853-78962f751f81
2025-09-22 18:40:20 +02:00
kevinveenbirkenbach b7a7be4737 Fix XWiki automation bootstrap:
- Accept HTTP 302 (Distribution Wizard redirects) in REST readiness and extension checks
- Treat 302 as missing admin user during bootstrap
- Move superadmin password to xwiki.cfg (correct location)
- Disable automatic Distribution Wizard start in xwiki.properties
- Standardize run_once includes for postgres, cdn, and xwiki roles

See: https://chatgpt.com/share/68c3a67b-80b4-800f-8a90-ebdcd4abb86c
2025-09-12 06:50:24 +02:00
kevinveenbirkenbach 022800425d THE HUGE REFACTORING CALENDER WEEK 33; Optimized Matrix and during this updated variables, and implemented better reset and cleanup mode handling, also solved some initial setup bugs 2025-08-15 15:15:48 +02:00
kevinveenbirkenbach 5d36a806ff svc-db-postgres: add retry mechanism to all PostgreSQL tasks and fix condition handling
- Added register, until, retries, and delay to all PostgreSQL-related tasks
  in 02_init.yml to handle transient 'tuple concurrently updated' and similar errors.
- Changed 'when: "{{ postgres_init }}"' to 'when: postgres_init | bool' in main.yml
  for correct boolean evaluation.
- Switched 'role' to 'roles' in postgresql_privs tasks for forward compatibility.
- Added postgres_retry_retries and postgres_retry_delay defaults in vars/main.yml
  to centralize retry configuration.

  https://chatgpt.com/share/689b2360-a8a4-800f-9acb-6d88d6aa5cb7
2025-08-12 13:20:30 +02:00
kevinveenbirkenbach 6e04ac58d2 Moved blocks to include_tasks to raise performance. Deploy was really slow 2025-08-11 12:28:31 +02:00
kevinveenbirkenbach 8e0341c120 Solved some handler reloading bugs 2025-08-08 19:33:16 +02:00
kevinveenbirkenbach 22c8c395f0 Refactored handlers loading 2025-08-08 19:01:12 +02:00
kevinveenbirkenbach aae69ea15b Ensure that keycloak is up 2025-08-08 17:25:31 +02:00
kevinveenbirkenbach c7b25ed093 Normalized run_once_, made openresty handlers without when aviable and forced flush in run_once when blocks to avoid handlers with when conditions 2025-08-08 15:32:26 +02:00
kevinveenbirkenbach 0472fecd64 Solved ooauth2 bugs and restructured postgres roile to implement extensions used by discourse 2025-07-23 13:24:55 +02:00
kevinveenbirkenbach e1d36045da Solved open run_once issues 2025-07-20 14:23:08 +02:00
kevinveenbirkenbach 169493179e Restructuring for new backup solution 2025-07-16 19:09:31 +02:00
kevinveenbirkenbach 8161dd1b6d Optimized postgres 2025-07-14 10:05:22 +02:00
kevinveenbirkenbach fb0ca533ae Moved database port mapping to central port configuration file 2025-07-13 23:06:11 +02:00
kevinveenbirkenbach 6fbe550afe Solved Database bugs 2025-07-13 22:30:41 +02:00
kevinveenbirkenbach 756597668c Semi bsr for applications[] to prevent heavy to debug bugs in j2 - part 1 2025-07-13 15:11:38 +02:00
kevinveenbirkenbach a18e888044 Implemented new matomo setup 2025-07-13 12:58:10 +02:00
kevinveenbirkenbach 3b03c5171d Renamed the mariadb, openldap and postgres database 2025-07-12 16:06:13 +02:00