Commit Graph
92 Commits
Author SHA1 Message Date
kevinveenbirkenbach b2914b7cd1 Merge branch 'main' into fix/users-applications.yml 2026-04-23 18:14:17 +02:00
Alejandro Roman b61359c85c feat(prometheus-intergration): Add all the logic need it for all apps with prometheus, missin playwright still 2026-04-23 00:31:11 +04:00
kevinveenbirkenbachandClaude Opus 4.7 1abc797644 refactor(config): nest service-scoped keys under compose.services
Moves gitea `title`/`configuration` and web-svc-asset `source_directory`/`url`
under their compose.services.<svc> block so config layout mirrors the service
they belong to. Updates the five call sites that dereferenced the old
top-level paths.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-21 13:42:53 +02:00
kevinveenbirkenbach 9b5de4e477 refactor(runtime): replace DictRenderer and domains fact with cached lookups
Removes the placeholder interpreter (utils/dict_renderer.py) in favor of
standard {{ lookup('config', ...) }} / {{ lookup('domain', ...) }} calls
inside role configs, and eliminates the `domains` set_fact stage in
tasks/stages/01_constructor.yml by introducing a cached get_merged_domains
helper plus a new `domains` lookup plugin (full map + by-app subtree).

The `domain` lookup no longer depends on a pre-computed `domains` variable;
it delegates to get_merged_domains, which respects inventory-level overrides
from both `applications:` (authoritative canonical source) and `domains:`
(direct override map). Caching mirrors the applications pipeline — keyed on
(roles_dir, variables signature, override fingerprint) with a re-entry guard.

Replaced `{{ domains[app_id] }}` / `{{ domains }}` usages across web-app-matrix,
-bluesky, -mastodon, -peertube, -keycloak, test-e2e-playwright, and
group_vars. Added two performance tests for lookup('domains') mirroring the
applications cache assertions (≥50x warm-to-hot ratio, 120s cold budget).
2026-04-21 13:29:30 +02:00
kevinveenbirkenbachandClaude Opus 4.7 8ce9179706 fix(lookups,web-app-peertube): finish users/applications migration and harden OIDC plugin install
- Resolve merge conflict in web-app-peertube/vars/main.yml; migrate
  remaining `applications |` and `users.*` references to
  `lookup('applications')` / `lookup('users', ...)` in web-app-odoo and
  web-app-decidim (compose, env, playwright, setup tasks).
- web-app-peertube: split OIDC tasks into oidc/{enable,disable}.yml,
  add install_mem_limit filter + PEERTUBE_MEM_LIMIT /
  PEERTUBE_OIDC_PLUGIN_INSTALL_MEM_LIMIT vars to raise the cgroup ceiling
  during the one-off `npm run plugin:install` exec (prevents OOM-SIGKILL
  that was surfacing as "Container is restarting"); add
  upsert-oidc-plugin.sql.j2 and diagnostics.sh.
- CI: test-deploy-{local,server,universal,workstation}.yml tweaks plus
  scripts/github/{dump_runner_state,enlarge_swap}.sh for constrained
  runners.
- cli/meta/applications/ressources: new CLI entry with unit tests.
- Misc config/docs updates across dashboard, mailu, mattermost, asset,
  cdn, file roles and contributing docs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-21 10:14:21 +02:00
kevinveenbirkenbachandClaude Opus 4.7 1c85dec2b5 Add realistic resource caps to shared and mailu services
Fills in mem_reservation/mem_limit/pids_limit/cpus on service entries
that previously had none, so the role resource aggregator reports a
realistic per-role footprint. Sets light caps on static/SPA providers
(asset, cdn, file, dashboard) and flips mailu sub-services (admin,
imap, smtp, antispam, antivirus, oletools, webdav, fetchmail, webmail,
resolver, front) to enabled: true so the existing values are picked up
during the shared-service walk.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-20 02:17:52 +02:00
kevinveenbirkenbach 262e1b43fc Merge branch 'main' into fix/users-applications.yml
# Conflicts:
#	docs/contributing/design/services.md
#	plugins/lookup/applications_current_play.py
#	plugins/lookup/service.py
#	roles/sys-stk-backend/lookup_plugins/service_should_load.py
#	roles/sys-stk-backend/tasks/_load_service.yml
#	roles/sys-stk-backend/vars/main.yml
#	roles/sys-svc-compose/templates/base.yml.j2
#	roles/sys-svc-mail/tasks/01_core.yml
#	roles/sys-utils-service-loader/tasks/load_frontend_service.yml
#	tests/unit/plugins/lookup/test_applications_current_play.py
#	tests/unit/plugins/lookup/test_service.py
#	tests/unit/roles/sys-stk-backend/lookup_plugins/test_service_should_load.py
2026-04-15 23:59:54 +02:00
kevinveenbirkenbachandClaude Opus 4.6 b70a783423 refactor(lookups): migrate to runtime applications/users lookups
Replaces inline `applications`/`users` merging with dedicated lookup
plugins and runtime hydration. Introduces `add_csp_hash` filter,
integrates sort into `find_dock_val_by_bkp_entr`, splits the Keycloak
one-shot realm import into `05a_realm_import_run.yml`, and fixes the
Mastodon administrator task YAML. Removes obsolete `csp_hashes` filter,
`sys-token-store` Jinja filter, and empty CLI setup modules.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 23:44:19 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 be6a310d59 refactor(services): implement role-local service registry (req-001)
- Delete group_vars/all/20_services.yml; all service metadata now lives
  in each role's config/main.yml under compose.services.<name>
- Rename generic 'database' service key to explicit 'mariadb'/'postgres'
  across all app roles; remove legacy type: field
- Add utils/service_registry.py and plugins/lookup/service_registry.py
  as the single source of truth for service discovery
- Add utils/database_service.py to centralize DB service resolution
- Delete service_load_enabled filter and service_should_load lookup;
  replaced by service lookup plugin
- Update sys-utils-service-loader to run in constructor stage and use
  generic load_service.yml; remove load_frontend_service.yml
- Add _service_loader_preloading guard to sys-svc-compose,
  sys-svc-webserver-core and sys-ctl-alm-compose to prevent
  health-checks/alarms from firing during preload phase
- Replace manual load_app.yml calls for shared DB/Mailu with assert-
  based preload validation in sys-svc-rdbms and sys-svc-mail
- Update networks.yml.j2 templates to resolve LDAP/AI networks via
  service_registry instead of hardcoded role names
- Migrate all tests to new service discovery API; delete obsolete tests
- Update docs/contributing/code/services.md to reflect role-local model

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 19:33:20 +02:00
kevinveenbirkenbach 5db1ff8e7f fix(services): resolve transitive shared asset dependencies 2026-04-01 23:01:09 +02:00
kevinveenbirkenbach 49fad676d6 Solved assets bug 2026-04-01 06:40:47 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 f03a2f4613 fix: move run_once flag to first task in all service 01_core.yml files
Ensures the run_once guard is set before any other task executes,
preventing infinite recursion when load_app.yml re-evaluates service
roles mid-run. Also introduces new 01_core.yml + main.yml for
web-svc-asset and web-svc-libretranslate following the standard schema,
and adds a lint test enforcing this structure across all SERVICE_REGISTRY
roles.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-01 02:42:51 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 3bdbd1831e feat(asset): register web-svc-asset as frontend service and wire dependencies
Adds 'asset' service key in 20_services.yml pointing to web-svc-asset so
it is loaded via 01_services.yml when needed. Sets asset: enabled: false,
shared: true in web-svc-asset/config to avoid self-injection. Enables
asset: enabled+shared in dashboard, discourse, and listmonk — the three
roles that consume the asset server for logo/image URLs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 18:25:31 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 563b0c3c8c feat(cdn): add css/javascript as frontend services pointing to web-svc-cdn
Ensures web-svc-cdn is loaded whenever any app has css or javascript
injection enabled, by registering css and javascript as service keys in
20_services.yml (both mapping to web-svc-cdn). Adds shared: true to all
css/javascript compose service entries across app and svc role configs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 15:47:07 +02:00
kevinveenbirkenbach b95a231a37 refactor(webserver): remove explicit fs owner/group handling 2026-02-21 21:41:14 +01:00
kevinveenbirkenbach b78ab29218 Renamed config entry from docker to compose
https://chatgpt.com/share/69870f54-d708-800f-9870-e9ae5a4bff44
2026-02-07 11:09:36 +01:00
kevinveenbirkenbach 3fa7887dc8 Added whitespaces for better readability 2026-02-06 16:21:10 +01:00
kevinveenbirkenbach 5100c2c1ce refactor(config): replace get_app_conf filter with config lookup
Replace usages of the get_app_conf filter with the config lookup plugin
across roles and templates. Adjust quoting where required to keep YAML
and Jinja syntax valid. No functional changes intended.

https://chatgpt.com/share/69860678-cef8-800f-8402-7fe6231fdc2d
2026-02-06 16:19:14 +01:00
kevinveenbirkenbach 7cb881183b refactor: remove implicit True flag from get_app_conf calls
The get_app_conf() helper no longer requires an explicit third
parameter to enable application-specific lookups. All templates,
tasks, and vars have been updated to rely on the new default behavior
by removing the redundant `True` argument.

This simplifies usage, improves readability, and enforces a consistent
API across Jinja2 templates and Ansible logic.

https://chatgpt.com/share/6985f5f6-c084-800f-887a-4a70d62e6dcf
2026-02-06 15:08:47 +01:00
kevinveenbirkenbach 48237d758c Rename nginx_paths lookup to nginx and update all references
* Rename lookup plugin: lookup_plugins/nginx_paths.py -> lookup_plugins/nginx.py
* Update lookup usage from lookup('nginx_paths', ...) to lookup('nginx', ...) across roles/templates/vars
* Update unit tests accordingly (imports + patch targets)

https://chatgpt.com/share/6985e705-4040-800f-a779-347f06e46f82
2026-02-06 14:05:04 +01:00
kevinveenbirkenbach 8fe6b92e63 refactor(lookups): migrate nginx_paths, tls_resolve and cert_plan to strict positional want-path API
- Replace legacy want= keyword usage with mandatory positional want-path
- Align all nginx_paths, tls_resolve and cert_plan lookups to strict API
- Keep behavior unchanged while enforcing consistent lookup semantics

https://chatgpt.com/share/6985d642-b40c-800f-9cdf-ed54fc484a92
2026-02-06 13:06:09 +01:00
kevinveenbirkenbach 82e2cde747 refactor(nginx): replace NGINX vars with nginx_paths lookup and add http/https support
- Drop legacy NGINX path definitions from group_vars
- Add nginx_paths lookup plugin (paths + ensure dirs + http_includes)
- Update roles/templates to use nginx_paths for config/data/cache/user paths
- Add per-domain HTTP->HTTPS redirect + ACME challenge include
- Add unit tests for nginx_paths lookup
- Clean up obsolete sys-svc-letsencrypt role files

https://chatgpt.com/share/697d4466-af54-800f-a243-b28367638c85
2026-01-31 00:52:42 +01:00
kevinveenbirkenbach 9fdb0cddd0 Replace global WEB_PROTOCOL / WEB_PORT / WEBSOCKET_PROTOCOL logic
with explicit per-application TLS resolution.

Key changes:
- Introduce sys-front-tls-layer as the single TLS abstraction
- Centralize certificate paths via TLS_CERT_FILE / TLS_KEY_FILE / TLS_CA_FILE
- Replace sys-svc-letsencrypt SSL templates with generic TLS layer includes
- Add tls_* Jinja filters (enabled, mode, protocol, port, websocket)
- Resolve TLS mode per application (letsencrypt | self_signed | off)
- Remove implicit defaults and WEB_* globals to fail fast on misconfiguration
- Support self-signed certificates with SAN awareness
- Make TLS behavior explicit and consistent across all roles

This decouples application logic from certificate providers and
allows switching TLS flavor without touching app roles.

https://chatgpt.com/share/69727c61-8e68-800f-a267-473a7e627978
2026-01-22 20:36:49 +01:00
kevinveenbirkenbach c104867735 Changed License to Infinito.Nexus Community License (Non-Commercial) 2026-01-17 18:26:05 +01:00
kevinveenbirkenbach 054810050d refactor(features): migrate feature flags to docker.services and update AI and LDAP network wiring
* Move top-level features flags to docker.services.<feature>.enabled
* Replace features.local_ai with docker.services.ollama.shared for central AI networking
* Switch central LDAP network gating from docker.services.ldap.enabled to docker.services.ldap.shared
* Add per-app service feature entries like css, javascript, hcaptcha, recaptcha under docker.services
* Update Gitea to use derived GITEA_LDAP_ENABLED and adjust IAM conditions

https://chatgpt.com/share/69641f2f-5d6c-800f-8927-fdb3580e7964
2026-01-11 23:07:25 +01:00
kevinveenbirkenbach e2bf636ba9 Changed features structure 2026-01-10 11:13:29 +01:00
kevinveenbirkenbach dd729fd35d Introduced lifecycle into meta/main.yml roles 2026-01-06 14:42:52 +01:00
kevinveenbirkenbach bf4af7c823 Renamed domain variables 2025-12-29 20:37:47 +01:00
kevinveenbirkenbach 05ff3d3d61 Added matomo to roles and optimized features configuration 2025-11-27 14:46:04 +01:00
kevinveenbirkenbach 5e616d3962 web: general domain cleanup (canonical/aliases normalization)
- Normalize domain blocks across apps:
  - Add explicit 'aliases: []' everywhere (no implicit aliases)
  - Standardize canonical subdomains for consistency:
    * Bluesky: web/api under *.bluesky.<PRIMARY_DOMAIN>
    * EspoCRM: espo.crm.<PRIMARY_DOMAIN>
    * Gitea:   tea.git.<PRIMARY_DOMAIN>
    * GitLab:  lab.git.<PRIMARY_DOMAIN>
    * Joomla:  joomla.cms.<PRIMARY_DOMAIN>
    * Magento: magento.shop.<PRIMARY_DOMAIN>
    * OpenProject: open.project.<PRIMARY_DOMAIN>
    * Pretix:  ticket.shop.<PRIMARY_DOMAIN>
    * Taiga:   kanban.project.<PRIMARY_DOMAIN>
  - Remove legacy/duplicate aliases and use empty list instead
  - Fix 'alias' -> 'aliases' where applicable

Context: preparing for AUTO_BUILD_ALIASES=False and deterministic redirect mapping.

Ref: conversation https://chatgpt.com/share/68cd512c-c878-800f-bdf2-81737adf7e0e
2025-09-19 14:51:56 +02:00
kevinveenbirkenbach 022800425d THE HUGE REFACTORING CALENDER WEEK 33; Optimized Matrix and during this updated variables, and implemented better reset and cleanup mode handling, also solved some initial setup bugs 2025-08-15 15:15:48 +02:00
kevinveenbirkenbach 0228014d34 Replaced .infinito.service and .infinito.timer by SOFTWARE_NAME suffix, optimized LICENSE link and update OIDC Realm and ID conf 2025-08-14 14:39:18 +02:00
kevinveenbirkenbach db0e030900 Renamed general and mode constants and implemented a check to verify that constants are just defined ones over the whole repository 2025-08-13 19:11:14 +02:00
kevinveenbirkenbach f31565e4c5 Optimized URLS 2025-08-13 00:33:47 +02:00
kevinveenbirkenbach 7f53cc3a12 Replaced web_protocol by WEB_PROTOCOL 2025-08-07 12:31:20 +02:00
kevinveenbirkenbach 9228d51e86 Restructured server config 2025-08-07 11:31:06 +02:00
kevinveenbirkenbach 44e0fea0b2 Renamed cymais to infinito and did some other optimations and logout implementations 2025-07-29 16:35:42 +02:00
kevinveenbirkenbach bba663f95d Added missing canonicals 2025-07-22 19:20:29 +02:00
kevinveenbirkenbach 756597668c Semi bsr for applications[] to prevent heavy to debug bugs in j2 - part 1 2025-07-13 15:11:38 +02:00
kevinveenbirkenbach f07557c322 Implemented dict renderer to resolve assets 2025-07-12 23:25:31 +02:00
kevinveenbirkenbach e174523fc6 Solved collection dependency bug 2025-07-12 13:42:45 +02:00
kevinveenbirkenbach 5a3535187a Restructured service und web role naming in inventor 2025-07-10 14:01:12 +02:00