Commit Graph
185 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.7 34d6121293 refactor(utils): move config_utils to utils.applications.config, rename resolve_app_config -> get
Consolidates application-config helpers under utils/applications/ with a
concise public API (config.get). Updates all plugin, role, and test
imports; removes the legacy get_app_conf filter and its tests.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-19 18:32:30 +02:00
kevinveenbirkenbach 262e1b43fc Merge branch 'main' into fix/users-applications.yml
# Conflicts:
#	docs/contributing/design/services.md
#	plugins/lookup/applications_current_play.py
#	plugins/lookup/service.py
#	roles/sys-stk-backend/lookup_plugins/service_should_load.py
#	roles/sys-stk-backend/tasks/_load_service.yml
#	roles/sys-stk-backend/vars/main.yml
#	roles/sys-svc-compose/templates/base.yml.j2
#	roles/sys-svc-mail/tasks/01_core.yml
#	roles/sys-utils-service-loader/tasks/load_frontend_service.yml
#	tests/unit/plugins/lookup/test_applications_current_play.py
#	tests/unit/plugins/lookup/test_service.py
#	tests/unit/roles/sys-stk-backend/lookup_plugins/test_service_should_load.py
2026-04-15 23:59:54 +02:00
kevinveenbirkenbachandClaude Opus 4.6 b70a783423 refactor(lookups): migrate to runtime applications/users lookups
Replaces inline `applications`/`users` merging with dedicated lookup
plugins and runtime hydration. Introduces `add_csp_hash` filter,
integrates sort into `find_dock_val_by_bkp_entr`, splits the Keycloak
one-shot realm import into `05a_realm_import_run.yml`, and fixes the
Mastodon administrator task YAML. Removes obsolete `csp_hashes` filter,
`sys-token-store` Jinja filter, and empty CLI setup modules.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 23:44:19 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 be6a310d59 refactor(services): implement role-local service registry (req-001)
- Delete group_vars/all/20_services.yml; all service metadata now lives
  in each role's config/main.yml under compose.services.<name>
- Rename generic 'database' service key to explicit 'mariadb'/'postgres'
  across all app roles; remove legacy type: field
- Add utils/service_registry.py and plugins/lookup/service_registry.py
  as the single source of truth for service discovery
- Add utils/database_service.py to centralize DB service resolution
- Delete service_load_enabled filter and service_should_load lookup;
  replaced by service lookup plugin
- Update sys-utils-service-loader to run in constructor stage and use
  generic load_service.yml; remove load_frontend_service.yml
- Add _service_loader_preloading guard to sys-svc-compose,
  sys-svc-webserver-core and sys-ctl-alm-compose to prevent
  health-checks/alarms from firing during preload phase
- Replace manual load_app.yml calls for shared DB/Mailu with assert-
  based preload validation in sys-svc-rdbms and sys-svc-mail
- Update networks.yml.j2 templates to resolve LDAP/AI networks via
  service_registry instead of hardcoded role names
- Migrate all tests to new service discovery API; delete obsolete tests
- Update docs/contributing/code/services.md to reflect role-local model

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 19:33:20 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 acd3b716eb feat(config): set dashboard shared for all enabled dashboard roles
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 19:47:05 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 563b0c3c8c feat(cdn): add css/javascript as frontend services pointing to web-svc-cdn
Ensures web-svc-cdn is loaded whenever any app has css or javascript
injection enabled, by registering css and javascript as service keys in
20_services.yml (both mapping to web-svc-cdn). Adds shared: true to all
css/javascript compose service entries across app and svc role configs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 15:47:07 +02:00
kevinveenbirkenbach 3d493192e6 Add min_storage metadata for invokable services 2026-03-25 15:43:52 +01:00
kevinveenbirkenbach 189267496d Rename module_utils package to utils 2026-03-21 02:05:39 +01:00
kevinveenbirkenbach 6efde55647 refactor(dashboard): rename web-app-desktop to web-app-dashboard and update service toggle
* replace compose.services.desktop with compose.services.dashboard across all roles
* rename role web-app-desktop to web-app-dashboard
* rename sys-front-inj-desktop to sys-front-inj-dashboard
* update resolver mapping to include web-app-dashboard when dashboard is enabled
* adjust CSP filters to reference dashboard for frame-ancestors handling
* update lookup plugins, filter plugins, templates, vars and run_once flags
* migrate all tests to dashboard naming and service key
* update role dependencies and port mappings
* remove legacy desktop role and variables

BREAKING CHANGE:
The service key compose.services.desktop has been renamed to compose.services.dashboard.
All custom configs, inventories, or overrides must be updated accordingly.

https://chatgpt.com/share/698da7ad-1978-800f-82b4-4755b17be026
2026-02-12 11:12:49 +01:00
kevinveenbirkenbach 7cf13b34f4 Removed unnecessary link 2026-02-10 18:42:20 +01:00
kevinveenbirkenbach 3a86443218 refactor(web-svc-logout): switch to registry image and config-driven service definition
Remove git-based build of universal-logout and deploy the service via a
prebuilt container image configured through compose service metadata.
Align logout service with entity_name-based lookup and resource limits.

Also tighten OnlyOffice status code checks to require HTTP 200 responses.

https://chatgpt.com/share/698b4699-ff08-800f-b88f-26d41910c3a9
2026-02-10 15:54:07 +01:00
kevinveenbirkenbach ce64265cca Optimized docu in preparation for deploy script 2026-02-07 23:17:17 +01:00
kevinveenbirkenbach a55c99ef62 refactor(compose): replace docker lookup with container and unify compose paths
- Rename lookup plugin from `docker` to `container`
- Switch all lookups from `lookup('docker', ...)` to `lookup('container', ...)`
- Normalize compose file keys:
  - docker_compose → compose
  - docker_compose_override → compose_override
  - docker_compose_ca_override → compose_ca_override
- Centralize compose path resolution via docker_paths_utils
- Update compose_file_args to always derive paths from DIR_COMPOSITIONS
- Align all roles, templates, and vars with container/compose naming
- Update and rename unit tests accordingly

This prepares the codebase for engine-agnostic container backends
(e.g. Docker ↔ Podman) and removes legacy docker-specific assumptions.

https://chatgpt.com/share/6987a7cc-d374-800f-99ad-bb4c8f92c59e
2026-02-07 21:59:39 +01:00
kevinveenbirkenbach b78ab29218 Renamed config entry from docker to compose
https://chatgpt.com/share/69870f54-d708-800f-9870-e9ae5a4bff44
2026-02-07 11:09:36 +01:00
kevinveenbirkenbach 53e817fa22 refactor(compose): migrate from docker-compose.yml to compose.yml everywhere
- Rename all docker-compose.yml templates to compose.yml
- Update override and CA override filenames accordingly
- Adjust compose wrapper, lookup plugins, filters and path helpers
- Update roles, tasks, handlers and scripts to use compose CLI
- Rewrite documentation and admin guides to reference compose
- Update unit and integration tests to expect compose.yml
- Keep internal variable keys for backward compatibility where needed

https://chatgpt.com/share/69870272-b168-800f-84c3-6417398e441f
2026-02-07 10:14:21 +01:00
kevinveenbirkenbach 3fa7887dc8 Added whitespaces for better readability 2026-02-06 16:21:10 +01:00
kevinveenbirkenbach 5100c2c1ce refactor(config): replace get_app_conf filter with config lookup
Replace usages of the get_app_conf filter with the config lookup plugin
across roles and templates. Adjust quoting where required to keep YAML
and Jinja syntax valid. No functional changes intended.

https://chatgpt.com/share/69860678-cef8-800f-8402-7fe6231fdc2d
2026-02-06 16:19:14 +01:00
kevinveenbirkenbach 3bb8c0ce30 refactor(tls): rename tls_resolve lookup to tls
The TLS lookup has been renamed from `tls_resolve` to `tls` to better
reflect its responsibility as a strict TLS state resolver.

This refactor updates all usages across the codebase, including:
- lookup plugins and internal helpers
- roles, vars, tasks, and templates
- tests, stubs, and error messages
- documentation and changelog references

Behavior, lookup API, and semantics remain unchanged.

https://chatgpt.com/share/6985e635-5a94-800f-b794-937e566ee308
2026-02-06 14:01:32 +01:00
kevinveenbirkenbach d16f006c50 refactor: replace legacy want= kwarg with positional want-path in lookups
Migrate all lookup() usages from the deprecated want= keyword argument to the
new strict positional want-path API. Update call order where required
(e.g. nginx_paths) and ensure consistency across vars, tasks, and templates.

https://chatgpt.com/share/6985d642-b40c-800f-9cdf-ed54fc484a92
2026-02-06 13:37:09 +01:00
kevinveenbirkenbach 8fe6b92e63 refactor(lookups): migrate nginx_paths, tls_resolve and cert_plan to strict positional want-path API
- Replace legacy want= keyword usage with mandatory positional want-path
- Align all nginx_paths, tls_resolve and cert_plan lookups to strict API
- Keep behavior unchanged while enforcing consistent lookup semantics

https://chatgpt.com/share/6985d642-b40c-800f-9cdf-ed54fc484a92
2026-02-06 13:06:09 +01:00
kevinveenbirkenbach 7bb15d1b5e refactor(container): rename docker-container/docker services to sys-svc-container and update all references, schedules, tests, and templates
https://chatgpt.com/share/6985b09d-8ff8-800f-800c-7a694c0983e2
2026-02-06 10:12:31 +01:00
kevinveenbirkenbach 51239968fd refactor: rename docker-compose roles to sys-svc-compose and update paths
Renamed docker-compose and docker-compose-ca roles to sys-svc-compose and sys-svc-compose-ca, updated template includes, handlers, lookups, tests, and variables, and moved compose instances to /opt/compose.

https://chatgpt.com/share/6985ab2c-8344-800f-9e9a-768f562d138d
2026-02-06 09:49:42 +01:00
kevinveenbirkenbach 26a5b159e0 refactor(docker-lookup): replace want= syntax with path-based access
- Replace lookup('docker', ..., want='...') with lookup('docker', ..., 'path')
- Allow both top-level keys (directories/files) and dotted paths
- Simplify docker lookup API and reduce lazy dict propagation
- Update unit tests to reflect new lookup semantics

https://chatgpt.com/share/6984ccea-8290-800f-a6d7-ec0e94df95a0
2026-02-05 18:45:43 +01:00
kevinveenbirkenbach 56aea31667 refactor(docker): replace docker_compose vars with docker lookup across roles
Remove implicit docker_compose variable usage and resolve all docker paths
(files, directories, env) via lookup('docker', application_id).
This prevents scope leakage, centralizes path resolution, and makes docker
context deterministic across roles and handlers.

https://chatgpt.com/share/6984ccea-8290-800f-a6d7-ec0e94df95a0
2026-02-05 18:15:10 +01:00
kevinveenbirkenbach 8a81630391 Added newline at file and POSIX compatible 2026-02-04 19:49:56 +01:00
kevinveenbirkenbach b2545fb1a9 Optimized container healthcheck facts and variable passing 2026-02-01 11:00:49 +01:00
kevinveenbirkenbach 2ea72854b9 refactor(docker-compose): replace git branch with explicit git version/ref
Rename docker_git_repository_branch to docker_git_repository_version
to correctly reflect git refs (branch, tag, commit) and pin
universal-logout to v1.1.0 for reproducible deployments.

https://chatgpt.com/share/697a2741-bc3c-800f-9713-1b43d353e962
2026-01-28 16:11:45 +01:00
kevinveenbirkenbach 8b94174d07 fix(cors): remove trailing slash from Access-Control-Allow-Origin for logout proxy
https://chatgpt.com/share/6978b236-3924-800f-b76c-450702240996
2026-01-27 13:39:31 +01:00
kevinveenbirkenbach 0b9f71463c refactor(tls): replace legacy tls filters with strict tls_resolve lookup
- Remove deprecated filter_plugins/tls.py and all dependent filters
- Introduce strict lookup plugin lookup_plugins/tls_resolve.py as single TLS authority
- Migrate all roles, templates, vars, and tasks to use tls_resolve (protocols, ports, URLs, cert paths)
- Enforce explicit TLS context (domains, applications, TLS_ENABLED, TLS_MODE)
- Support letsencrypt and self_signed with SAN handling and per-app overrides
- Update TLS provider roles to assert resolver state instead of setting facts
- Add comprehensive unit tests for tls_resolve lookup

https://chatgpt.com/share/6972b49b-b074-800f-a323-dd9fe69d02eb
2026-01-23 00:36:46 +01:00
kevinveenbirkenbach fb5abced91 refactor(tls): modularize TLS into orchestrator and providers 2026-01-22 21:38:31 +01:00
kevinveenbirkenbach 9fdb0cddd0 Replace global WEB_PROTOCOL / WEB_PORT / WEBSOCKET_PROTOCOL logic
with explicit per-application TLS resolution.

Key changes:
- Introduce sys-front-tls-layer as the single TLS abstraction
- Centralize certificate paths via TLS_CERT_FILE / TLS_KEY_FILE / TLS_CA_FILE
- Replace sys-svc-letsencrypt SSL templates with generic TLS layer includes
- Add tls_* Jinja filters (enabled, mode, protocol, port, websocket)
- Resolve TLS mode per application (letsencrypt | self_signed | off)
- Remove implicit defaults and WEB_* globals to fail fast on misconfiguration
- Support self-signed certificates with SAN awareness
- Make TLS behavior explicit and consistent across all roles

This decouples application logic from certificate providers and
allows switching TLS flavor without touching app roles.

https://chatgpt.com/share/69727c61-8e68-800f-a267-473a7e627978
2026-01-22 20:36:49 +01:00
kevinveenbirkenbach f52f49118a Disable 'proxy_hide_header Strict-Transport-Security;' when TLS disabled 2026-01-22 16:26:37 +01:00
kevinveenbirkenbach c104867735 Changed License to Infinito.Nexus Community License (Non-Commercial) 2026-01-17 18:26:05 +01:00
kevinveenbirkenbach 6d54901be0 Rename LOGOUT_ORIGINS to LOGOUT_URLS and propagate full scheme-based URLs
into the container environment and CSP connect-src configuration.

This clarifies the semantic difference between host-only LOGOUT_DOMAINS
and fully qualified logout targets used by the universal logout service.

https://chatgpt.com/share/696b82a2-e89c-800f-bbd2-ff7e347c9a85
2026-01-17 14:09:51 +01:00
kevinveenbirkenbach 989c0c0fa2 refactor(stacks): merge stateful/stateless backend+full stacks into sys-stk-backend/sys-stk-full
- Replace sys-stk-back-stateful + sys-stk-back-stateless with unified sys-stk-backend role
- Replace sys-stk-full-stateful + sys-stk-full-stateless with unified sys-stk-full role
- Add reusable task helpers: _load_service.yml (optional deps) and 01_database.yml (DB bootstrap)
- Switch optional dependency loading (OpenLDAP, Ollama) to service_load_enabled (enabled && shared) + run_once guard
- Remove legacy OLLAMA_LOCAL_ENABLED flag; rely on docker.services.ollama.enabled/shared config
- Update all affected app/service roles and web-app template to use new stack role names

https://chatgpt.com/share/69685fb2-cb1c-800f-a9e8-2f5f765a1a53
2026-01-15 04:31:54 +01:00
kevinveenbirkenbach 054810050d refactor(features): migrate feature flags to docker.services and update AI and LDAP network wiring
* Move top-level features flags to docker.services.<feature>.enabled
* Replace features.local_ai with docker.services.ollama.shared for central AI networking
* Switch central LDAP network gating from docker.services.ldap.enabled to docker.services.ldap.shared
* Add per-app service feature entries like css, javascript, hcaptcha, recaptcha under docker.services
* Update Gitea to use derived GITEA_LDAP_ENABLED and adjust IAM conditions

https://chatgpt.com/share/69641f2f-5d6c-800f-8927-fdb3580e7964
2026-01-11 23:07:25 +01:00
kevinveenbirkenbach c9a3c811c6 feat(config): migrate docker.service flags to docker.services
* Switch feature flag paths from docker.service.<name>.enabled to docker.services.<name>.enabled
* Update CSP builder and smart-default toggles and unit tests
* Update desktop cards iframe flag and desktop domain validation
* Update logout domain filter and tests
* Update recaptcha enable flags in Keycloak and EspoCRM
* Fix sys-ctl-cln-docker meta description formatting
* Follow-up: reorder imports in web_health_expectations so module_utils path injection runs before importing get_app_conf

https://chatgpt.com/share/6963f484-4cb4-800f-a9f4-8d70a66e4d32
2026-01-11 22:27:36 +01:00
kevinveenbirkenbach 4f0e45159d Optimized database_type loading 2026-01-10 16:51:18 +01:00
kevinveenbirkenbach 0208f9f29c Optimized OAuth2 2026-01-10 11:57:57 +01:00
kevinveenbirkenbach e2bf636ba9 Changed features structure 2026-01-10 11:13:29 +01:00
kevinveenbirkenbach dd729fd35d Introduced lifecycle into meta/main.yml roles 2026-01-06 14:42:52 +01:00
kevinveenbirkenbach 4b23d9fda9 Reset database to null for logout 2026-01-05 18:32:39 +01:00
kevinveenbirkenbach 4dcffa8601 Changed 127.0.0.1 to DOCKER_BIND_HOST 2026-01-02 04:26:41 +01:00
kevinveenbirkenbach bf4af7c823 Renamed domain variables 2025-12-29 20:37:47 +01:00
kevinveenbirkenbach 7123841fb4 style: apply ruff autoformat 2025-12-17 16:37:21 +01:00
kevinveenbirkenbach c549febe8a Autofix ruff F401 2025-12-17 14:41:42 +01:00
kevinveenbirkenbach 176c8799ce lint(ruff): fix E402 and related import issues
- Fix E402 by removing sys.path hacks and ensuring imports are at module top level
- Replace ad-hoc imports with proper package imports (filter_plugins, lookup_plugins, module_utils, cli)
- Make lookup plugins importable via package paths (e.g. lookup_plugins.application_gid)
- Clean up test imports to rely on installed package layout instead of sys.path manipulation
- Partially fix F821 caused by missing/invalid imports

This establishes a consistent, ruff-compliant import architecture across CLI, filters, plugins, and tests.
2025-12-17 14:36:29 +01:00
kevinveenbirkenbach 9db66f6f98 Ruff autofix 2025-12-17 12:56:47 +01:00
kevinveenbirkenbach ecfdac6764 Fix CORS handling by adding 'always' to Access-Control-Allow-Origin
Ensures the header is included for all HTTP status codes, preventing CORS failures during logout requests.
Details: https://chatgpt.com/share/69328659-8d80-800f-9dbc-2bcf15923a1f
2025-12-05 08:14:51 +01:00
kevinveenbirkenbach 986f959696 Refactor webserver proxy variables and fix BigBlueButton deployment behavior
Refactor proxy/webserver configuration variables to a consistent webserver_* naming scheme across roles. Replace legacy variables like proxy_extra_configuration, client_max_body_size, vhost_flavour, location_ws and ws_port with webserver_extra_configuration, webserver_client_max_body_size, webserver_vhost_flavour, webserver_websocket_location and webserver_websocket_port. Update NGINX vhost and location templates (html, upload, ws, basic, ws_generic) as well as callers (sys-front-inj-all, sys-stk-front-proxy, various web-app-* and web-svc-* roles) to use the new naming.

Tighten docker-compose Git repository handling by making docker_git_repository_pull depend on docker_git_repository_address being defined, a string and non-empty. This avoids accidental Git operations when the repository address is unset or of the wrong type.

Refactor the BigBlueButton role structure and fix deployment bugs: introduce 01_core.yml to orchestrate docker/proxy setup, database seeding, websocket map deployment, docker-compose overrides and admin/bootstrap logic in a single once-executed entrypoint. Rename supporting task files (02_docker-compose.yml, 03_administrator.yml, 04_dependencies.yml) and update tasks/main.yml to delegate via include_tasks with run_once_web_app_bigbluebutton. Improve Greenlight admin creation behavior by treating the 'Email has already been taken' error as a non-fatal, unchanged outcome and running user:set_admin_role as a fallback, both for the primary password and the OIDC starred-password path.

Also standardize vhost flavour selection for services like Mailu, Discourse, CDN, Collabora, Coturn, OnlyOffice, Simpleicons and web-svc-logout by explicitly passing webserver_vhost_flavour where needed and aligning client_max_body_size and websocket configuration with the new webserver_* variables.

Reference: ChatGPT conversation https://chatgpt.com/share/6931c530-bba8-800f-9997-dd61dc1d497b
2025-12-04 18:31:09 +01:00