Commit Graph
229 Commits
Author SHA1 Message Date
kevinveenbirkenbach 9f169321ed Merge branch 'main' into feature/playwright-e2e-test 2026-03-18 02:18:44 +01:00
kevinveenbirkenbach 19c8701735 Run Playwright E2E without MODE_TEST gate 2026-03-18 00:16:26 +01:00
kevinveenbirkenbach f482e7e63a Replace Cypress E2E role with Playwright 2026-03-17 01:57:57 +01:00
kevinveenbirkenbach 8ed583dde9 fix(ansible): remove deprecated disable_lookups and set loop_var 2026-03-16 11:51:56 +01:00
kevinveenbirkenbach 1fb252fc7d refactor(inventories,roles): migrate util-* to bundle inventories and unify AUR handling
Migrates large parts of the legacy util-* role structure to inventory-driven
bundles and standardizes AUR installation via central system roles.

- Adds `web-svc-legal`, `web-svc-asset`, and `web-app-dashboard` to all server bundles.
- Removes `util-srv-corporate-identity` entirely.
- Introduces new workstation bundle inventories:
  `admin`, `admin-network`, `browser`, `design`, `dev-*`, `game-*`, `office`.
- Uses bundle variables `SYS_PACKAGES` and `SYS_AUR_PACKAGES` as the primary
  installation mechanism.
- Removes/replaces obsolete util-desk/util-dev roles where fully covered.
- Renames dev roles:
  `util-desk-dev-core` -> `dev-core`,
  `util-desk-dev-python` -> `dev-python`,
  `util-desk-dev-arduino` -> `dev-arduino`.
- Adds missing `application_id` vars for renamed roles.
- Removes the entire `util` section from `roles/categories.yml`.
- Replaces direct `kewlfft.aur.aur` calls in desktop/driver roles with
  a centralized wrapper role.
- Introduces new role `sys-aur-install` as AUR installation wrapper.
- Migrates `dev-yay` to `sys-aur` and adds `SYS_AUR_PACKAGES: []` default.
- Extends constructor stage:
  auto-load `sys-package` when `SYS_PACKAGES` is set,
  and `sys-aur` + `sys-aur-install` when `SYS_AUR_PACKAGES` is set.
- Cleans up stage loading:
  removes util-srv from `02_server.yml`,
  reduces workstation loading to `desk-roles.yml` + `drv-non-free`.
- Updates integration/unit tests for the new invokable and include behavior.
- Integration test now accepts static `include_tasks` references.
- Fixes recursion risk between `sys-aur` and `sys-aur-install` by setting
  `run_once_sys_aur` at the beginning of `sys-aur/tasks/01_core.yml`.

BREAKING CHANGE:
- `util-*` categories and bundle paths are no longer part of stage loading.
- Multiple `util-desk-*`, `util-dev-*`, and `util-srv-*` roles were removed.
- Deployment groups must be migrated to the new bundle/role naming.
2026-02-19 14:10:59 +01:00
kevinveenbirkenbach 0aef93f4c9 feat(sys-package): add inventory-driven system package role 2026-02-19 10:16:37 +01:00
kevinveenbirkenbach 37ecc016ce fix(constructor): normalize svc-prx-openresty domain override to list format
* change `svc-prx-openresty` override in `CURRENT_PLAY_DOMAINS` from `{ canonical: [...] }` to `list[str]`
* align with legacy `domains` schema expected by `tls_common.resolve_primary_domain_from_app()`
* prevent *invalid primary domain* error in TLS lookup
* ensure consistent domain structure across applications

https://chatgpt.com/share/698e5ccd-7658-800f-9614-211013316bdf
2026-02-13 00:05:44 +01:00
kevinveenbirkenbach 535ff88189 fix(domains): robust recursive flattening in generate_all_domains and remove json workaround
* replace shallow flatten logic with recursive _add helper to correctly handle nested dict/list structures
* prevent unhashable _AnsibleLazyTemplateList from reaching set(flat)
* remove unnecessary to_json/from_json roundtrip in constructor
* add unittest coverage for mixed str/list/dict domain inputs and deduplication behavior

https://chatgpt.com/share/698e598e-3284-800f-9917-ca1eaa4e6572
2026-02-12 23:51:37 +01:00
kevinveenbirkenbach 640988690a refactor(constructor): normalize domain handling and fully qualify ansible builtins
* replace implicit modules with ansible.builtin.* for consistency
* normalize redirect domain sources and targets to avoid string slicing issues
* wrap domain lookups in list before flatten/first to prevent single-character truncation
* materialize CURRENT_PLAY_DOMAINS before generate_all_domains to eliminate _AnsibleLazyTemplateList errors
* minor whitespace and formatting cleanup for improved readability

https://chatgpt.com/share/698de3f8-2d50-800f-ab99-edcce4f0734a
2026-02-12 18:11:13 +01:00
kevinveenbirkenbach a09479a5b6 fix(constructor): normalize redirect domain sources to avoid unhashable lazy list errors
* ensure source and target domains are flattened and cast to string
* prevent _AnsibleLazyTemplateList from being used as dict keys in items2dict
* stabilize generate_all_domains execution for redirect mappings
* switch to ansible.builtin.set_fact for consistency

https://chatgpt.com/share/698de3f8-2d50-800f-ab99-edcce4f0734a
2026-02-12 17:18:21 +01:00
kevinveenbirkenbach 9f973f4236 fix(constructor): normalize DOMAIN_PRIMARY to flat string list for svc-prx-openresty
* flatten DOMAIN_PRIMARY before merging into CURRENT_PLAY_DOMAINS
* convert lazy template values to plain strings to avoid unhashable type errors
* fix generate_all_domains failure caused by nested _AnsibleLazyTemplateList

https://chatgpt.com/share/698de3f8-2d50-800f-ab99-edcce4f0734a
2026-02-12 15:58:00 +01:00
kevinveenbirkenbach b29dc58f1f fix(webserver,tls): ensure primary domain mapping for svc-prx-openresty in constructor
* explicitly merge DOMAIN_PRIMARY as canonical domain for svc-prx-openresty into CURRENT_PLAY_DOMAINS
* prevents tls lookup failures when rendering default_server.conf.j2
* stabilize domain mapping for webserver + OpenResty bootstrap phase
* normalize ansible.builtin.set_fact usage in constructor stage
* minor whitespace alignment in WEBSERVER_CPUS_EFFECTIVE

https://chatgpt.com/share/698de3f8-2d50-800f-ab99-edcce4f0734a
2026-02-12 15:30:11 +01:00
kevinveenbirkenbach 92946ee008 refactor(core): introduce SOFTWARE_DOMAIN and unify homepage, tls, service, and oidc defaults
* add SOFTWARE_DOMAIN derived from SOFTWARE_NAME lowercased
* introduce DOMAIN_HOMEPAGE defaulting to dashboard domain if present, otherwise SOFTWARE_DOMAIN
* add automatic primary → homepage redirect mapping in constructor
* migrate TLS and CA paths from SOFTWARE_NAME|lower to SOFTWARE_DOMAIN
* update systemd unit suffix and timer/service discovery globs to SOFTWARE_DOMAIN
* switch OIDC default realm and client id to SOFTWARE_DOMAIN
* adjust SuiteCRM well-known software name to SOFTWARE_DOMAIN
* update unit tests to use SOFTWARE_DOMAIN for strict Jinja expansion

https://chatgpt.com/share/698db094-22bc-800f-8454-10d97f817659
2026-02-12 11:50:55 +01:00
kevinveenbirkenbach 94f5d735ad Just show message in debug mode 2026-02-11 12:05:56 +01:00
kevinveenbirkenbach c2449e372f refactor(lifecycle): handle universal roles via constructor and destructor only
Universal roles are now covered exclusively by the constructor and destructor
stages, removing the universal stage execution to prevent duplicate role
invocations and redundant lifecycle handling.

https://chatgpt.com/share/698bcad1-d964-800f-8e23-2ac459da8c34
2026-02-11 01:18:10 +01:00
kevinveenbirkenbach 2d344fb129 refactor(domains): switch to domain() lookup across config and templates
Replace `domains | get_domain(...)` usage with `lookup('domain', ...)` throughout
group_vars, roles, and templates. Fix load_app domain resolution to use the
current application_id.

https://chatgpt.com/share/6985ef1e-11b8-800f-b231-52523055060b
2026-02-06 14:46:29 +01:00
kevinveenbirkenbach b0c29b6ea0 Optimized text 2026-02-06 10:31:04 +01:00
kevinveenbirkenbach 26aeed417b Add sys-version role to persist project version from pyproject.toml into PATH_INFINITO_VERSION
https://chatgpt.com/share/6985b3a4-7634-800f-9404-021b1d1f0935
2026-02-06 10:25:49 +01:00
kevinveenbirkenbach 57df7e6c0b Deleted legacy debug function 2026-02-04 22:42:11 +01:00
kevinveenbirkenbach 8a81630391 Added newline at file and POSIX compatible 2026-02-04 19:49:56 +01:00
kevinveenbirkenbach 1860727ef8 Optimized domain redirects 2026-02-04 16:26:06 +01:00
kevinveenbirkenbach 9f24a0a0fa Optimized debugging 2026-02-04 15:07:24 +01:00
kevinveenbirkenbach 3e8abe3cb4 fix(rdbms): preserve database_consumer_id across role boundaries
- Introduce database_consumer_id_passed to avoid fact/vars collisions
- Explicitly pass consumer context into svc-db-postgres via include_role
- Strengthen context assertion in postgres init tasks
- Move once-flag include to the beginning of postgres core tasks
- Make POSTGRES_EXEC_SHELL non-interactive by injecting PGPASSWORD
- Improve test coverage to detect facts overridden by include-time vars

https://chatgpt.com/share/69833e56-93a0-800f-8e3d-d970cf87daef
2026-02-04 13:40:44 +01:00
kevinveenbirkenbach ea2ca2b7ac Solved test 2026-02-04 11:51:23 +01:00
kevinveenbirkenbach 8517b26444 Activate update of package manager before every deploy 2026-01-31 22:37:29 +01:00
kevinveenbirkenbach 36c30431c0 refactor(systemd): introduce sys-service-terminator and deploy-safe timer lockdown
- Rename sys-service-runner to sys-service-terminator
- Add deploy-wide timer lockdown using SOFTWARE_NAME-based unit matching
- Mask timers and stop services during deploy, unmask and re-enable timers at finalization
- Ensure deterministic end-of-deploy service execution with diagnostics

https://chatgpt.com/share/697e595b-f378-800f-9f2b-5c5f416c2981
2026-01-31 20:34:46 +01:00
kevinveenbirkenbach a354578844 feat(tls): expand Jinja paths inside cert_plan lookup
cert_plan now strictly renders Jinja-containing path variables (e.g. TLS_SELFSIGNED_BASE_PATH / LETSENCRYPT_LIVE_PATH) using hostvars context and fails fast if markers remain. Add unit tests covering successful expansion, hostvars precedence, and hard-fail behavior when variables are missing or Jinja leaks through. Include SOFTWARE_NAME in constructor debug output for easier runtime inspection.

https://chatgpt.com/share/697b73c6-f5f8-800f-96cc-e360a137e7a7
2026-01-29 15:50:32 +01:00
kevinveenbirkenbach b1bf9386e2 Track async enabled 2026-01-23 21:48:23 +01:00
kevinveenbirkenbach 9fdb0cddd0 Replace global WEB_PROTOCOL / WEB_PORT / WEBSOCKET_PROTOCOL logic
with explicit per-application TLS resolution.

Key changes:
- Introduce sys-front-tls-layer as the single TLS abstraction
- Centralize certificate paths via TLS_CERT_FILE / TLS_KEY_FILE / TLS_CA_FILE
- Replace sys-svc-letsencrypt SSL templates with generic TLS layer includes
- Add tls_* Jinja filters (enabled, mode, protocol, port, websocket)
- Resolve TLS mode per application (letsencrypt | self_signed | off)
- Remove implicit defaults and WEB_* globals to fail fast on misconfiguration
- Support self-signed certificates with SAN awareness
- Make TLS behavior explicit and consistent across all roles

This decouples application logic from certificate providers and
allows switching TLS flavor without touching app roles.

https://chatgpt.com/share/69727c61-8e68-800f-a267-473a7e627978
2026-01-22 20:36:49 +01:00
kevinveenbirkenbach 0eb7071a49 fix(local-tests): force python3 for local inventory init and stabilize runtime vars
- Force PYTHON=python3 for test-local-init to avoid missing venv python
- Rename test-local-inventory-init-all target to test-local-init
- Fix JSON quoting for --vars in local deploy scripts
- Add RUNTIME default to group_vars
- Improve debug output for initial host vars
- Update user hint in local run script

https://chatgpt.com/share/697222e1-5edc-800f-b9ab-511d8afc3142
2026-01-22 14:14:50 +01:00
kevinveenbirkenbach 10d422c75d feat(testing): add generic E2E Cypress runner role with auto-discovery
Introduce reusable test-e2e-cypress role that:
- auto-discovers application roles providing tests/cypress/env.j2
- stages per-role Cypress projects into /tmp
- renders .env from env.j2 with application context
- optionally waits for application readiness (HTTP 200/302)
- runs Cypress in a pinned Docker image with stable browser settings
- collects failures and fails the play at the end
- stores per-role reports and artifacts

Designed as a test-agnostic execution framework.
Each application role owns its Cypress project and specs.

https://chatgpt.com/share/696d4ad5-740c-800f-904d-84c6fff83e3c
2026-01-18 22:04:09 +01:00
kevinveenbirkenbach 9b87089bb0 Assert application_id integrity in svc-db-postgres init and harden load_app debug output
Add a defensive assertion in svc-db-postgres init to ensure application_id scope
is not polluted by upstream loaders.

Also normalize load_app debug output formatting and add a no-op hardening step
to make variable intent explicit for future scope-guarding refactors.

https://chatgpt.com/share/696ce9ee-68ac-800f-9ea0-d73c285af9a8
2026-01-18 18:20:04 +01:00
kevinveenbirkenbach 0f3364f18c sys-stk-backend: add debug context and extract service enable flags into vars
- add optional debug output for application context
- extract database and oauth2 enable checks into role vars
- use normalized flags in task conditions
- fix typo in load_app task log message

https://chatgpt.com/share/696cbc91-cef8-800f-acfd-9cef34e28b0b
2026-01-18 11:56:56 +01:00
kevinveenbirkenbach 95e3823d2f refactor: centralize token store init and allow Mailu 301 status checks
- move sys-token-store initialization to constructor stage
- remove duplicate token store includes from frontend and app roles
- allow Mailu reachability check to accept configurable status codes (200, 301)
- make Mailu health check consistent with web-app-mailu config
- increase test-local-rapid verbosity for better diagnostics

https://chatgpt.com/share/696c0f49-7888-800f-8175-fb21893c0d67
2026-01-17 23:37:54 +01:00
kevinveenbirkenbach e9f332f510 fix(load_app): prevent self-include by loading target role instead of caller
https://chatgpt.com/share/6968f421-e260-800f-9227-01ca2d5a88ac
2026-01-15 15:05:15 +01:00
kevinveenbirkenbach 456bc6f5f0 ci: add aggressive cleanup for docker, nix and system caches to prevent disk pressure
Introduce a new sys-cleanup role to perform cross-distro cleanup (apt, pacman,
dnf, yum, language caches, temp dirs and journald) for CI/container environments.

Add aggressive Docker prune handler (system + buildx + builder + log truncate)
to prevent GitHub Actions images from growing uncontrollably.

Integrate Nix garbage collection and store optimisation for CI runs.

Cleanup is guarded by MODE_CLEANUP and DOCKER_IN_CONTAINER to ensure it never
runs on real production hosts.

https://chatgpt.com/share/6967ad99-8e50-800f-b4e9-4cd8e45c29bc
2026-01-14 16:28:20 +01:00
kevinveenbirkenbach 517ccf44c0 ci: replace rest deploy mode with universal and add universal stage
- Rename CI workflow and MODE from "rest" to "universal"
- Extend CLI and test runner to support universal deploy type
- Add tasks/stages/02_universal.yml for universal host type
- Update discover logic to use universal app selection
- Remove legacy rest->server mapping and obsolete select helper

https://chatgpt.com/share/6967625e-81c8-800f-8592-5d5b8edec1e2
2026-01-14 10:31:06 +01:00
kevinveenbirkenbach 423603b3bc debug(constructor): print group_names when MODE_DEBUG is enabled
Add conditional debug output in constructor stage to display all Ansible
group memberships of the current host when MODE_DEBUG is set. This helps
with inventory, persona, and stack debugging.

https://chatgpt.com/share/696567d5-9808-800f-b3fc-75080cb7d422
2026-01-12 22:29:47 +01:00
kevinveenbirkenbach 9905ed619a refactor(user): migrate shared user handling to strict user_key SPOT and introduce workstation role
- Refactor shared `user` role to use `user_key` exclusively (remove user_name)
- Resolve system username, home, group, and ownership via users[user_key]
- Update SSH authorized_keys handling to read from users[user_key]
- Adjust documentation to clearly describe user_key-based SPOT model
- Migrate all calling roles (administrator, backup, root) to pass user_key
- Introduce new `user-workstation` role with dedicated README, vars, tasks, and metadata
- Add workstation stage integration to ensure workstation user setup runs explicitly
- Ensure consistent variable naming (lowercase internals, uppercase constants)
- Improve robustness and clarity of user management across all roles

https://chatgpt.com/share/695ea93e-3fa0-800f-b848-ec2ced2a8291
2026-01-07 20:27:28 +01:00
kevinveenbirkenbach 8d3e8d0b73 feat(email): unify SYSTEM_EMAIL_* vars and improve msmtp/postfix local delivery
- Replace DEFAULT_SYSTEM_EMAIL merge with flat SYSTEM_EMAIL_* variables
- Update templates across apps to new variable names
- Fix msmtp health check to use -t and local username fallback
- Improve postfix localhost config (localdomain, mydestination, timeouts)
- Extend sys-hostname to ensure /etc/hosts mapping once

https://chatgpt.com/share/695e8f39-0694-800f-b91c-a74f4b78d7c8
2026-01-07 17:52:04 +01:00
kevinveenbirkenbach 5175aefaf8 Renamed desktop to workstation 2025-12-29 22:44:20 +01:00
kevinveenbirkenbach bf4af7c823 Renamed domain variables 2025-12-29 20:37:47 +01:00
kevinveenbirkenbach d734a15ad6 feat(ci,webserver): add localhost DNS + default vhost 404 assertion
- switch CI DNS to CoreDNS-based *.localhost resolution (A/AAAA → loopback)
- set PRIMARY_DOMAIN to localhost for CI deploys
- add explicit DNS resolution assert for test.localhost
- deploy explicit default_server vhost returning 404 text/plain
- assert default vhost behavior via HTTP before app domains exist

https://chatgpt.com/share/6952cf2a-bbd4-800f-b2b8-c4baba66e301
2025-12-29 19:57:41 +01:00
kevinveenbirkenbach 45ce64ed43 Solved systemctl runner bug 2025-12-29 11:05:34 +01:00
kevinveenbirkenbach 04073cbf5c feat(sys-service): add deferred service execution via sys-service-runner
Introduce `system_service_run_final` to defer service execution until the end
of the play. Services marked with this flag are collected during setup and
executed later by the new `sys-service-runner` role.

Key changes:
- Add `system_service_run_final` flag to sys-service
- Automatically suppress flush for deferred services
- Collect deferred services into a final run list
- Introduce sys-service-runner role with per-unit rescue diagnostics
- Migrate health check services to deferred execution

https://chatgpt.com/share/6951921d-0c20-800f-8d42-1ff01a1f08ef
2025-12-28 21:24:54 +01:00
kevinveenbirkenbach b207134472 feat(ci): enable systemd-in-docker and use explicit container detection
- Install systemd and dbus in Docker image for CI service tests
- Run systemd as PID 1 with proper container conventions
- Mark container environment explicitly via ENV container=docker
- Simplify container detection to rely on explicit env flag
- Skip host-only tasks (timezone, systemctl handlers) inside containers
- Adjust docker-compose runner for privileged systemd execution
- Reduce CI parallelism to avoid OOM kills

This makes systemd-based Ansible roles reliable in Docker/CI
and avoids fragile virtualization heuristics.

https://chatgpt.com/share/6950110b-111c-800f-bbcd-83a551d5a3b8
2025-12-27 18:14:28 +01:00
kevinveenbirkenbach 5c67c42911 Restructured Handler Logic 2025-12-04 02:37:12 +01:00
kevinveenbirkenbach 8fad77f17f Solved CDN and Logout deployment bugs existing due to 2.18->2.20 update 2025-12-03 22:45:48 +01:00
kevinveenbirkenbach 031775c096 Removed redundancies and simplified run_once flag 2025-12-03 18:12:09 +01:00
kevinveenbirkenbach a31ef07718 Refactor CI workflows and fix timezone handling in containers
- Renamed test-cli.yml to test-code.yml and updated job name.
- Extended timeout for test-deploy workflow from 30 to 240 minutes.
- Skipped Ansible timezone configuration inside Docker/Podman/containerd to avoid write errors in CI.
- Added --skip-tests to the initial deploy step for improved CI stability.

Origin: https://chatgpt.com/share/69301c58-6628-800f-9e3a-f026c01b6e17
2025-12-03 12:17:59 +01:00