Commit Graph
2780 Commits
Author SHA1 Message Date
kevinveenbirkenbach bdaac5cc90 ci(deploy): split per-app deploy into server/workstation/rest workflows (env-only defaults)
- Remove _deploy-per-app.yml (inputs-based indirection)
- Add dedicated workflows:
  - test-deploy-server.yml
  - test-deploy-workstation.yml
  - test-deploy-universal.yml (rest)
- Configure via env defaults only (KISS / SRP / SPOT)
- Update Makefile to target the new workflows and pass overrides via env
- Ensure deploy-one-app.sh is executable
- Keep failure debug (last 200 chars) and upload logs only outside act
2026-01-13 05:23:47 +01:00
kevinveenbirkenbach 8a85f7e6b1 test(filters): exclude .github directory from filter usage scan
Exclude the .github directory from template/YAML scanning to avoid
false positives from GitHub Actions expressions. Directory exclusion
is now centralized via EXCLUDE_DIRS and respects EXCLUDE_TESTS.

https://chatgpt.com/share/6965b7a8-8084-800f-a8b5-446f265d5bf2
2026-01-13 04:10:25 +01:00
kevinveenbirkenbach 48b0bbb85e test(run_once): allow per-condition opt-out via inline pass marker
Add support for ignoring individual run_once_* when-conditions if the same
line contains the marker comment:

  # pass test_run_once_suffix_matches_role

This allows intentional cross-role run_once usage (e.g. shared provider
roles) without weakening the global schema enforcement.

Also document the exception behavior directly in the test.

https://chatgpt.com/share/69653e36-c058-800f-921d-b4af9014df04

fix(openldap): make memberof/refint provisioning idempotent and robust

Refactor OpenLDAP cn=config provisioning:
- load memberof and refint modules via ldapmodify
- add overlays via ldapadd with proper error handling
- make handlers idempotent (ignore duplicate/value-exists errors)
- dynamically resolve module DN and data backend DN (mdb/hdb)
- clean up legacy LDIF definitions

This fixes rc=80 handler failures and allows safe re-runs.

https://chatgpt.com/share/69653e42-e020-800f-a6c3-9c37ec5e8028
2026-01-12 19:32:14 +01:00
kevinveenbirkenbach c9a3c811c6 feat(config): migrate docker.service flags to docker.services
* Switch feature flag paths from docker.service.<name>.enabled to docker.services.<name>.enabled
* Update CSP builder and smart-default toggles and unit tests
* Update desktop cards iframe flag and desktop domain validation
* Update logout domain filter and tests
* Update recaptcha enable flags in Keycloak and EspoCRM
* Fix sys-ctl-cln-docker meta description formatting
* Follow-up: reorder imports in web_health_expectations so module_utils path injection runs before importing get_app_conf

https://chatgpt.com/share/6963f484-4cb4-800f-a9f4-8d70a66e4d32
2026-01-11 22:27:36 +01:00
kevinveenbirkenbach f584a61f1f refactor(csp, inj): switch to docker.service.*.enabled flags and align unit tests
- Update CSP smart-default toggles to use docker.service.<feature>.enabled
  (matomo, simpleicons, recaptcha, hcaptcha, logout, desktop)
- Refactor inj_enabled filter to resolve docker.service.<feature>.enabled only
- Migrate inj_enabled unit tests to the new flag layout
- Normalize and reformat CSP test fixtures for readability and consistency

https://chatgpt.com/share/6963f484-4cb4-800f-a9f4-8d70a66e4d32
2026-01-11 20:50:30 +01:00
kevinveenbirkenbach 47f777a8b2 test(csp): disable matomo in desktop/logout toggle tests to avoid unrelated domain dependency
- Set docker.service.matomo.enabled = false in TestCspTogglesDesktopLogout fixture
- Prevent accidental dependency on web-app-matomo domain mapping
- Keep desktop/logout toggle tests focused and isolated
2026-01-11 20:29:08 +01:00
kevinveenbirkenbach 158c6569dc test(csp): migrate feature toggles to docker.service.*.enabled and split unit tests
- Switch CSP smart-default toggles to docker.service.<name>.enabled flags
  (matomo, simpleicons, recaptcha, hcaptcha, logout, desktop)
- Update CSP header logic docs/comments to match new flag layout
- Split large CSP filter unit test into focused test modules for clarity
- Add coverage for service toggle behavior, determinism, union rules, and hash guards

https://chatgpt.com/share/6963f484-4cb4-800f-a9f4-8d70a66e4d32
2026-01-11 20:05:33 +01:00
kevinveenbirkenbach 1250e94b93 fix(web-svc-html): handle missing index.html and normalize yaml formatting
- Add default status codes (403, 200) for web-svc-html to reflect missing index.html on init
- Normalize YAML formatting and ensure trailing newlines
- Fix integration test formatting for config generation
- Align vars indentation with project style

https://chatgpt.com/share/6963a552-9580-800f-b6f2-2547959d9dc8
2026-01-11 14:27:21 +01:00
kevinveenbirkenbach 66d50e2dde feat(inventory): resolve shared providers transitively and support schema-less roles
- Allow credentials generator to run for roles without schema/main.yml so
  transitive config-based generation still works.
- Make InventoryManager resilient to missing YAML by defaulting to {} and
  only loading schema when schema/main.yml exists.
- Add recursive shared-provider include resolution based on docker.services
  config and apply provider schemas/special rules transitively.
- Require database_password injection only when database.enabled=true and shared=true.
- Update unit/integration tests to create required files for .exists() checks and
  validate transitive provider role credential generation.

https://chatgpt.com/share/69639d93-4c10-800f-bc6c-8e5b9fb75088
2026-01-11 13:54:37 +01:00
kevinveenbirkenbach 107b9345d6 test(sys-ctl-hlth-csp): suppress ruff E402 for intentional path bootstrap import
The test suite intentionally modifies PYTHONPATH to import the role-local script
module. Suppress ruff E402 for this specific import to keep linting strict
everywhere else while documenting the exception.

https://chatgpt.com/share/69637f6e-5370-800f-806f-d5d0820c005f
2026-01-11 11:45:55 +01:00
kevinveenbirkenbach 0e9506eba6 test(sys-ctl-hlth-csp): fix main() exit handling by asserting SystemExit instead of mocking sys.exit
Update unit tests to:
- Add role files directory to PYTHONPATH for importing script.py
- Assert SystemExit and exit codes instead of mocking sys.exit
- Ensure run_checker is not called when no domains or config dir is missing
- Keep existing behavior and expectations intact

This makes the tests robust and prevents false positives caused by mocked sys.exit continuing execution.

https://chatgpt.com/share/69637cc6-c1d0-800f-a196-df86ccbaee14
2026-01-11 11:34:34 +01:00
kevinveenbirkenbach f601d3d911 feat(sys-ctl-hlth-csp): run CSP checker container in host network by default
Run the CSP checker container with --network host by default so localhost and
internal vhosts are reachable when executed inside Docker-in-Docker setups.

Adds:
- Optional --no-host-network flag to disable host networking
- Extended docker command builder with host network support
- Unit tests for domain extraction, docker command generation, and CLI behavior

This ensures CSP health checks work reliably for services bound to the
parent container's network namespace.

https://chatgpt.com/share/69630726-4514-800f-b350-61ae7a4010b1
2026-01-11 03:12:45 +01:00
kevinveenbirkenbach b694f9335f fix(sys-ctl-cln-bkps): prevent infinite cleanup loop and add regression tests
Stop cleanup script from busy-looping when disk usage is above threshold but no
backup versions exist by breaking early when average_version_directories == 0.

Add unit tests to verify:
- script exits immediately when disk usage is below threshold
- script breaks safely when disk usage is high but no backups are present

This prevents CI hangs and excessive CPU usage caused by infinite iterations.

https://chatgpt.com/share/6962de45-3750-800f-bc8d-5ba205beeaaf
2026-01-11 01:22:29 +01:00
kevinveenbirkenbach e4baa6d7ab Ruff 2026-01-11 01:01:44 +01:00
kevinveenbirkenbach 64cab86be3 refactor(inventory): extract secret generation into ValueGenerator and add integration test
- Move generate_value/generate_secure_alphanumeric out of InventoryManager into new ValueGenerator
- Wire InventoryManager to use self.value_generator for all secret generation
- Tighten schema include logic for shared databases: require enabled=true and shared=true
- Update unit tests to patch ValueGenerator.generate_value instead of InventoryManager.generate_value
- Add end-to-end integration test for InventoryManager using real files and a fake VaultHandler
- Remove redundant legacy unit test file and add dedicated ValueGenerator unit tests

https://chatgpt.com/share/6962d2e6-7fdc-800f-ba83-9e150b19a5f0
2026-01-11 00:56:18 +01:00
kevinveenbirkenbach 3ce6eb348a refactor(inventory): switch feature flags to docker service config paths
Replace legacy feature flags (features.central_database, features.oauth2) with
docker service based configuration:

- docker.services.database.shared
- docker.services.oauth2.enabled

Update InventoryManager logic accordingly and adapt all unit tests to mock
the new config structure.

This ensures inventory generation now follows the canonical docker service
configuration model.

https://chatgpt.com/share/69628906-5efc-800f-999b-2ae3ece1907f
2026-01-10 18:14:29 +01:00
kevinveenbirkenbach 04c9490745 Solved unit and integration tests 2026-01-10 13:07:12 +01:00
kevinveenbirkenbach eec7bb17db feat(docker-compose): add robust compose pull helper with retries and unit tests
- Add compose_pull.py helper with retry/backoff logic and transient network detection
- Replace shell-based docker compose pull handler with script-based implementation
- Add lockfile handling to avoid repeated pulls
- Add comprehensive unit tests for retry logic, locking, and command execution
- Improve logging with explicit command tracing

https://chatgpt.com/share/6960dc8d-f884-800f-b6fe-c23a5edf5ba3
2026-01-09 11:46:30 +01:00
kevinveenbirkenbach 151cfdac31 fix(sys-util-git-pull): add retries/backoff for transient git network failures
Introduce retry logic with exponential backoff for git operations to reduce flaky
CI/Docker failures (e.g., "Connection reset by peer"). Replace run(check=True)
with run_checked() and add transient error detection. Update unit tests to match
new execution and error-handling behavior.

https://chatgpt.com/share/6960d28d-3e2c-800f-943b-a80630a6a395
2026-01-09 11:03:53 +01:00
kevinveenbirkenbach 82e4082a6d sys-ctl-bkp-docker-2-loc: add file_has_content module and conditional backup flush
- Add custom Ansible module file_has_content to validate databases.csv on target host
- Fail hard if databases.csv is missing or not a regular file
- Compute backup force-flush flag based on MODE_BACKUP and actual file content
- Prevent instant backup flush when no database secrets are present
- Add full unittest coverage for file_has_content module
- Make sys-service flush behavior data-driven and deterministic

https://chatgpt.com/share/6960cdb6-dd28-800f-8427-2e375de4e61b
2026-01-09 10:43:13 +01:00
kevinveenbirkenbach d99c7e49a5 fix(ci): reintroduce lifecycle gating in deploy test runner
- Restore lifecycle-based role exclusion via cli.meta.roles.lifecycle_filter
- Make tested lifecycles configurable through TESTED_LIFECYCLES env
- Apply lifecycle gating consistently for workstation and server tests
- Fix minor bug in invokable parsing (avoid shadowing built-in list)
- Improve logging with lifecycle exclusion details

https://chatgpt.com/share/695fbcf4-2148-800f-8685-56ee73a9bae0
2026-01-08 15:19:26 +01:00
kevinveenbirkenbach 851caf9a3e refactor(valid_deploy_id): resolve roles directory from repo root and adapt tests
- Always derive roles/ path from repository root instead of accepting a parameter
- Fail fast if roles directory is missing or mislocated
- Add explicit typing and Path-based path handling
- Update unit tests to rely on real repo roles instead of temporary fixtures
- Ensure tests cover correct role discovery and inventory validation

https://chatgpt.com/share/695fb926-1f5c-800f-9012-e2922bb1df3f
2026-01-08 15:13:14 +01:00
kevinveenbirkenbach 83985163e2 feat(cli): add run_after resolution command with cycle-safe traversal
- Introduce CLI tool to resolve galaxy_info.run_after dependencies transitively
- Detect and report circular run_after dependencies explicitly
- Validate role existence and run_after schema strictly
- Provide whitespace-separated output for easy shell usage
- Add comprehensive unittest coverage including CLI behavior and error cases

https://chatgpt.com/share/695f8e09-54f4-800f-9f5f-ed9c2fdd664b
2026-01-08 11:59:17 +01:00
kevinveenbirkenbach e64aa27825 feat(svc-bkp-rmt-2-loc): fail fast per host, continue across providers
- Make pull-specific-host.py fail on any command error (and rsync after retries), emit clear stderr, and exit 0/1 via main()
- Keep remote `find ... -execdir basename {} ;` unchanged to stay compatible with existing ssh-wrapper.sh
- Update wrapper script to continue across hosts but return exit 1 if any host failed
- Adjust unit tests to expect exceptions instead of SystemExit

https://chatgpt.com/share/695e9615-6014-800f-a77d-5809cf45559d
2026-01-07 18:21:21 +01:00
kevinveenbirkenbach 29483a6415 cli: add lifecycle_filter command and fix help forwarding
- Introduce meta roles lifecycle_filter CLI to select roles by lifecycle
- Support whitelist/blacklist modes, selection filtering, and auto-detected roles dir
- Fix blacklist handling for roles with missing lifecycle
- Forward --help correctly to subcommand argparse
- Add unit tests for lifecycle filtering logic

https://chatgpt.com/share/695d09d2-c864-800f-9e23-b9db0c6f3537
2026-01-06 14:10:33 +01:00
kevinveenbirkenbach 0f7add0f1a Unify Mailu token handling via users[*].tokens and clean up inventory validation
- Migrate Mailu credentials from legacy mailu_token to tokens['web-app-mailu']
- Update all affected application templates to use the new token structure
- Adjust inventory validation to no longer special-case mailu_token
- Minor typing and formatting cleanups in inventory creation and tests

https://chatgpt.com/share/69554915-87c8-800f-afd5-12118dbb1107
2025-12-31 17:02:20 +01:00
kevinveenbirkenbach 93f09ac0a7 feat(token-store): hydrate runtime tokens + unify Matomo token usage
**Body**

* add sys-token-store filter plugin to hydrate users.*.tokens from persisted store (users win if non-empty)
* load token store during sys-token-store core run and inject hydrated tokens into users fact
* remove legacy lookup task and simplify docs + store path naming
* switch Matomo frontend injection to use hydrated users.administrator.tokens[web-app-matomo] directly
* ensure Mailu token handling uses sys-token-store write task and checks token presence via users.*.tokens
* add unittest for token hydration filter plugin with path-based module loading
2025-12-31 13:43:29 +01:00
kevinveenbirkenbach 533a1235fe Implement SPOT-based authorized_keys management for users
* Move SSH key handling to users.<name>.authorized_keys (list-only) and write keys via shared user role
* Enforce at least one SSH key for administrator user
* Wrap backup user keys with restricted command wrapper before writing authorized_keys
* Remove legacy inventory file copy and CLI flag/function for administrator authorized_keys
* Extend user setup generator to include authorized_keys field
* Rename administrator scripts path to root scripts path and update dependent variables
* Update inventory sample and docs to reflect new SSH key workflow

https://chatgpt.com/share/695510b8-23f4-800f-b5db-877b07a1469b
2025-12-31 13:01:45 +01:00
kevinveenbirkenbach e4c41add54 test(integration): allow run-once flag anywhere inside run-once blocks
Relax run-once inclusion validation to accept utils/once/flag.yml
at any position within a run-once block instead of enforcing it
as the final task.

https://chatgpt.com/share/6954eba2-05c0-800f-baad-a7546d434627
2025-12-31 10:23:42 +01:00
kevinveenbirkenbach 244e56fed9 Optimized ruff 2025-12-30 19:33:00 +01:00
kevinveenbirkenbach 1a81ac69f3 Ignored diff ansible_service_mgr in tests -> Official ansible variable 2025-12-30 19:17:59 +01:00
kevinveenbirkenbach ea04035e12 feat(config): rename SSL_ENABLED to TLS_ENABLED across stack
* Replace global `SSL_ENABLED` flag with `TLS_ENABLED`
* Update web protocol/ports (http/https, ws/wss) to use `TLS_ENABLED`
* Switch cert-related roles and nginx TLS header gating to `TLS_ENABLED`
* Update app env templates (BookWyrm, Confluence, Jira, Magento, OpenProject, WordPress, etc.) to use `TLS_ENABLED`
* Adjust inventory generator to write `TLS_ENABLED` and update CLI help text accordingly
* Update unit tests for inventory vars overrides to expect `TLS_ENABLED`

https://chatgpt.com/share/695306ef-a798-800f-b118-e07b525e1811
2025-12-29 23:55:20 +01:00
kevinveenbirkenbach 6166740c25 Included missing import 2025-12-29 21:36:36 +01:00
kevinveenbirkenbach 1e5ebbb05f feat(inventory): add --vars-file support for YAML-based host_vars overrides
- add --vars-file CLI option to load host_vars overrides from YAML
- merge vars-file overrides before JSON --vars (JSON takes precedence)
- implement deep-merge logic for YAML vars files
- update CI deploy script to use vars file instead of inline JSON
- add unit test for apply_vars_overrides_from_file
- remove obsolete MASK_CREDENTIALS_IN_LOGS TODO entry

https://chatgpt.com/share/6952e51d-6c50-800f-8764-31dc8c609cdb
2025-12-29 21:31:05 +01:00
kevinveenbirkenbach bf4af7c823 Renamed domain variables 2025-12-29 20:37:47 +01:00
kevinveenbirkenbach c7851645b4 refactor(backup): move backups under /var/lib/infinito and parameterize pull script
- switch compose volumes from /Backups to /var/lib/infinito
- add PATH_INFINITO_BACKUP_DIR and drop BACKUPS_FOLDER_PATH
- update backup provider permissions + cleanup service to use new path
- make pull-specific-host accept --folder and raise if no remote backups
- adjust unit tests accordingly

https://chatgpt.com/share/69515eed-001c-800f-b1da-aee8d8683e63
2025-12-28 19:15:29 +01:00
kevinveenbirkenbach 5a74f80e85 Executed ruff 2025-12-28 15:23:07 +01:00
kevinveenbirkenbach 160642fe43 Removed unnecessary warnings 2025-12-28 11:06:21 +01:00
kevinveenbirkenbach 426e0c2489 Moved Corefile 2025-12-28 11:03:56 +01:00
kevinveenbirkenbach fc1388a8e4 tests: recognize ansible.builtin.set_fact in run_once validation
The run_once integration test only detected short-form `set_fact`
and failed to recognize fully qualified module usage
(`ansible.builtin.set_fact`).

This caused false negatives for roles correctly using FQCN-compliant
Ansible syntax.

Fix:
- Accept both `set_fact` and `ansible.builtin.set_fact` when collecting
  global run_once_<suffix> definitions
- Accept both forms when detecting per-role run_once definitions
- Remove redundant prefilter token, as structural detection is sufficient

No production logic is affected.

https://chatgpt.com/share/6950fe2c-4b24-800f-9f40-85b2fed3bfba
2025-12-28 10:53:36 +01:00
kevinveenbirkenbach dcd0bed303 refactor(sys-ctl-bkp-docker-2-loc): move databases.csv permission handling to core
- Move databases.csv permission task from seed phase to core tasks
- Ensure root-only permissions are enforced early and unconditionally
- Remove custom run-once logic and related state tracking
- Simplify role logic and improve determinism
- Update run-once integration test documentation and expectations
- Add TODO to verify Docker-in-Docker behavior
2025-12-28 10:41:42 +01:00
kevinveenbirkenbach 1efab4011a ci(docker-compose): add CoreDNS service and CI-specific DNS wiring
- introduce CoreDNS service behind CI profile
- route CI containers via fixed DNS (172.30.0.53)
- add CoreDNS Corefile for docker.test and external forwarding
- make infinito image distro-selectable via INFINITO_DISTRO
- align volumes, env, and networking for CI runner

https://chatgpt.com/share/694d1f08-9794-800f-b1ff-548e9e6c500f
2025-12-25 12:24:51 +01:00
kevinveenbirkenbach 9118b47654 test(integration): allow regex_findall as builtin Jinja2 filter
Add regex_findall to BUILTIN_FILTERS to reflect Jinja2-provided filters
available in Ansible templates and avoid false positives in filter
definition checks.

https://chatgpt.com/share/694929e5-e960-800f-9649-47b21d2c7fc0
2025-12-22 12:22:07 +01:00
kevinveenbirkenbach d9485bcdd1 test(integration): restrict handler discovery to handlers/main.yml
Only handlers defined in handlers/main.yml/main.yaml are considered.
Task files included via include_tasks/import_tasks under handlers/ are
regular tasks and must not be treated as standalone handlers.

This prevents false positives where internal handler sub-steps were
incorrectly reported as missing notify targets.

https://chatgpt.com/share/6949290a-a694-800f-bb29-60cd96321f62
2025-12-22 12:18:32 +01:00
kevinveenbirkenbach d2e88b6a8c feat(cli): eliminate runpy warnings for inventory + container entrypoints
* Remove legacy **main** re-export wrappers to avoid `runpy` "already in sys.modules" warnings
* Introduce `cli.deploy.container.command` as the stable implementation module
* Reduce `cli.deploy.container.__main__` to a minimal entrypoint delegating to `command.main`
* Re-export container helpers from package `__init__` without importing `__main__`
* Update unit tests to import/patch the new command module and keep CLI dispatch stable

https://chatgpt.com/share/69484a22-60dc-800f-a23d-068d7c45fe37
2025-12-21 20:27:26 +01:00
kevinveenbirkenbach 6726f36799 feat(sys-util-git-pull): introduce reusable shallow git pull role with tag pinning
- Add new role sys-util-git-pull as a generic system utility
- Implement robust shallow clone/update logic in Python helper
- Support detached-safe branch updates and optional tag pinning
- Handle annotated vs lightweight tags and heal local tag conflicts
- Provide deterministic Ansible change detection via machine-readable output
- Add verbose logging to stderr for debuggability
- Integrate sys-util-git-pull into pkgmgr role to replace inline git logic
- Add comprehensive unit tests (option A: file-based import) for helper script
- Add role README and meta information with platform-agnostic support
- Ensure role conforms to existing test expectations (meta/main.yml present)

https://chatgpt.com/share/69481902-a98c-800f-9b3f-9ee13e8e50b7
2025-12-21 16:57:45 +01:00
kevinveenbirkenbach 38c8b59fe0 test(integration): ignore **pycache** directories in role validation tests
https://chatgpt.com/share/69481867-784c-800f-ab60-50ce3b241dd4
2025-12-21 16:55:06 +01:00
kevinveenbirkenbach 1fe0a420fa fix(cli): avoid runpy warnings by moving logic into command modules
Stop importing __main__ from package __init__.py to prevent runpy RuntimeWarning when using `python -m`.

- Move CLI logic into command.py modules
- Keep __main__.py as thin wrappers
- Update unit tests to import command modules directly

https://chatgpt.com/share/6947fb8e-550c-800f-8323-e36f92cb5bba
2025-12-21 14:52:09 +01:00
kevinveenbirkenbach fd54faff65 refactor(ansible): replace pacman installs with package
Migrate most roles from community.general.pacman to the generic package module for better portability. Keep pacman where it is required (system upgrades and Arch-specific tasks). Split VirtualBox setup into an Arch-only task file and add a warning when skipped on non-Arch systems. Update integration test examples accordingly.

https://chatgpt.com/share/6947ce9f-3228-800f-8365-2253b5ed9451
2025-12-21 11:40:10 +01:00
kevinveenbirkenbach 71c541ee05 test(cli): use unittest.mock.patch consistently
- Remove direct imports of unittest.mock alias
- Use unittest.mock.patch via unittest for all test decorators
- Keep behavior identical while simplifying imports
2025-12-21 11:10:03 +01:00