Commit Graph
40 Commits
Author SHA1 Message Date
kevinveenbirkenbach 92946ee008 refactor(core): introduce SOFTWARE_DOMAIN and unify homepage, tls, service, and oidc defaults
* add SOFTWARE_DOMAIN derived from SOFTWARE_NAME lowercased
* introduce DOMAIN_HOMEPAGE defaulting to dashboard domain if present, otherwise SOFTWARE_DOMAIN
* add automatic primary → homepage redirect mapping in constructor
* migrate TLS and CA paths from SOFTWARE_NAME|lower to SOFTWARE_DOMAIN
* update systemd unit suffix and timer/service discovery globs to SOFTWARE_DOMAIN
* switch OIDC default realm and client id to SOFTWARE_DOMAIN
* adjust SuiteCRM well-known software name to SOFTWARE_DOMAIN
* update unit tests to use SOFTWARE_DOMAIN for strict Jinja expansion

https://chatgpt.com/share/698db094-22bc-800f-8454-10d97f817659
2026-02-12 11:50:55 +01:00
kevinveenbirkenbach 7a3c44b013 refactor(invokable): derive deployment types from invokable roles via DEFAULT_RULES
- Remove alias-based group name handling from get_deployment_types_from_groups
- Make DEFAULT_RULES the single source of truth for server/workstation/universal
- Determine "universal" strictly as invokable roles not matched by server/workstation
- Update unit tests to reflect invokable-based semantics

This aligns Ansible filters and CLI behavior and avoids inventory-specific heuristics.

https://chatgpt.com/share/698b8125-d2a0-800f-9fbd-f3f9c8bc5023
2026-02-10 20:04:00 +01:00
kevinveenbirkenbach 84782f3b22 refactor(invokable): centralize app discovery, add type grouping, and derive deploy stages from inventory groups
https://chatgpt.com/share/698b0039-c4c4-800f-b962-7033324c07a8
2026-02-10 10:53:53 +01:00
kevinveenbirkenbach 88c3bbb721 test: fix templating env lookup fallback and unify unittest.mock import usage
https://chatgpt.com/share/698992ef-215c-800f-a93e-5d8e75b81f23
2026-02-09 08:55:17 +01:00
kevinveenbirkenbachandCopilot Autofix powered by AI 26851bcfe1 Potential fix for code scanning alert no. 139: Module is imported with 'import' and 'import from'
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-02-09 08:00:59 +01:00
kevinveenbirkenbach 4e5e8d2a7d Fix compose down behavior and harden templar rendering
- Restore full teardown semantics in Compose.down by delegating to down_stack
  (including volume removal and CI cleanup)
- Normalize templar render output to string to avoid None-related failures
- Clean up unused variable in templating fallback path
- Reorder test imports without changing behavior

https://chatgpt.com/share/69890935-4f1c-800f-8f2c-1b18a1df0ea5
2026-02-08 23:07:44 +01:00
kevinveenbirkenbach 50fc018f39 Merge branch 'alert-autofix-201' 2026-02-08 22:35:27 +01:00
kevinveenbirkenbach 5ffc0c8390 tests: fix domain_mapper unit tests to use explicit module references
- Call iter_app_domains and build_domain_index via module_utils.domain_mapper
- Avoid NameError caused by missing direct imports in test module
- Improve formatting consistency in assertions for readability

https://chatgpt.com/share/69890141-da40-800f-ab40-1b30ba15ac7c
2026-02-08 22:33:35 +01:00
kevinveenbirkenbach d4a41bc5d8 Merge branch 'alert-autofix-139' 2026-02-08 22:26:06 +01:00
kevinveenbirkenbach 7d8d024868 tests: fix missing mock import in inventory manager tests
Add missing `from unittest import mock` import to prevent NameError
in test_inventory_manager when using mock.patch().

https://chatgpt.com/share/6988ff29-8470-800f-97a5-2c3ad1a224b1
2026-02-08 22:24:47 +01:00
kevinveenbirkenbachandCopilot Autofix powered by AI b5ffea5034 Potential fix for code scanning alert no. 139: Module is imported with 'import' and 'import from'
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-02-08 18:36:22 +01:00
kevinveenbirkenbachandCopilot Autofix powered by AI abe7512b75 Potential fix for code scanning alert no. 194: Module is imported with 'import' and 'import from'
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-02-08 18:35:24 +01:00
kevinveenbirkenbachandCopilot Autofix powered by AI c3ccfff16c Potential fix for code scanning alert no. 201: Module is imported with 'import' and 'import from'
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-02-08 18:32:48 +01:00
kevinveenbirkenbach b78ab29218 Renamed config entry from docker to compose
https://chatgpt.com/share/69870f54-d708-800f-9870-e9ae5a4bff44
2026-02-07 11:09:36 +01:00
kevinveenbirkenbach 8a81630391 Added newline at file and POSIX compatible 2026-02-04 19:49:56 +01:00
kevinveenbirkenbach ea8fbbdd71 fix(tls): resolve domains via app canonicals/aliases
- add module_utils/domain_mapper to map domains (canonical/aliases) -> application_id
  supporting str/list/dict and nested structures
- extend tls_common.resolve_term to optionally resolve alias domains via applications
  while keeping legacy lookup via global domains mapping intact
- pass applications into tls_resolve and cert_plan
- add unit tests for domain_mapper

https://chatgpt.com/share/697d500f-b884-800f-ba5b-088a2340f9d6
2026-01-31 01:42:43 +01:00
kevinveenbirkenbach 2678090934 feat(ca-trust): unify trust injection + strict templating
Move `with-ca-trust.sh` to `sys-ca-selfsigned` as the single source of truth, add `CA_TRUST_NAME`/`trust_name`, and pass it through `compose_ca_inject.py` + `compose_ca_inject_cmd`. Introduce `module_utils/templating.py` for strict multi-pass rendering with fallback env lookup, update TLS self-signed regen logic, purge script, and extend unit tests accordingly.

https://chatgpt.com/share/697c876d-7140-800f-9981-762dfeeafdfe
2026-01-30 11:26:46 +01:00
kevinveenbirkenbach 8bff1b984d refactor(jinja): add strict recursive renderer and use in tls/compose lookups
Introduce module_utils/jinja_strict.py to render nested Jinja-in-vars with hard-fail semantics. Refactor cert_plan, compose_f_args, and compose_ca_inject_cmd to use render_strict so no "{{ ... }}" leaks into generated paths/commands, and add unit tests.

https://chatgpt.com/share/697b8e1b-54fc-800f-b895-7161f00ebd51
2026-01-29 17:42:28 +01:00
kevinveenbirkenbach c3c54ce043 refactor(tls): split resolution and certificate planning, add global self-signed SAN support
- Extract shared TLS logic into module_utils/tls_common
- Introduce cert_plan lookup for certificate paths and SAN computation
- Simplify tls_resolve to strict protocol/port/domain resolution only
- Add global self-signed certificate scope covering all domains
- Mount self-signed cert base into OpenResty container
- Update Nginx templates to use cert_plan for ssl_certificate paths
- Add comprehensive unit tests for tls_common, tls_resolve, and cert_plan

https://chatgpt.com/share/6972c568-9e8c-800f-b89a-a0061966f9a8
2026-01-23 01:48:31 +01:00
kevinveenbirkenbach ba4527a8db ruff 2026-01-19 21:17:56 +01:00
kevinveenbirkenbach f56cfa773e **test(unit): add full unittest coverage for sufficient_storage CLI and role_resource_validation**
https://chatgpt.com/share/696e3d20-276c-800f-87a7-a7052989ffe4
2026-01-19 19:29:08 +01:00
kevinveenbirkenbach c92aff464b feat(ci): add min_storage validation and CI filtering with warnings
- Add sufficient_storage CLI for validating docker.services.<entity>.min_storage
- Introduce role_resource_validation utility with human-friendly size parsing
- Filter CI app discovery by available runner storage (default: 12GB)
- Emit GitHub Actions warnings for:
  - missing min_storage metadata (treated as 0GB)
  - roles exceeding available runner storage
- Add BigBlueButton min_storage metadata (50GB)
- Add unit tests for validation logic and CLI behavior
- Add humanfriendly dependency for size parsing

https://chatgpt.com/share/696e097b-1038-800f-b8fc-23e9dadfa021
2026-01-19 11:37:28 +01:00
kevinveenbirkenbach 66d50e2dde feat(inventory): resolve shared providers transitively and support schema-less roles
- Allow credentials generator to run for roles without schema/main.yml so
  transitive config-based generation still works.
- Make InventoryManager resilient to missing YAML by defaulting to {} and
  only loading schema when schema/main.yml exists.
- Add recursive shared-provider include resolution based on docker.services
  config and apply provider schemas/special rules transitively.
- Require database_password injection only when database.enabled=true and shared=true.
- Update unit/integration tests to create required files for .exists() checks and
  validate transitive provider role credential generation.

https://chatgpt.com/share/69639d93-4c10-800f-bc6c-8e5b9fb75088
2026-01-11 13:54:37 +01:00
kevinveenbirkenbach e4baa6d7ab Ruff 2026-01-11 01:01:44 +01:00
kevinveenbirkenbach 64cab86be3 refactor(inventory): extract secret generation into ValueGenerator and add integration test
- Move generate_value/generate_secure_alphanumeric out of InventoryManager into new ValueGenerator
- Wire InventoryManager to use self.value_generator for all secret generation
- Tighten schema include logic for shared databases: require enabled=true and shared=true
- Update unit tests to patch ValueGenerator.generate_value instead of InventoryManager.generate_value
- Add end-to-end integration test for InventoryManager using real files and a fake VaultHandler
- Remove redundant legacy unit test file and add dedicated ValueGenerator unit tests

https://chatgpt.com/share/6962d2e6-7fdc-800f-ba83-9e150b19a5f0
2026-01-11 00:56:18 +01:00
kevinveenbirkenbach 3ce6eb348a refactor(inventory): switch feature flags to docker service config paths
Replace legacy feature flags (features.central_database, features.oauth2) with
docker service based configuration:

- docker.services.database.shared
- docker.services.oauth2.enabled

Update InventoryManager logic accordingly and adapt all unit tests to mock
the new config structure.

This ensures inventory generation now follows the canonical docker service
configuration model.

https://chatgpt.com/share/69628906-5efc-800f-999b-2ae3ece1907f
2026-01-10 18:14:29 +01:00
kevinveenbirkenbach a38f36837f style: reformat code for readability and lint compliance 2025-12-18 14:08:01 +01:00
kevinveenbirkenbach b512d6c4b6 Solved mock path bugs 2025-12-18 13:03:12 +01:00
kevinveenbirkenbachandCopilot Autofix powered by AI dda9505801 Potential fix for code scanning alert no. 62: Module is imported with 'import' and 'import from'
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-12-18 10:33:41 +01:00
kevinveenbirkenbach 7123841fb4 style: apply ruff autoformat 2025-12-17 16:37:21 +01:00
kevinveenbirkenbach 9e6256dc6e lint(test,core): remove misplaced test code and fix undefined names
- Import subprocess where used to fix F821 in cli/setup/applications
- Remove accidental test methods from RoleDependencyResolver implementation
- Remove invalid and non-discoverable test functions from test_role_dependency_resolver
- Eliminate test code that was never executed due to wrong scope/indentation
- Restore clear separation between production code and unit tests

https://chatgpt.com/share/6942c3ec-8a10-800f-9a15-c319f5e93089
2025-12-17 15:53:25 +01:00
kevinveenbirkenbach c549febe8a Autofix ruff F401 2025-12-17 14:41:42 +01:00
kevinveenbirkenbach 176c8799ce lint(ruff): fix E402 and related import issues
- Fix E402 by removing sys.path hacks and ensuring imports are at module top level
- Replace ad-hoc imports with proper package imports (filter_plugins, lookup_plugins, module_utils, cli)
- Make lookup plugins importable via package paths (e.g. lookup_plugins.application_gid)
- Clean up test imports to rely on installed package layout instead of sys.path manipulation
- Partially fix F821 caused by missing/invalid imports

This establishes a consistent, ruff-compliant import architecture across CLI, filters, plugins, and tests.
2025-12-17 14:36:29 +01:00
kevinveenbirkenbach 14a0268ec3 Solved ruff E741 2025-12-17 13:51:03 +01:00
kevinveenbirkenbach c0e26275f8 Refactor defaults generation, credential creation, and inventory management
### Overview
This commit introduces a broad set of improvements across the defaults
generator, credential creation subsystem, inventory creation workflow,
and InventoryManager core logic.

### Major Changes
- Support empty or  config/main.yml in defaults generator and ensure that
  applications with empty configs are still included in defaults_applications.
- Add '--snippet' and '--allow-empty-plain' modes to create/credentials.py
  with non-destructive merging and correct plain-secret handling.
- Ensure empty strings for 'plain' credentials are never encrypted.
- Update InventoryManager to fully support allow_empty_plain and prevent
  accidental overwriting or encrypting existing VaultScalar or dict values.
- Add full-size implementation of cli/create/inventory.py including
  dynamic inventory building, role filtering, host_vars management, and
  parallelised credential snippet generation.
- Fix schemas (Magento, Nextcloud, OAuth2-Proxy, keyboard-color, etc.) to
  align with the new credential model and avoid test failures.
- Improve get_app_conf consistency by ensuring credentials.* paths are
  always resolvable for applications even when config/main.yml is empty.

### Added Test Coverage
- Unit tests for defaults generator handling empty configs.
- Full test suite for create/inventory.py including merge logic and
  vault-safe host_vars loading.
- Extensive tests for InventoryManager: plain-secret behavior,
  vault handling, and recursion logic.
- Update or remove outdated tests referencing old schema behaviour.

### Context
This commit is associated with a refactoring and debugging session documented here:
https://chatgpt.com/share/692ec0e1-5018-800f-b568-d09a53e9d0ee
2025-12-02 11:54:55 +01:00
kevinveenbirkenbach d76e384ae3 Enhance CertUtils to return the newest matching certificate and add comprehensive unit tests
- Added run_openssl_dates() to extract notBefore/notAfter timestamps.
- Modified mapping logic to store multiple cert entries per SAN with metadata.
- find_cert_for_domain() now selects the newest certificate based on notBefore and mtime.
- Exact SAN matches take precedence over wildcard matches.
- Added new unit tests (test_cert_utils_newest.py) verifying freshness logic, fallback handling, and wildcard behavior.

Reference: https://chatgpt.com/share/68ef4b4c-41d4-800f-9e50-5da4b6be1105
2025-10-15 09:21:00 +02:00
kevinveenbirkenbach ed73a37795 Improve get_app_conf robustness and add skip_missing_app parameter support
- Added new optional parameter 'skip_missing_app' to get_app_conf() in module_utils/config_utils.py to safely return defaults when applications are missing.
- Updated group_vars/all/00_general.yml and roles/web-app-nextcloud/config/main.yml to include skip_missing_app=True in all Nextcloud-related calls.
- Added comprehensive unit tests under tests/unit/module_utils/test_config_utils.py covering missing app handling, schema enforcement, nested lists, and index edge cases.

Ref: https://chatgpt.com/share/68ee6b5c-6db0-800f-bc20-d51470d7b39f
2025-10-14 17:25:37 +02:00
kevinveenbirkenbach bcee1fecdf feat(inventory): add random_hex_32 generator
feat(bbb/schema): auto-generate etherpad_api_key; set fsesl_password to alphanumeric_32
test(unit): add InventoryManager tests (Option B) expecting feature-generated creds as plain strings
docs: full autocreation of credentials for BigBlueButton now enabled
See: https://chatgpt.com/share/68d69ee8-3fd4-800f-9209-60026b338934
2025-09-26 16:11:05 +02:00
kevinveenbirkenbach b867a52471 Refactor and extend role dependency resolution:
- Introduced module_utils/role_dependency_resolver.py with full support for include_role, import_role, meta dependencies, and run_after.
- Refactored cli/build/tree.py to use RoleDependencyResolver (added toggles for include/import/dependencies/run_after).
- Extended filter_plugins/canonical_domains_map.py with optional 'recursive' mode (ignores run_after by design).
- Updated roles/web-app-nextcloud to properly include Collabora dependency.
- Added comprehensive unittests under tests/unit/module_utils for RoleDependencyResolver.

Ref: https://chatgpt.com/share/68a519c8-8e54-800f-83c0-be38546620d9
2025-08-20 02:42:07 +02:00
kevinveenbirkenbach a9f55579a2 Reorgenized test structure and added validation of inventory before deployment 2025-07-04 01:14:00 +02:00