Commit Graph
36 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Sonnet 4.6 9d863f1ef4 feat(inventory): add SERVICES_DISABLED env var support and environment test flow
- Add services_disabler.py: parse SERVICES_DISABLED, disable services in host_vars,
  remove provider roles from inventory
- Wire apply_services_disabled_from_env into cli/create/inventory/cli.py after deploy
- Add unit tests for all services_disabler functions
- Update hardware.md and variables.md with SERVICES_DISABLED documentation
- Update environment.sh: deploy dashboard with SERVICES_DISABLED=matomo first,
  assert dashboard reachable + matomo not reachable, then deploy matomo and assert
  reachable; extract assert_http_status / assert_http_status_not helpers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 15:00:24 +02:00
kevinveenbirkenbach 2f0edbb2fc Fix oauth2 proxy cookie secret generation 2026-03-26 01:15:08 +01:00
kevinveenbirkenbach 189267496d Rename module_utils package to utils 2026-03-21 02:05:39 +01:00
kevinveenbirkenbach b78ab29218 Renamed config entry from docker to compose
https://chatgpt.com/share/69870f54-d708-800f-9870-e9ae5a4bff44
2026-02-07 11:09:36 +01:00
kevinveenbirkenbach 8a81630391 Added newline at file and POSIX compatible 2026-02-04 19:49:56 +01:00
kevinveenbirkenbach 9a84fafec5 feat(inventory): add mirror_policy to control mirror image overrides
- Default policy (if_missing): manual image/version win, mirror fills blanks only
- Add mirror_policy=force to always overwrite image/version
- Add mirror_policy=skip to ignore mirror completely
- Treat empty/whitespace values as missing
- Preserve non-image/version service keys
- Extend unit tests to cover all mirror policy scenarios

https://chatgpt.com/share/6981036a-7e40-800f-b43e-13c66cc55fac
2026-02-02 21:04:53 +01:00
kevinveenbirkenbach a70a033864 feat(inventory): add --mirror support to override docker images from mirrors file
- add --mirror flag to inventory creator
- resolve mirrors.yml automatically or via explicit path
- apply mirror overrides to applications.*.docker.services.* (image/version only)
- keep existing host_vars keys untouched
- add unit tests for mirror override behavior

https://chatgpt.com/share/6980cdb1-1e5c-800f-b25f-a1af757e4239
2026-02-02 17:15:30 +01:00
kevinveenbirkenbach 4a5736615f Solved falsy file paths 2026-01-29 17:43:22 +01:00
kevinveenbirkenbach 66d50e2dde feat(inventory): resolve shared providers transitively and support schema-less roles
- Allow credentials generator to run for roles without schema/main.yml so
  transitive config-based generation still works.
- Make InventoryManager resilient to missing YAML by defaulting to {} and
  only loading schema when schema/main.yml exists.
- Add recursive shared-provider include resolution based on docker.services
  config and apply provider schemas/special rules transitively.
- Require database_password injection only when database.enabled=true and shared=true.
- Update unit/integration tests to create required files for .exists() checks and
  validate transitive provider role credential generation.

https://chatgpt.com/share/69639d93-4c10-800f-bc6c-8e5b9fb75088
2026-01-11 13:54:37 +01:00
kevinveenbirkenbach 0f7add0f1a Unify Mailu token handling via users[*].tokens and clean up inventory validation
- Migrate Mailu credentials from legacy mailu_token to tokens['web-app-mailu']
- Update all affected application templates to use the new token structure
- Adjust inventory validation to no longer special-case mailu_token
- Minor typing and formatting cleanups in inventory creation and tests

https://chatgpt.com/share/69554915-87c8-800f-afd5-12118dbb1107
2025-12-31 17:02:20 +01:00
kevinveenbirkenbach 533a1235fe Implement SPOT-based authorized_keys management for users
* Move SSH key handling to users.<name>.authorized_keys (list-only) and write keys via shared user role
* Enforce at least one SSH key for administrator user
* Wrap backup user keys with restricted command wrapper before writing authorized_keys
* Remove legacy inventory file copy and CLI flag/function for administrator authorized_keys
* Extend user setup generator to include authorized_keys field
* Rename administrator scripts path to root scripts path and update dependent variables
* Update inventory sample and docs to reflect new SSH key workflow

https://chatgpt.com/share/695510b8-23f4-800f-b5db-877b07a1469b
2025-12-31 13:01:45 +01:00
kevinveenbirkenbach ea04035e12 feat(config): rename SSL_ENABLED to TLS_ENABLED across stack
* Replace global `SSL_ENABLED` flag with `TLS_ENABLED`
* Update web protocol/ports (http/https, ws/wss) to use `TLS_ENABLED`
* Switch cert-related roles and nginx TLS header gating to `TLS_ENABLED`
* Update app env templates (BookWyrm, Confluence, Jira, Magento, OpenProject, WordPress, etc.) to use `TLS_ENABLED`
* Adjust inventory generator to write `TLS_ENABLED` and update CLI help text accordingly
* Update unit tests for inventory vars overrides to expect `TLS_ENABLED`

https://chatgpt.com/share/695306ef-a798-800f-b118-e07b525e1811
2025-12-29 23:55:20 +01:00
kevinveenbirkenbach 6166740c25 Included missing import 2025-12-29 21:36:36 +01:00
kevinveenbirkenbach 1e5ebbb05f feat(inventory): add --vars-file support for YAML-based host_vars overrides
- add --vars-file CLI option to load host_vars overrides from YAML
- merge vars-file overrides before JSON --vars (JSON takes precedence)
- implement deep-merge logic for YAML vars files
- update CI deploy script to use vars file instead of inline JSON
- add unit test for apply_vars_overrides_from_file
- remove obsolete MASK_CREDENTIALS_IN_LOGS TODO entry

https://chatgpt.com/share/6952e51d-6c50-800f-8764-31dc8c609cdb
2025-12-29 21:31:05 +01:00
kevinveenbirkenbach bf4af7c823 Renamed domain variables 2025-12-29 20:37:47 +01:00
kevinveenbirkenbach a354992f27 refactor(create-inventory): modularize CLI, stabilize path handling, add unit tests
- Split create-inventory logic into focused modules (cli, project, filters, host_vars, credentials, etc.)
- Replace fragile path construction with python -m module execution
- Add robust project root detection
- Centralize password generation
- Improve test coverage with per-module unit tests
- Preserve existing inventory and host_vars behavior

https://chatgpt.com/share/69471157-79c0-800f-8dec-25c0f8e38f03
2025-12-20 22:12:44 +01:00
kevinveenbirkenbach a38f36837f style: reformat code for readability and lint compliance 2025-12-18 14:08:01 +01:00
kevinveenbirkenbachandCopilot Autofix powered by AI 1188bd51f3 Potential fix for code scanning alert no. 60: Module is imported with 'import' and 'import from'
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-12-18 13:59:09 +01:00
kevinveenbirkenbachandCopilot Autofix powered by AI 1d381f9fe4 Potential fix for code scanning alert no. 80: File is not always closed
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-12-18 12:45:12 +01:00
kevinveenbirkenbachandCopilot Autofix powered by AI 9a6387a0c2 Potential fix for code scanning alert no. 81: File is not always closed
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-12-18 10:28:04 +01:00
kevinveenbirkenbach 7123841fb4 style: apply ruff autoformat 2025-12-17 16:37:21 +01:00
kevinveenbirkenbach c549febe8a Autofix ruff F401 2025-12-17 14:41:42 +01:00
kevinveenbirkenbach 176c8799ce lint(ruff): fix E402 and related import issues
- Fix E402 by removing sys.path hacks and ensuring imports are at module top level
- Replace ad-hoc imports with proper package imports (filter_plugins, lookup_plugins, module_utils, cli)
- Make lookup plugins importable via package paths (e.g. lookup_plugins.application_gid)
- Clean up test imports to rely on installed package layout instead of sys.path manipulation
- Partially fix F821 caused by missing/invalid imports

This establishes a consistent, ruff-compliant import architecture across CLI, filters, plugins, and tests.
2025-12-17 14:36:29 +01:00
kevinveenbirkenbach 9db66f6f98 Ruff autofix 2025-12-17 12:56:47 +01:00
kevinveenbirkenbach 0d18d86243 Add --vars support to inventory creation, implement deep JSON overrides for host_vars, and update CI workflow to pass MASK_CREDENTIALS_IN_LOGS=false.
Includes:
- New apply_vars_overrides() with deep merge logic
- New --vars CLI argument in cli/create/inventory.py
- Added unit tests for vars handling in test_inventory.py
- Updated test-deploy workflow to pass --vars in all deploy phases

Ref: ChatGPT conversation https://chatgpt.com/share/69320f49-6c00-800f-8875-49d36935ae3a
2025-12-04 23:48:43 +01:00
kevinveenbirkenbach bf0134b9c5 Extend inventory creation with --authorized-keys support, unify PATH_ADMINISTRATOR_HOME resolution via group_vars, update CI workflow to pass dummy SSH key, and add full unit test coverage for new features.
See details: https://chatgpt.com/share/6931f775-12d4-800f-9ccb-e6ce52097f9c
2025-12-04 22:06:10 +01:00
kevinveenbirkenbach b914fb9789 Refactor CLI filters: rename --ignore to --exclude and update all related logic and tests
- Updated CLI argument parsing to use --exclude instead of --ignore.
- Adjusted help texts, comments, and error messages accordingly.
- Updated role filtering logic and references (include → exclude).
- Added new unit tests for parse_roles_list(), filter_inventory_by_include(), and filter_inventory_by_ignore().
- Improved wording and consistency in docstrings.

This change is part of the refactoring required for the Ansible 2.18 → 2.20 upgrade, ensuring naming clarity and avoiding confusion with Python's 'ignore' semantics.

Conversation reference: https://chatgpt.com/share/69307ef2-1fb4-800f-a2ec-d56020019269
2025-12-03 19:18:39 +01:00
kevinveenbirkenbach d12f7a10a6 Add optional --become-password support with automatic Vault encryption
- Implement ensure_become_password() to handle explicit, generated, and existing become passwords
- Integrate VaultHandler for encrypted ansible_become_password storage
- Add CLI parameter --become-password to inventory creation workflow
- Ensure backwards compatibility: existing passwords remain untouched unless explicitly overridden
- Add unit test verifying non-overwrite behaviour when no password is provided
- Part of migration and refactoring for Ansible 2.20 upgrade

Reference: https://chatgpt.com/share/69301a6d-e920-800f-b19c-e5ca7c3bdd24
2025-12-03 12:09:47 +01:00
kevinveenbirkenbach f06460db16 Fix: Automatically set ansible_connection=local for localhost in inventory generator
Adds automatic local Ansible connection settings and new unit test for localhost.
Details: https://chatgpt.com/share/692f7649-d35c-800f-948f-19d45cb93512
2025-12-03 00:29:29 +01:00
kevinveenbirkenbach 5b18f39ccd Extend inventory create CLI with include/ignore filters and SSL/network defaults - https://chatgpt.com/share/692edecd-54c4-800f-b01b-35cf395a60f0 2025-12-02 13:44:16 +01:00
kevinveenbirkenbach c0e26275f8 Refactor defaults generation, credential creation, and inventory management
### Overview
This commit introduces a broad set of improvements across the defaults
generator, credential creation subsystem, inventory creation workflow,
and InventoryManager core logic.

### Major Changes
- Support empty or  config/main.yml in defaults generator and ensure that
  applications with empty configs are still included in defaults_applications.
- Add '--snippet' and '--allow-empty-plain' modes to create/credentials.py
  with non-destructive merging and correct plain-secret handling.
- Ensure empty strings for 'plain' credentials are never encrypted.
- Update InventoryManager to fully support allow_empty_plain and prevent
  accidental overwriting or encrypting existing VaultScalar or dict values.
- Add full-size implementation of cli/create/inventory.py including
  dynamic inventory building, role filtering, host_vars management, and
  parallelised credential snippet generation.
- Fix schemas (Magento, Nextcloud, OAuth2-Proxy, keyboard-color, etc.) to
  align with the new credential model and avoid test failures.
- Improve get_app_conf consistency by ensuring credentials.* paths are
  always resolvable for applications even when config/main.yml is empty.

### Added Test Coverage
- Unit tests for defaults generator handling empty configs.
- Full test suite for create/inventory.py including merge logic and
  vault-safe host_vars loading.
- Extensive tests for InventoryManager: plain-secret behavior,
  vault handling, and recursion logic.
- Update or remove outdated tests referencing old schema behaviour.

### Context
This commit is associated with a refactoring and debugging session documented here:
https://chatgpt.com/share/692ec0e1-5018-800f-b568-d09a53e9d0ee
2025-12-02 11:54:55 +01:00
kevinveenbirkenbach 751615b1a4 Changed 09_ports.yml to 10_ports.yml 2025-09-03 17:41:14 +02:00
kevinveenbirkenbach 3a839cfe37 Refactor systemctl services and categories due to alarm bugs
This commit restructures systemctl service definitions and category mappings.

Motivation: Alarm-related bugs revealed inconsistencies in service and role handling.

Preparation step: lays the groundwork for fixing the alarm issues by aligning categories, roles, and service templates.
2025-08-18 13:35:43 +02:00
kevinveenbirkenbach 562603a8cd Restructured libraries 2025-07-17 16:38:20 +02:00
kevinveenbirkenbach dea2669de2 Solved unclosed file <_io.TextIOWrapper warnings 2025-07-16 14:33:10 +02:00
kevinveenbirkenbach c160c58a5c Restructured CLI logic 2025-07-10 21:26:44 +02:00