Commit Graph
121 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Sonnet 4.6 9d863f1ef4 feat(inventory): add SERVICES_DISABLED env var support and environment test flow
- Add services_disabler.py: parse SERVICES_DISABLED, disable services in host_vars,
  remove provider roles from inventory
- Wire apply_services_disabled_from_env into cli/create/inventory/cli.py after deploy
- Add unit tests for all services_disabler functions
- Update hardware.md and variables.md with SERVICES_DISABLED documentation
- Update environment.sh: deploy dashboard with SERVICES_DISABLED=matomo first,
  assert dashboard reachable + matomo not reachable, then deploy matomo and assert
  reachable; extract assert_http_status / assert_http_status_not helpers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 15:00:24 +02:00
kevinveenbirkenbach 2b0ed2eacf Merge branch 'feature/pixelfed-playwright' into feature/taiga-ldap 2026-03-28 09:22:30 +01:00
kevinveenbirkenbach 67fadb57b8 refactor: normalize inventory filename to devices 2026-03-28 05:40:13 +01:00
kevinveenbirkenbach 2f0edbb2fc Fix oauth2 proxy cookie secret generation 2026-03-26 01:15:08 +01:00
kevinveenbirkenbach fe764735c3 Stabilize webserver and tree unit tests 2026-03-21 11:56:18 +01:00
kevinveenbirkenbach d8218babfc Fix DNS hosts fallback and domains CLI 2026-03-21 09:12:24 +01:00
kevinveenbirkenbach 189267496d Rename module_utils package to utils 2026-03-21 02:05:39 +01:00
kevinveenbirkenbach 44ea2df452 Fix dev workflow image build and docker repo bootstrap 2026-03-19 19:11:51 +01:00
kevinveenbirkenbach ad3f159548 Require Docker Compose in dev packaging 2026-03-19 11:25:49 +01:00
kevinveenbirkenbach ec30948546 fix(deploy): use bash -lc for commands with pipefail 2026-02-21 16:05:42 +01:00
kevinveenbirkenbach 0710c41f91 fix(ci): run mirror resolver with venv python in container 2026-02-20 11:25:46 +01:00
kevinveenbirkenbach 6efde55647 refactor(dashboard): rename web-app-desktop to web-app-dashboard and update service toggle
* replace compose.services.desktop with compose.services.dashboard across all roles
* rename role web-app-desktop to web-app-dashboard
* rename sys-front-inj-desktop to sys-front-inj-dashboard
* update resolver mapping to include web-app-dashboard when dashboard is enabled
* adjust CSP filters to reference dashboard for frame-ancestors handling
* update lookup plugins, filter plugins, templates, vars and run_once flags
* migrate all tests to dashboard naming and service key
* update role dependencies and port mappings
* remove legacy desktop role and variables

BREAKING CHANGE:
The service key compose.services.desktop has been renamed to compose.services.dashboard.
All custom configs, inventories, or overrides must be updated accordingly.

https://chatgpt.com/share/698da7ad-1978-800f-82b4-4755b17be026
2026-02-12 11:12:49 +01:00
kevinveenbirkenbach efe2ea3357 **test: align deployment type tests with invokable-based semantics**
Update unit tests to reflect the new invokable-driven classification logic by mocking invokable path discovery, removing legacy alias expectations, and adding coverage for universal and mixed cases. Also relocate CLI deploy tests into the unit test hierarchy for consistency.
2026-02-10 21:38:02 +01:00
kevinveenbirkenbach 5acce545e1 refactor(cli): remove sound, alarm and server-type flags
Drop the sound/alarm subsystem and deprecated CLI flags.

* Remove --sound, --no-signal and --alarm-timeout
* Remove server-type flag (-T / TEST_DEPLOY_TYPE)
* Delete sound-related modules
* Simplify RunConfig and runner error handling
* Update help output, scripts and tests
* Remove optional audio extras from pyproject.toml

https://chatgpt.com/share/698b74f1-0c0c-800f-98c4-183697d6a2a0
2026-02-10 19:10:37 +01:00
kevinveenbirkenbach 449781fa73 fix(ci): make sufficient_storage JSON-safe and stabilize test matrix generation
Switch sufficient_storage CLI to support JSON output and update CI matrix
generation to use JSON-only pipelines, preventing roles from collapsing
into a single matrix entry.
2026-02-10 14:06:54 +01:00
kevinveenbirkenbach df9a23ceed feat(ci): unify invokable app discovery and add lifecycle filtering
- Move server/workstation/universal resolution into Python CLI
- Add optional --lifecycles filter to cli.meta.applications.type
- Gate invokable roles by galaxy_info.lifecycle in module_utils.invokable
- Replace legacy resolve-invokable-roles.sh with CLI-based implementation
- Apply global hard excludes consistently across all deploy types
- Fix SC2086 issues and make lifecycle argument handling shell-safe
- Extend unit tests to cover lifecycle forwarding and normalization

https://chatgpt.com/share/698b0039-c4c4-800f-b962-7033324c07a8
2026-02-10 11:50:09 +01:00
kevinveenbirkenbach 84782f3b22 refactor(invokable): centralize app discovery, add type grouping, and derive deploy stages from inventory groups
https://chatgpt.com/share/698b0039-c4c4-800f-b962-7033324c07a8
2026-02-10 10:53:53 +01:00
kevinveenbirkenbachandCopilot Autofix powered by AI c84616fd50 Potential fix for code scanning alert no. 168: Module is imported with 'import' and 'import from'
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-02-09 07:59:57 +01:00
kevinveenbirkenbach b78ab29218 Renamed config entry from docker to compose
https://chatgpt.com/share/69870f54-d708-800f-9870-e9ae5a4bff44
2026-02-07 11:09:36 +01:00
kevinveenbirkenbach 8a81630391 Added newline at file and POSIX compatible 2026-02-04 19:49:56 +01:00
kevinveenbirkenbach 9a84fafec5 feat(inventory): add mirror_policy to control mirror image overrides
- Default policy (if_missing): manual image/version win, mirror fills blanks only
- Add mirror_policy=force to always overwrite image/version
- Add mirror_policy=skip to ignore mirror completely
- Treat empty/whitespace values as missing
- Preserve non-image/version service keys
- Extend unit tests to cover all mirror policy scenarios

https://chatgpt.com/share/6981036a-7e40-800f-b43e-13c66cc55fac
2026-02-02 21:04:53 +01:00
kevinveenbirkenbach a70a033864 feat(inventory): add --mirror support to override docker images from mirrors file
- add --mirror flag to inventory creator
- resolve mirrors.yml automatically or via explicit path
- apply mirror overrides to applications.*.docker.services.* (image/version only)
- keep existing host_vars keys untouched
- add unit tests for mirror override behavior

https://chatgpt.com/share/6980cdb1-1e5c-800f-b25f-a1af757e4239
2026-02-02 17:15:30 +01:00
kevinveenbirkenbach 4a5736615f Solved falsy file paths 2026-01-29 17:43:22 +01:00
kevinveenbirkenbach b2fb72d112 feat(tls): split TLS settings to 02_tls and add self-signed root CA role
Move TLS/ACME/CA vars out of 00_general into group_vars/all/02_tls.yml and renumber subsequent group_vars files. Add sys-ca-selfsigned to generate /etc/<software>/ca/root-ca.crt and make docker-compose-ca depend on it. Update setup script, templates, and tests to use the new applications/users file numbering.

https://chatgpt.com/share/697b8150-22ac-800f-98b1-e063a659b684
2026-01-29 16:48:03 +01:00
kevinveenbirkenbach fae7b82243 feat(security): introduce strong_password filter backed by ValueGenerator
- add strong_password Jinja2 filter as thin wrapper around ValueGenerator
- enforce password policy (upper, lower, digit, special, min length)
- replace insecure ascii-only password lookup with strong_password filter
- update CLI setup users and unit tests accordingly
- harden inventory sample password example

https://chatgpt.com/share/69722c7e-ebe0-800f-a9ab-a3e21b43269c
2026-01-22 14:56:05 +01:00
kevinveenbirkenbach a565344ed5 Solved shellcheck 2026-01-21 00:25:35 +01:00
kevinveenbirkenbach ea78ff376f test(container): adapt CLI tests for required INVENTORY_DIR and forwarding
- add --inventory-dir to container CLI test invocations
- assert inventory_dir is forwarded to run_in_container()
- add tests for required INVENTORY_DIR when env is missing
- add test for INVENTORY_DIR env fallback

https://chatgpt.com/share/69700d69-b270-800f-ad50-d9ff486378b9
2026-01-21 00:23:54 +01:00
kevinveenbirkenbach ba4527a8db ruff 2026-01-19 21:17:56 +01:00
kevinveenbirkenbach ab7bb0b6fb feat(mastodon): add HTTP-only override and disable force_ssl via initializer
- Add optional HTTP-only mode for Mastodon deployments
- Inject override initializer to hard-disable force_ssl and ActionDispatch::SSL
- Mount override into Mastodon and Sidekiq containers
- Prevent automatic compose flush before migrations
- Apply overrides before setup phase

Also:
- Remove unconditional HSTS header from ws_generic proxy vhost
- Make OIDC optional for CI (docker-in-docker)
- Fix env.j2 quoting and formatting
- Gate HSTS in Nextcloud behind TLS_ENABLED

https://chatgpt.com/share/696e914c-c758-800f-b16e-6341873c0e0e
2026-01-19 21:17:04 +01:00
kevinveenbirkenbach f56cfa773e **test(unit): add full unittest coverage for sufficient_storage CLI and role_resource_validation**
https://chatgpt.com/share/696e3d20-276c-800f-87a7-a7052989ffe4
2026-01-19 19:29:08 +01:00
kevinveenbirkenbach c92aff464b feat(ci): add min_storage validation and CI filtering with warnings
- Add sufficient_storage CLI for validating docker.services.<entity>.min_storage
- Introduce role_resource_validation utility with human-friendly size parsing
- Filter CI app discovery by available runner storage (default: 12GB)
- Emit GitHub Actions warnings for:
  - missing min_storage metadata (treated as 0GB)
  - roles exceeding available runner storage
- Add BigBlueButton min_storage metadata (50GB)
- Add unit tests for validation logic and CLI behavior
- Add humanfriendly dependency for size parsing

https://chatgpt.com/share/696e097b-1038-800f-b8fc-23e9dadfa021
2026-01-19 11:37:28 +01:00
kevinveenbirkenbach 7a1fb02554 test(cli): add unit tests for combined + services resolution (services edges, desktop, cycles)
- add role_config_path support and services edges in combined resolver/tree
- add services resolver mapping with desktop enabled rule (non-shared)
- add unittest coverage for repo_paths, yaml parsing, services mapping/BFS, and combined tree cycles
- update tests to reflect new stable path expectations and tree output

https://chatgpt.com/share/696cb80f-e8f4-800f-b062-e47985b4b074
2026-01-18 11:37:41 +01:00
kevinveenbirkenbach cf216014e3 refactor(cli): introduce combined role resolution and reorganize meta resolvers
- add cli.meta.applications.resolution.combined to resolve run_after + app-only dependencies
- add optional --tree output for combined resolution
- add cli.meta.applications.resolution.dependencies (app-only) resolver
- move run_after resolver to cli.meta.applications.resolution.run_after
- update test deploy dependency resolver to use combined module
- add unit tests for combined/dependencies/run_after resolvers
- minor cleanup: remove stray header comment, fix newline in vars, tidy YAML formatting

https://chatgpt.com/share/696ca531-9644-800f-98c0-bae58c0aff41
2026-01-18 10:17:26 +01:00
kevinveenbirkenbach fa422ed7fe cli: allow comma-separated application IDs for --id flag
- Support space-separated, comma-separated and mixed --id values
- Normalize and deduplicate application IDs before validation
- Fix Makefile to pass APP without quoting
- Add unittest coverage for ID normalization

https://chatgpt.com/share/696be0c2-816c-800f-8b07-3c5cd0304ac5
2026-01-17 20:19:13 +01:00
kevinveenbirkenbach 357a9bda83 fix(users): normalize reserved flag handling and align CLI/tests
- Always emit reserved flag in generated users (default: false)
- Do not override existing user definitions via --reserved-usernames
- Normalize description default to empty string
- Align unit tests with new reserved semantics
- Fix YAML quoting and trailing newline issues
- Clean up minor formatting and config inconsistencies

https://chatgpt.com/share/696bb761-0048-800f-9cf7-aebd3fda75be
2026-01-17 17:22:50 +01:00
kevinveenbirkenbach 505af3e2ad cli: make --log require explicit directory and create parents automatically
- Change --log flag to require explicit log directory (--log <LOG_DIR>)
- Automatically create log directory with parents=True
- Harden permissions best-effort (0700)
- Update CLI help text accordingly
- Adjust Makefile test targets to pass explicit log directory
- Ignore logs/ via .gitignore and .dockerignore
- Refactor flag parsing to correctly detect deploy with --log
- Add dedicated logging unit tests
- Update existing tests for new log semantics
- Remove deprecated -i alias for --id in deploy command

https://chatgpt.com/share/6969680a-dbf8-800f-aafb-3d32d6212ba0
2026-01-15 23:19:31 +01:00
kevinveenbirkenbach ea63818216 refactor(cli): simplify dedicated deploy wrapper and add ansible passthrough support
- Remove unused cli_root and logs handling from dedicated deploy wrapper
- Add parse_known_args() to forward native ansible-playbook flags
- Append passthrough args at the end for proper override behavior
- Clean up MODE_LOGS usage
- Update runner signature and cleanup phase logic
- Adapt unit tests to new wrapper contract
- Add unit test for ansible flag passthrough in command.main()

https://chatgpt.com/share/69694ad5-453c-800f-8047-cf210f60460f
2026-01-15 21:15:06 +01:00
kevinveenbirkenbach 66d50e2dde feat(inventory): resolve shared providers transitively and support schema-less roles
- Allow credentials generator to run for roles without schema/main.yml so
  transitive config-based generation still works.
- Make InventoryManager resilient to missing YAML by defaulting to {} and
  only loading schema when schema/main.yml exists.
- Add recursive shared-provider include resolution based on docker.services
  config and apply provider schemas/special rules transitively.
- Require database_password injection only when database.enabled=true and shared=true.
- Update unit/integration tests to create required files for .exists() checks and
  validate transitive provider role credential generation.

https://chatgpt.com/share/69639d93-4c10-800f-bc6c-8e5b9fb75088
2026-01-11 13:54:37 +01:00
kevinveenbirkenbach e4baa6d7ab Ruff 2026-01-11 01:01:44 +01:00
kevinveenbirkenbach 64cab86be3 refactor(inventory): extract secret generation into ValueGenerator and add integration test
- Move generate_value/generate_secure_alphanumeric out of InventoryManager into new ValueGenerator
- Wire InventoryManager to use self.value_generator for all secret generation
- Tighten schema include logic for shared databases: require enabled=true and shared=true
- Update unit tests to patch ValueGenerator.generate_value instead of InventoryManager.generate_value
- Add end-to-end integration test for InventoryManager using real files and a fake VaultHandler
- Remove redundant legacy unit test file and add dedicated ValueGenerator unit tests

https://chatgpt.com/share/6962d2e6-7fdc-800f-ba83-9e150b19a5f0
2026-01-11 00:56:18 +01:00
kevinveenbirkenbach 3ce6eb348a refactor(inventory): switch feature flags to docker service config paths
Replace legacy feature flags (features.central_database, features.oauth2) with
docker service based configuration:

- docker.services.database.shared
- docker.services.oauth2.enabled

Update InventoryManager logic accordingly and adapt all unit tests to mock
the new config structure.

This ensures inventory generation now follows the canonical docker service
configuration model.

https://chatgpt.com/share/69628906-5efc-800f-999b-2ae3ece1907f
2026-01-10 18:14:29 +01:00
kevinveenbirkenbach 83985163e2 feat(cli): add run_after resolution command with cycle-safe traversal
- Introduce CLI tool to resolve galaxy_info.run_after dependencies transitively
- Detect and report circular run_after dependencies explicitly
- Validate role existence and run_after schema strictly
- Provide whitespace-separated output for easy shell usage
- Add comprehensive unittest coverage including CLI behavior and error cases

https://chatgpt.com/share/695f8e09-54f4-800f-9f5f-ed9c2fdd664b
2026-01-08 11:59:17 +01:00
kevinveenbirkenbach 29483a6415 cli: add lifecycle_filter command and fix help forwarding
- Introduce meta roles lifecycle_filter CLI to select roles by lifecycle
- Support whitelist/blacklist modes, selection filtering, and auto-detected roles dir
- Fix blacklist handling for roles with missing lifecycle
- Forward --help correctly to subcommand argparse
- Add unit tests for lifecycle filtering logic

https://chatgpt.com/share/695d09d2-c864-800f-9e23-b9db0c6f3537
2026-01-06 14:10:33 +01:00
kevinveenbirkenbach 0f7add0f1a Unify Mailu token handling via users[*].tokens and clean up inventory validation
- Migrate Mailu credentials from legacy mailu_token to tokens['web-app-mailu']
- Update all affected application templates to use the new token structure
- Adjust inventory validation to no longer special-case mailu_token
- Minor typing and formatting cleanups in inventory creation and tests

https://chatgpt.com/share/69554915-87c8-800f-afd5-12118dbb1107
2025-12-31 17:02:20 +01:00
kevinveenbirkenbach 533a1235fe Implement SPOT-based authorized_keys management for users
* Move SSH key handling to users.<name>.authorized_keys (list-only) and write keys via shared user role
* Enforce at least one SSH key for administrator user
* Wrap backup user keys with restricted command wrapper before writing authorized_keys
* Remove legacy inventory file copy and CLI flag/function for administrator authorized_keys
* Extend user setup generator to include authorized_keys field
* Rename administrator scripts path to root scripts path and update dependent variables
* Update inventory sample and docs to reflect new SSH key workflow

https://chatgpt.com/share/695510b8-23f4-800f-b5db-877b07a1469b
2025-12-31 13:01:45 +01:00
kevinveenbirkenbach ea04035e12 feat(config): rename SSL_ENABLED to TLS_ENABLED across stack
* Replace global `SSL_ENABLED` flag with `TLS_ENABLED`
* Update web protocol/ports (http/https, ws/wss) to use `TLS_ENABLED`
* Switch cert-related roles and nginx TLS header gating to `TLS_ENABLED`
* Update app env templates (BookWyrm, Confluence, Jira, Magento, OpenProject, WordPress, etc.) to use `TLS_ENABLED`
* Adjust inventory generator to write `TLS_ENABLED` and update CLI help text accordingly
* Update unit tests for inventory vars overrides to expect `TLS_ENABLED`

https://chatgpt.com/share/695306ef-a798-800f-b118-e07b525e1811
2025-12-29 23:55:20 +01:00
kevinveenbirkenbach 6166740c25 Included missing import 2025-12-29 21:36:36 +01:00
kevinveenbirkenbach 1e5ebbb05f feat(inventory): add --vars-file support for YAML-based host_vars overrides
- add --vars-file CLI option to load host_vars overrides from YAML
- merge vars-file overrides before JSON --vars (JSON takes precedence)
- implement deep-merge logic for YAML vars files
- update CI deploy script to use vars file instead of inline JSON
- add unit test for apply_vars_overrides_from_file
- remove obsolete MASK_CREDENTIALS_IN_LOGS TODO entry

https://chatgpt.com/share/6952e51d-6c50-800f-8764-31dc8c609cdb
2025-12-29 21:31:05 +01:00
kevinveenbirkenbach bf4af7c823 Renamed domain variables 2025-12-29 20:37:47 +01:00
kevinveenbirkenbach d2e88b6a8c feat(cli): eliminate runpy warnings for inventory + container entrypoints
* Remove legacy **main** re-export wrappers to avoid `runpy` "already in sys.modules" warnings
* Introduce `cli.deploy.container.command` as the stable implementation module
* Reduce `cli.deploy.container.__main__` to a minimal entrypoint delegating to `command.main`
* Re-export container helpers from package `__init__` without importing `__main__`
* Update unit tests to import/patch the new command module and keep CLI dispatch stable

https://chatgpt.com/share/69484a22-60dc-800f-a23d-068d7c45fe37
2025-12-21 20:27:26 +01:00