Commit Graph
14682 Commits
Author SHA1 Message Date
kevinveenbirkenbach c158832b52 Minor optimations 2026-01-09 12:09:41 +01:00
kevinveenbirkenbach eec7bb17db feat(docker-compose): add robust compose pull helper with retries and unit tests
- Add compose_pull.py helper with retry/backoff logic and transient network detection
- Replace shell-based docker compose pull handler with script-based implementation
- Add lockfile handling to avoid repeated pulls
- Add comprehensive unit tests for retry logic, locking, and command execution
- Improve logging with explicit command tracing

https://chatgpt.com/share/6960dc8d-f884-800f-b6fe-c23a5edf5ba3
2026-01-09 11:46:30 +01:00
kevinveenbirkenbach 151cfdac31 fix(sys-util-git-pull): add retries/backoff for transient git network failures
Introduce retry logic with exponential backoff for git operations to reduce flaky
CI/Docker failures (e.g., "Connection reset by peer"). Replace run(check=True)
with run_checked() and add transient error detection. Update unit tests to match
new execution and error-handling behavior.

https://chatgpt.com/share/6960d28d-3e2c-800f-943b-a80630a6a395
2026-01-09 11:03:53 +01:00
kevinveenbirkenbach 82e4082a6d sys-ctl-bkp-docker-2-loc: add file_has_content module and conditional backup flush
- Add custom Ansible module file_has_content to validate databases.csv on target host
- Fail hard if databases.csv is missing or not a regular file
- Compute backup force-flush flag based on MODE_BACKUP and actual file content
- Prevent instant backup flush when no database secrets are present
- Add full unittest coverage for file_has_content module
- Make sys-service flush behavior data-driven and deterministic

https://chatgpt.com/share/6960cdb6-dd28-800f-8427-2e375de4e61b
2026-01-09 10:43:13 +01:00
kevinveenbirkenbach 853528a4ca Removed fail-fast strategy for workstations 2026-01-09 01:06:41 +01:00
kevinveenbirkenbach 4390e3e847 Changed to fail fast strategy for workstations to improve test time 2026-01-09 01:03:42 +01:00
kevinveenbirkenbach d7f13d30c6 fix(ci): wait for systemd readiness after compose up
Add a systemd readiness gate to the CI compose wrapper to avoid a race
where systemctl is called before DBus/systemd is fully ready.
This prevents sporadic failures like:
"System has not been booted with systemd as init system (PID 1)".

No Dockerfile or Ansible role changes required.

https://chatgpt.com/share/696045b5-bd08-800f-acd9-e3a4e2e23e52
2026-01-09 01:02:54 +01:00
kevinveenbirkenbach f0e777dca1 refactor(ci-deploy): run run_after resolution inside compose container
Resolve run_after dependencies via Compose.exec inside the infinito
CI container instead of invoking the resolver on the host.

This ensures:
- consistent runtime environment (PYTHONPATH, installed code)
- correct resolution when local and container sources differ
- alignment with other compose-based CI execution paths

https://chatgpt.com/share/69603903-d158-800f-815d-c506e3a3a44a
2026-01-09 00:08:39 +01:00
kevinveenbirkenbach c88419c487 fix(deploy-test): do not swallow exit code when keeping stack on failure
Avoid returning from finally in the test runner, which previously
overrode non-zero exit codes when --keep-stack-on-failure was set.
The compose stack is now conditionally kept without affecting the
actual test result.

https://chatgpt.com/share/696032fd-57d8-800f-bcc8-a0edf7b829a4
2026-01-08 23:43:05 +01:00
kevinveenbirkenbach 1595520dc1 Added "--keep-stack-on-failure" to workflow for local debugging 2026-01-08 23:32:43 +01:00
kevinveenbirkenbach 5a6c58eabf Optimized workflow test 2026-01-08 22:39:41 +01:00
kevinveenbirkenbach 6b99fd7462 fix(ci): restore fail-fast behavior for server deploy tests
- stop iterating over distros after first deploy failure
- rely on set -euo pipefail instead of manual rc handling
- keep failure log output in dedicated workflow step

https://chatgpt.com/share/69601617-9808-800f-a7cd-f5e65e40c04e
2026-01-08 22:00:25 +01:00
kevinveenbirkenbach 1ca4fa5025 fix(ci): make server deploy tests robust and show logs on failure
- continue testing all distros and fail at end if any deploy fails
- prevent make down from aborting the test loop
- move tail-on-failure logic to a dedicated workflow step
- improve local debugging with act and skip artifact upload there

https://chatgpt.com/share/69601617-9808-800f-a7cd-f5e65e40c04e
2026-01-08 21:39:47 +01:00
kevinveenbirkenbach fbca540a4e ci: disable deploy log artifact upload when ACT=true
actions/upload-artifact requires ACTIONS_RUNTIME_TOKEN which is not
available when running workflows locally via act. Skip the upload
step in that case to avoid false CI failures.

https://chatgpt.com/share/6960147b-4a2c-800f-bb40-0d31bf1ebaa5
2026-01-08 21:32:54 +01:00
kevinveenbirkenbach 927048c798 Adapted Test-Deploy function 2026-01-08 21:18:59 +01:00
kevinveenbirkenbach b9174cce60 ci(test-deploy-server): fix artifact path for per-app deploy logs
https://chatgpt.com/share/696009f9-80b0-800f-a19c-3f5e7e108ae3
2026-01-08 20:50:39 +01:00
kevinveenbirkenbach 47cbc1e409 ci(test-deploy-server): exclude web-app-oauth2-proxy from server deploy matrix
Filter web-app-oauth2-proxy during app discovery so it is not included
in the server deploy test matrix.

https://chatgpt.com/share/696009f9-80b0-800f-a19c-3f5e7e108ae3
2026-01-08 20:47:56 +01:00
kevinveenbirkenbach 493c908931 fix(vars): guard applications access and fix OIDC flavor ternary precedence
- Default applications to an empty mapping to avoid undefined access
- Add explicit parentheses to ensure correct Jinja filter and ternary evaluation
- Stabilize Nextcloud OIDC flavor resolution when inventory provides no applications

https://chatgpt.com/share/695fff6e-8d20-800f-be9a-d0ad30ee198e
2026-01-08 20:17:58 +01:00
kevinveenbirkenbach 7cbbb4a35e Added whitepaces 2026-01-08 20:06:07 +01:00
kevinveenbirkenbach 1c0ca937e1 fix(nextcloud): guard applications access and correct 2FA OIDC negation
- Prevent 'applications is undefined' by defaulting to an empty mapping
- Fix operator precedence when negating OIDC feature for 2FA plugins
- Make Nextcloud feature toggles robust when inventory provides no applications

https://chatgpt.com/share/695fff6e-8d20-800f-be9a-d0ad30ee198e
2026-01-08 20:02:47 +01:00
kevinveenbirkenbach 1dac0fe82e fix(openresty): bootstrap minimal nginx.conf before container startup
Ensure OpenResty can start even when no configuration is present yet by
providing a minimal, valid nginx.conf bootstrap.

This decouples container startup from configuration deployment and prevents
handler execution from failing when Docker/OpenResty is initialized before
NGINX config templates are rendered.

https://chatgpt.com/share/695ff412-4324-800f-a76d-27ff2bf2dff7
2026-01-08 19:13:58 +01:00
kevinveenbirkenbach 54a027ad8c test(ci): apply lifecycle gating during server app discovery
Move lifecycle filtering into discover-server-apps.sh so only
lifecycle-eligible web-app-* and web-svc-* roles are discovered.

This prevents invalid per-app deploy tests where applications are
excluded later during inventory generation, causing "not present in
inventory" errors.

- Filter invokable roles by server prefixes
- Apply lifecycle_filter blacklist using TESTED_LIFECYCLES
- Always emit valid JSON for CI matrix generation

https://chatgpt.com/share/695fd936-0ee4-800f-a57f-2d7995e15ba0
2026-01-08 17:19:53 +01:00
kevinveenbirkenbach 7941af48b9 ci(test-deploy-server): enable fail-fast for app matrix
Stop running remaining matrix jobs as soon as one app deployment fails,
reducing CI runtime and avoiding unnecessary resource usage.

https://chatgpt.com/share/695fd23c-c924-800f-8d8e-d88b44480eb9
2026-01-08 16:50:11 +01:00
kevinveenbirkenbach d99c7e49a5 fix(ci): reintroduce lifecycle gating in deploy test runner
- Restore lifecycle-based role exclusion via cli.meta.roles.lifecycle_filter
- Make tested lifecycles configurable through TESTED_LIFECYCLES env
- Apply lifecycle gating consistently for workstation and server tests
- Fix minor bug in invokable parsing (avoid shadowing built-in list)
- Improve logging with lifecycle exclusion details

https://chatgpt.com/share/695fbcf4-2148-800f-8685-56ee73a9bae0
2026-01-08 15:19:26 +01:00
kevinveenbirkenbach 851caf9a3e refactor(valid_deploy_id): resolve roles directory from repo root and adapt tests
- Always derive roles/ path from repository root instead of accepting a parameter
- Fail fast if roles directory is missing or mislocated
- Add explicit typing and Path-based path handling
- Update unit tests to rely on real repo roles instead of temporary fixtures
- Ensure tests cover correct role discovery and inventory validation

https://chatgpt.com/share/695fb926-1f5c-800f-9012-e2922bb1df3f
2026-01-08 15:13:14 +01:00
kevinveenbirkenbach b547bec656 chore(debug): log roles directory used for deploy id validation
- Print effective roles_dir in ValidDeployId to simplify CI debugging
- Helps diagnose invalid application_id errors caused by wrong working directory

https://chatgpt.com/share/695fb926-1f5c-800f-9012-e2922bb1df3f
2026-01-08 15:03:10 +01:00
kevinveenbirkenbach 3135d2377a Solved missing imports 2026-01-08 14:51:02 +01:00
kevinveenbirkenbach 5a9f1f1dfb feat(deploy): dump full inventory on invalid application_id
- Print full raw inventory file when application_id validation fails
- Fix invalid reason detection to use in_roles / in_inventory flags
- Simplify CI debugging for per-app dedicated deploy tests

https://chatgpt.com/share/695fb5d2-801c-800f-a1c8-06c2bcace450
2026-01-08 14:48:45 +01:00
kevinveenbirkenbach b30335903f Adapted default file 2026-01-08 14:21:01 +01:00
kevinveenbirkenbach 29c3fdcc08 fix(user): make authorized_keys optional
Only validate authorized_keys when defined and skip SSH key setup
when no keys are configured for a user.

https://chatgpt.com/share/695fac99-c6a4-800f-8914-1d56c63e8ccf
2026-01-08 14:09:29 +01:00
kevinveenbirkenbach d6b0fb3be7 refactor(ci): run server deploy per app across all distros in single job
* Avoid GitHub Actions matrix job limits by looping distros inside the per-app job
* Simplify discover output emission and pin checkout/upload-artifact to stable versions
* Tidy long log lines and minor formatting in deploy/test helpers

https://chatgpt.com/share/695fa9b9-4460-800f-bcc3-d566b83649b2
2026-01-08 13:57:22 +01:00
kevinveenbirkenbach fab08aadf6 refactor(ci): split deploy tests into per-app server matrix and workstation workflow
- Replace shared test-deploy workflow with dedicated server/workstation workflows
- Add discovery step to generate per-app server matrix from invokable roles
- Introduce helper script to output discovered server apps as JSON
- Keep compose stack cleanup robust via trap and always upload per-run logs

https://chatgpt.com/share/695f9e22-6e9c-800f-9a50-65f553619af2
2026-01-08 13:30:07 +01:00
kevinveenbirkenbach dba121ecf4 refactor(ci): move deploy test runner from bash to python orchestrator
* Add `cli.deploy.test` module to orchestrate compose-based CI deploy tests
* Support server per-app runs (including run_after dependencies) and workstation selection
* Add optional image build flags (--no-cache, --missing) and stack retention on failure
* Simplify `scripts/tests/deploy.sh` to a thin wrapper that forwards args to the Python runner

https://chatgpt.com/share/695f9e22-6e9c-800f-9a50-65f553619af2
2026-01-08 13:07:57 +01:00
kevinveenbirkenbach cf79a3e7ff Release version 0.10.0 v0.10.0 2026-01-08 12:41:30 +01:00
kevinveenbirkenbach 047c86058a Added workstation user to pass variables to pass integration tests 2026-01-08 12:36:57 +01:00
kevinveenbirkenbach 316a693ec1 Merge pull request #49 from kevinveenbirkenbach/dependabot/github_actions/actions/upload-artifact-6
chore(deps): bump actions/upload-artifact from 4 to 6
2026-01-08 12:29:08 +01:00
kevinveenbirkenbach 3a5912845f fix(docker): remove implicit cgroup volume from base image
Drop VOLUME /sys/fs/cgroup from Dockerfile and rely on docker-compose
to mount cgroups explicitly for systemd-based CI containers.
Prevents creation of anonymous volumes in derived images.

https://chatgpt.com/share/695f94d9-d4a0-800f-a9eb-19e67445a0fe
2026-01-08 12:28:07 +01:00
kevinveenbirkenbach 83985163e2 feat(cli): add run_after resolution command with cycle-safe traversal
- Introduce CLI tool to resolve galaxy_info.run_after dependencies transitively
- Detect and report circular run_after dependencies explicitly
- Validate role existence and run_after schema strictly
- Provide whitespace-separated output for easy shell usage
- Add comprehensive unittest coverage including CLI behavior and error cases

https://chatgpt.com/share/695f8e09-54f4-800f-9f5f-ed9c2fdd664b
2026-01-08 11:59:17 +01:00
dependabot[bot] 15956efef6 chore(deps): bump actions/upload-artifact from 4 to 6
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4 to 6.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v4...v6)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-08 02:34:21 +00:00
kevinveenbirkenbach 9905ed619a refactor(user): migrate shared user handling to strict user_key SPOT and introduce workstation role
- Refactor shared `user` role to use `user_key` exclusively (remove user_name)
- Resolve system username, home, group, and ownership via users[user_key]
- Update SSH authorized_keys handling to read from users[user_key]
- Adjust documentation to clearly describe user_key-based SPOT model
- Migrate all calling roles (administrator, backup, root) to pass user_key
- Introduce new `user-workstation` role with dedicated README, vars, tasks, and metadata
- Add workstation stage integration to ensure workstation user setup runs explicitly
- Ensure consistent variable naming (lowercase internals, uppercase constants)
- Improve robustness and clarity of user management across all roles

https://chatgpt.com/share/695ea93e-3fa0-800f-b848-ec2ced2a8291
2026-01-07 20:27:28 +01:00
kevinveenbirkenbach e2fee890c4 fix(desk-git): use gitcon instead of gitconfig
- Install gitcon via system pip
- Invoke gitcon CLI for Git setup
- Avoid broken gitconfig dependency (fullpath)

https://chatgpt.com/share/695ea93e-3fa0-800f-b848-ec2ced2a8291
2026-01-07 19:42:52 +01:00
kevinveenbirkenbach 335201d8a0 ci(deploy): show deploy type in job name (server/workstation)
https://chatgpt.com/share/695ea753-04a8-800f-91a4-4ec5b41cfdc0
2026-01-07 19:34:48 +01:00
kevinveenbirkenbach c02ac72a16 fix(mailu): handle optional user roles safely
- Make user roles optional and default to empty list
- Avoid failures when roles key is missing or null
- Centralize role-based logic via mailu_user_roles
- Ensure admin/user action selection remains consistent

https://chatgpt.com/share/695ea61f-2fc8-800f-adba-60b2f12763e8
2026-01-07 19:29:31 +01:00
kevinveenbirkenbach 975dd516d4 fix(desk-git): install gitconfig system-wide and run setup as workstation user
- Replace pkgmgr-install with sys-pip-install for gitconfig CLI
- Ensure gitconfig is available in PATH during Ansible runs
- Run git configuration as WORKSTATION_USER to write into correct home
- Prevent command-not-found errors during initial setup

https://chatgpt.com/share/695ea278-7888-800f-aa16-d17b7d3317ef
2026-01-07 19:14:08 +01:00
kevinveenbirkenbach d57843492b Release version 0.9.0 v0.9.0 2026-01-07 19:06:09 +01:00
kevinveenbirkenbach 9dbfe081c4 fix(sys-hostname): skip hostname setup inside Docker containers
Prevent hostname configuration from running when DOCKER_IN_CONTAINER is true,
in addition to the existing run-once guard.

https://chatgpt.com/share/695e9dfe-b3b0-800f-a139-393c506d88f0
2026-01-07 18:55:06 +01:00
kevinveenbirkenbach 6f00f737f8 refactor(workstation): unify user handling via WORKSTATION_USER
- Introduce WORKSTATION_USER in sample inventory
- Switch desktop roles from desktop_username to users[WORKSTATION_USER]
- Add placeholder GPG key for git signing configuration
- Modernize Arduino user group task to structured user module syntax
- Rename desk-docker README title for clarity

https://chatgpt.com/share/695e9c5b-4f80-800f-a31d-95d800f98c21
2026-01-07 18:48:07 +01:00
kevinveenbirkenbach f517012934 fix(mail): handle postfix ↔ msmtp-mta conflict cleanly
- Remove postfix (+ postfix-lmdb) when external SMTP is enabled to avoid pacman conflict with msmtp-mta
- Ensure postfix-lmdb is installed alongside postfix in local SMTP role

https://chatgpt.com/share/695e9b98-4f94-800f-90fd-9a81fe8405de
2026-01-07 18:44:48 +01:00
kevinveenbirkenbach e64aa27825 feat(svc-bkp-rmt-2-loc): fail fast per host, continue across providers
- Make pull-specific-host.py fail on any command error (and rsync after retries), emit clear stderr, and exit 0/1 via main()
- Keep remote `find ... -execdir basename {} ;` unchanged to stay compatible with existing ssh-wrapper.sh
- Update wrapper script to continue across hosts but return exit 1 if any host failed
- Adjust unit tests to expect exceptions instead of SystemExit

https://chatgpt.com/share/695e9615-6014-800f-a77d-5809cf45559d
2026-01-07 18:21:21 +01:00
kevinveenbirkenbach 8d3e8d0b73 feat(email): unify SYSTEM_EMAIL_* vars and improve msmtp/postfix local delivery
- Replace DEFAULT_SYSTEM_EMAIL merge with flat SYSTEM_EMAIL_* variables
- Update templates across apps to new variable names
- Fix msmtp health check to use -t and local username fallback
- Improve postfix localhost config (localdomain, mydestination, timeouts)
- Extend sys-hostname to ensure /etc/hosts mapping once

https://chatgpt.com/share/695e8f39-0694-800f-b91c-a74f4b78d7c8
2026-01-07 17:52:04 +01:00