Commit Graph
242 Commits
Author SHA1 Message Date
kevinveenbirkenbach 49df004a5f General py optimations 2026-01-15 05:02:55 +01:00
kevinveenbirkenbach ef4a6611d7 ci(deploy): log effective compose env without leaking secrets
Print selected INFINITO_* and GITHUB_SHA variables before starting the CI
compose stack and only indicate whether NIX_CONFIG is set, avoiding secret
exposure while making the effective image/tag visible in logs.

https://chatgpt.com/share/69684776-86c0-800f-b089-7e549a83e87f
2026-01-15 02:48:29 +01:00
kevinveenbirkenbach 334c4b70c7 fix: stabilize rapid deploy env, improve entry.sh debugging and harden contact config
- Fix Makefile rapid deploy target to properly export INFINITO_DISTRO
- Capture stdout/stderr from entry.sh init and raise explicit error on failure
- Add fallback contact addresses for demo environments (email, matrix, mastodon, etc.)
- Quote mailto URLs and identifiers to ensure valid YAML output
- Replace placeholder legal contact with demo user

This prevents YAML parser errors (e.g. `url: mailto:`), improves CI diagnostics,
and ensures desktop contact menu is always rendered with valid defaults.

https://chatgpt.com/share/69679c78-f338-800f-be86-54b3195d19b7
2026-01-14 14:38:42 +01:00
kevinveenbirkenbach 3f667aadda ci: switch CI images to latest tag and skip compose build in deploy
- Build CI images workflow now always publishes :latest instead of ci-<sha>
- Removed obsolete tag input and CI tag computation logic
- Deploy in real GitHub Actions now pulls ghcr.io/.../infinito-<distro>:latest
- Skip docker compose build when INFINITO_NO_BUILD=1
- Respect INFINITO_IMAGE for --missing image existence check

This ensures deploy jobs use the already published production image and never rebuild in CI.

https://chatgpt.com/share/69678e2c-7d7c-800f-b422-15e98c8e4b6e
2026-01-14 13:37:54 +01:00
kevinveenbirkenbach 4a008d8b97 ci: centralize INSTALL_LOCAL_BUILD handling across Makefile, compose and runner
- Control INSTALL_LOCAL_BUILD centrally via Makefile depending on CI context:
  - real GitHub Actions runner: disable local rebuild
  - act and local runs: enable local rebuild
- Pass INSTALL_LOCAL_BUILD through docker-compose using environment expansion
- Remove hardcoded INSTALL_LOCAL_BUILD=1 from CI container runner
- Keep Dockerfile local-build during image build for CI images

This ensures consistent local/CI behavior and avoids unnecessary rebuilds on real CI runners.

https://chatgpt.com/share/6967876f-e400-800f-90ec-fe870e565bf3
2026-01-14 13:09:13 +01:00
kevinveenbirkenbach 517ccf44c0 ci: replace rest deploy mode with universal and add universal stage
- Rename CI workflow and MODE from "rest" to "universal"
- Extend CLI and test runner to support universal deploy type
- Add tasks/stages/02_universal.yml for universal host type
- Update discover logic to use universal app selection
- Remove legacy rest->server mapping and obsolete select helper

https://chatgpt.com/share/6967625e-81c8-800f-8592-5d5b8edec1e2
2026-01-14 10:31:06 +01:00
kevinveenbirkenbach 638d9f2dfa Docker optimations 2026-01-14 00:37:48 +01:00
kevinveenbirkenbach 3cf7d20d86 feat(test-deploy): add debug toggle and rapid deploy target
- add --debug / --no-debug flag to deploy test CLI and runner
- make Ansible debug mode optional instead of always enabled
- extend deploy.sh with debug flag forwarding
- add make target test-deploy-rapid for fast single-app deploys

https://chatgpt.com/share/6966c80c-661c-800f-8fb9-769659da290d
2026-01-13 23:32:37 +01:00
kevinveenbirkenbach 585fcb4b53 Used correct relative path 2026-01-13 21:32:25 +01:00
kevinveenbirkenbach 6b7b67681e fix(ci): unblock systemd startup and stabilize Docker healthcheck
Ensure systemd is started immediately as PID 1 in the infinito container
to avoid blocking on pkgmgr during boot. Increase compose runner timeout
to match healthcheck window and make healthcheck script executable in the
image.

This fixes containers getting stuck in "starting" state and makes CI
startup deterministic.

https://chatgpt.com/share/696683a2-9350-800f-a0cf-587d4013cb29
2026-01-13 19:02:43 +01:00
kevinveenbirkenbach 8c82db3556 feat(ci): add Docker healthcheck and wait for healthy container in compose runner
Add a systemd-based Docker HEALTHCHECK that verifies:
- systemd readiness
- infinito CLI availability and executability
- infinito CLI smoke test (--help)

Update CI compose runner to wait for Docker health status instead of
manual systemd polling. The runner now:
- resolves the infinito container via docker compose
- waits for healthy state
- dumps journalctl and docker logs on timeout

This removes boot race conditions and makes container readiness deterministic.

https://chatgpt.com/share/696683a2-9350-800f-a0cf-587d4013cb29
2026-01-13 18:40:33 +01:00
kevinveenbirkenbach 75de900549 test(compose): dump last 200 log lines when systemd wait times out
Extend wait_for_systemd_ready() to automatically print the last 200 log
lines on timeout for easier CI debugging.

- Adds journalctl output (preferred for systemd containers)
- Adds docker logs fallback
- Improves timeout error diagnostics

https://chatgpt.com/share/69667eca-088c-800f-af12-5682d0038351
2026-01-13 18:19:37 +01:00
kevinveenbirkenbach 673dab59d2 ci(deploy): prevent docker compose builds in GitHub Actions
Switch CI detection to GITHUB_ACTIONS and disable compose builds via
INFINITO_NO_BUILD=1. The deploy workflows now rely on pulling the prebuilt
CI image instead of triggering local Docker builds.

- Remove legacy UP_BUILD handling
- Enable --no-build in compose when running in real CI
- Keep local/act builds working as before

https://chatgpt.com/share/69666e0b-f08c-800f-93fd-611eced4b100
2026-01-13 17:08:34 +01:00
kevinveenbirkenbach dac111ccb7 ci(deploy): simplify per-app deploy workflow and Makefile act targets
- Rename _deploy-matrix.yml -> _deploy-per-app.yml and drop unused inputs
  (tested_lifecycles, only_distro, final_exclude_re).
- Keep one runner per app (matrix.app) and run all distros inside that job.
- Makefile: run deploy tests through act workflow_dispatch (no serial script).
- Add scripts/ci/discover.sh and scripts/ci/deploy-one-app.sh as small SRP helpers.
- Update reusable workflow callers to use the renamed file.

https://chatgpt.com/share/69659ff3-1414-800f-8652-5298fcf55398
2026-01-13 03:41:25 +01:00
kevinveenbirkenbach 3329fbdfd5 ci(act): unify deploy matrix + force serial execution with max-parallel
- Add reusable _deploy-matrix workflow with discovery + per-app deploy across distros
- Enforce serial execution via strategy.max-parallel=1 and act --concurrent-jobs 1
- Support filters: mode (server|workstation|rest), only_app, only_distro, include/exclude regex
- Refactor test runner: remove discovery, require --app, deploy single app with deps only
- Generalize discover script to discover-apps.sh with INCLUDE_RE/EXCLUDE_RE + lifecycle gating
- Update Makefile targets to drive workflow_dispatch inputs for local act runs

https://chatgpt.com/share/69659ff3-1414-800f-8652-5298fcf55398
2026-01-13 03:06:56 +01:00
kevinveenbirkenbach 8bf2b4c1e1 Ruff 2026-01-12 17:50:43 +01:00
kevinveenbirkenbach 198a2fe3cf fix(ansible): enforce system python interpreter and install python-ldap via OS packages
- Stop auto-injecting ansible_python_interpreter from PYTHON env for localhost
- Set ansible_python_interpreter globally to /usr/bin/python3
- Add interpreter to sample inventory
- Remove role-local interpreter overrides
- Switch python-ldap installation to distro package (python-ldap / python3-ldap)
- Drop pip-based python-ldap installation to avoid venv/libldap mismatches

https://chatgpt.com/share/6964335a-29cc-800f-ae65-6a35c7cd2559
2026-01-12 14:48:53 +01:00
kevinveenbirkenbach e5f8067db2 fix(deploy-test): apply lifecycle gating to workstation and server deploy tests
Apply TESTED_LIFECYCLES filtering directly in the deploy test runner so roles
with unsupported lifecycles (e.g. pre-alpha) are excluded consistently for
both workstation and server workflows.

This prevents roles like desk-copyq (lifecycle: pre-alpha) from being deployed
in CI unless explicitly enabled via TESTED_LIFECYCLES.

https://chatgpt.com/share/6963e316-0a50-800f-a895-8828269a0ca3
2026-01-11 18:51:09 +01:00
kevinveenbirkenbach 66d50e2dde feat(inventory): resolve shared providers transitively and support schema-less roles
- Allow credentials generator to run for roles without schema/main.yml so
  transitive config-based generation still works.
- Make InventoryManager resilient to missing YAML by defaulting to {} and
  only loading schema when schema/main.yml exists.
- Add recursive shared-provider include resolution based on docker.services
  config and apply provider schemas/special rules transitively.
- Require database_password injection only when database.enabled=true and shared=true.
- Update unit/integration tests to create required files for .exists() checks and
  validate transitive provider role credential generation.

https://chatgpt.com/share/69639d93-4c10-800f-bc6c-8e5b9fb75088
2026-01-11 13:54:37 +01:00
kevinveenbirkenbach e4baa6d7ab Ruff 2026-01-11 01:01:44 +01:00
kevinveenbirkenbach 39f7f8fbc7 feat(create-credentials): generate shared provider app credentials from role schemas
Extend credentials generator to include schemas for shared provider roles
(OpenLDAP, Keycloak, Matomo, and svc-db-<type>) based on docker.services.*.shared
flags in role config. Credentials are now emitted/added for all involved
applications while preserving existing inventory formatting, and CLI overrides
can target specific apps via applications.<app>.credentials.<key>.
2026-01-11 00:00:29 +01:00
kevinveenbirkenbach 2d7a904f2b ruff optimation 2026-01-10 20:06:53 +01:00
kevinveenbirkenbach c0b03bb95a refactor(up): route make up through python orchestrator and entry.sh init
- Replace direct docker compose up with python-based orchestrator (Compose.up)
- Centralize stack startup, systemd readiness wait, and init logic in Compose.up
- Run container init via scripts/docker/entry.sh instead of make target
- Remove duplicate systemd wait in deploy test runner
- Add cli.deploy.test.up entrypoint for make up integration

https://chatgpt.com/share/6962a2c8-0f50-800f-8bfb-2efd16adaf40
2026-01-10 20:04:33 +01:00
kevinveenbirkenbach 9d95eea3b0 fix(inventory): pin ansible_python_interpreter for localhost to avoid auto-discovery issues
Ensure that localhost inventories always define ansible_python_interpreter when using
ansible_connection=local. This prevents Ansible from falling back to interpreter
auto-discovery (interpreter_python=auto_silent), which could select unexpected Python
binaries inside containers (e.g. /usr/sbin/python3.13) and break module execution.

The interpreter is now pinned to $PYTHON if set, otherwise to sys.executable.

https://chatgpt.com/share/69628603-3e18-800f-a6f9-2b367728a9ff
2026-01-10 18:01:20 +01:00
kevinveenbirkenbach 1ca0fd05b0 fix(deploy-test): capture run_after resolver output for dependency resolution
Ensure stdout is captured when calling run_after_resolution inside the infinito
container so declared galaxy_info.run_after deps (e.g. matomo, mailu) are
correctly detected and added to deploy_ids.

https://chatgpt.com/share/69617fbb-5af4-800f-848d-edeb2c1125a8
2026-01-09 23:22:41 +01:00
kevinveenbirkenbach 4146535404 Adapted path 2026-01-09 22:35:06 +01:00
kevinveenbirkenbach 2d25928333 test(deploy): simplify runner by removing lifecycle gating and using include-based inventory
- remove TESTED_LIFECYCLES handling from runner
- build CI inventory via --include instead of large exclude lists
- deploy app together with its run_after dependencies

https://chatgpt.com/share/69616d7d-4af0-800f-8c68-c4ed78f0fbf7
2026-01-09 22:04:50 +01:00
kevinveenbirkenbach 38fcbbc451 fix(deploy-test): do not let lifecycle blacklist override explicit run_after deps
Ensure explicitly included applications (run_after dependencies) are not excluded
again by lifecycle blacklist filtering. Also log lifecycle hits for easier CI
debugging.

https://chatgpt.com/share/6960fe98-683c-800f-9419-0078dc67f195
2026-01-09 14:11:31 +01:00
kevinveenbirkenbach 4390e3e847 Changed to fail fast strategy for workstations to improve test time 2026-01-09 01:03:42 +01:00
kevinveenbirkenbach d7f13d30c6 fix(ci): wait for systemd readiness after compose up
Add a systemd readiness gate to the CI compose wrapper to avoid a race
where systemctl is called before DBus/systemd is fully ready.
This prevents sporadic failures like:
"System has not been booted with systemd as init system (PID 1)".

No Dockerfile or Ansible role changes required.

https://chatgpt.com/share/696045b5-bd08-800f-acd9-e3a4e2e23e52
2026-01-09 01:02:54 +01:00
kevinveenbirkenbach f0e777dca1 refactor(ci-deploy): run run_after resolution inside compose container
Resolve run_after dependencies via Compose.exec inside the infinito
CI container instead of invoking the resolver on the host.

This ensures:
- consistent runtime environment (PYTHONPATH, installed code)
- correct resolution when local and container sources differ
- alignment with other compose-based CI execution paths

https://chatgpt.com/share/69603903-d158-800f-815d-c506e3a3a44a
2026-01-09 00:08:39 +01:00
kevinveenbirkenbach c88419c487 fix(deploy-test): do not swallow exit code when keeping stack on failure
Avoid returning from finally in the test runner, which previously
overrode non-zero exit codes when --keep-stack-on-failure was set.
The compose stack is now conditionally kept without affecting the
actual test result.

https://chatgpt.com/share/696032fd-57d8-800f-bcc8-a0edf7b829a4
2026-01-08 23:43:05 +01:00
kevinveenbirkenbach d99c7e49a5 fix(ci): reintroduce lifecycle gating in deploy test runner
- Restore lifecycle-based role exclusion via cli.meta.roles.lifecycle_filter
- Make tested lifecycles configurable through TESTED_LIFECYCLES env
- Apply lifecycle gating consistently for workstation and server tests
- Fix minor bug in invokable parsing (avoid shadowing built-in list)
- Improve logging with lifecycle exclusion details

https://chatgpt.com/share/695fbcf4-2148-800f-8685-56ee73a9bae0
2026-01-08 15:19:26 +01:00
kevinveenbirkenbach 3135d2377a Solved missing imports 2026-01-08 14:51:02 +01:00
kevinveenbirkenbach 5a9f1f1dfb feat(deploy): dump full inventory on invalid application_id
- Print full raw inventory file when application_id validation fails
- Fix invalid reason detection to use in_roles / in_inventory flags
- Simplify CI debugging for per-app dedicated deploy tests

https://chatgpt.com/share/695fb5d2-801c-800f-a1c8-06c2bcace450
2026-01-08 14:48:45 +01:00
kevinveenbirkenbach b30335903f Adapted default file 2026-01-08 14:21:01 +01:00
kevinveenbirkenbach d6b0fb3be7 refactor(ci): run server deploy per app across all distros in single job
* Avoid GitHub Actions matrix job limits by looping distros inside the per-app job
* Simplify discover output emission and pin checkout/upload-artifact to stable versions
* Tidy long log lines and minor formatting in deploy/test helpers

https://chatgpt.com/share/695fa9b9-4460-800f-bcc3-d566b83649b2
2026-01-08 13:57:22 +01:00
kevinveenbirkenbach dba121ecf4 refactor(ci): move deploy test runner from bash to python orchestrator
* Add `cli.deploy.test` module to orchestrate compose-based CI deploy tests
* Support server per-app runs (including run_after dependencies) and workstation selection
* Add optional image build flags (--no-cache, --missing) and stack retention on failure
* Simplify `scripts/tests/deploy.sh` to a thin wrapper that forwards args to the Python runner

https://chatgpt.com/share/695f9e22-6e9c-800f-9a50-65f553619af2
2026-01-08 13:07:57 +01:00
kevinveenbirkenbach 83985163e2 feat(cli): add run_after resolution command with cycle-safe traversal
- Introduce CLI tool to resolve galaxy_info.run_after dependencies transitively
- Detect and report circular run_after dependencies explicitly
- Validate role existence and run_after schema strictly
- Provide whitespace-separated output for easy shell usage
- Add comprehensive unittest coverage including CLI behavior and error cases

https://chatgpt.com/share/695f8e09-54f4-800f-9f5f-ed9c2fdd664b
2026-01-08 11:59:17 +01:00
kevinveenbirkenbach 8d3e8d0b73 feat(email): unify SYSTEM_EMAIL_* vars and improve msmtp/postfix local delivery
- Replace DEFAULT_SYSTEM_EMAIL merge with flat SYSTEM_EMAIL_* variables
- Update templates across apps to new variable names
- Fix msmtp health check to use -t and local username fallback
- Improve postfix localhost config (localdomain, mydestination, timeouts)
- Extend sys-hostname to ensure /etc/hosts mapping once

https://chatgpt.com/share/695e8f39-0694-800f-b91c-a74f4b78d7c8
2026-01-07 17:52:04 +01:00
kevinveenbirkenbach 29483a6415 cli: add lifecycle_filter command and fix help forwarding
- Introduce meta roles lifecycle_filter CLI to select roles by lifecycle
- Support whitelist/blacklist modes, selection filtering, and auto-detected roles dir
- Fix blacklist handling for roles with missing lifecycle
- Forward --help correctly to subcommand argparse
- Add unit tests for lifecycle filtering logic

https://chatgpt.com/share/695d09d2-c864-800f-9e23-b9db0c6f3537
2026-01-06 14:10:33 +01:00
kevinveenbirkenbach 0f7add0f1a Unify Mailu token handling via users[*].tokens and clean up inventory validation
- Migrate Mailu credentials from legacy mailu_token to tokens['web-app-mailu']
- Update all affected application templates to use the new token structure
- Adjust inventory validation to no longer special-case mailu_token
- Minor typing and formatting cleanups in inventory creation and tests

https://chatgpt.com/share/69554915-87c8-800f-afd5-12118dbb1107
2025-12-31 17:02:20 +01:00
kevinveenbirkenbach 533a1235fe Implement SPOT-based authorized_keys management for users
* Move SSH key handling to users.<name>.authorized_keys (list-only) and write keys via shared user role
* Enforce at least one SSH key for administrator user
* Wrap backup user keys with restricted command wrapper before writing authorized_keys
* Remove legacy inventory file copy and CLI flag/function for administrator authorized_keys
* Extend user setup generator to include authorized_keys field
* Rename administrator scripts path to root scripts path and update dependent variables
* Update inventory sample and docs to reflect new SSH key workflow

https://chatgpt.com/share/695510b8-23f4-800f-b5db-877b07a1469b
2025-12-31 13:01:45 +01:00
kevinveenbirkenbach ea04035e12 feat(config): rename SSL_ENABLED to TLS_ENABLED across stack
* Replace global `SSL_ENABLED` flag with `TLS_ENABLED`
* Update web protocol/ports (http/https, ws/wss) to use `TLS_ENABLED`
* Switch cert-related roles and nginx TLS header gating to `TLS_ENABLED`
* Update app env templates (BookWyrm, Confluence, Jira, Magento, OpenProject, WordPress, etc.) to use `TLS_ENABLED`
* Adjust inventory generator to write `TLS_ENABLED` and update CLI help text accordingly
* Update unit tests for inventory vars overrides to expect `TLS_ENABLED`

https://chatgpt.com/share/695306ef-a798-800f-b118-e07b525e1811
2025-12-29 23:55:20 +01:00
kevinveenbirkenbach 5175aefaf8 Renamed desktop to workstation 2025-12-29 22:44:20 +01:00
kevinveenbirkenbach 1e5ebbb05f feat(inventory): add --vars-file support for YAML-based host_vars overrides
- add --vars-file CLI option to load host_vars overrides from YAML
- merge vars-file overrides before JSON --vars (JSON takes precedence)
- implement deep-merge logic for YAML vars files
- update CI deploy script to use vars file instead of inline JSON
- add unit test for apply_vars_overrides_from_file
- remove obsolete MASK_CREDENTIALS_IN_LOGS TODO entry

https://chatgpt.com/share/6952e51d-6c50-800f-8764-31dc8c609cdb
2025-12-29 21:31:05 +01:00
kevinveenbirkenbach bf4af7c823 Renamed domain variables 2025-12-29 20:37:47 +01:00
kevinveenbirkenbach a9e1a84b71 feat(ci): run Docker-in-Docker in deploy container via fuse-overlayfs
* Remove host Docker socket (/var/run/docker.sock) and DOCKER_HOST wiring for CI container
* Drop /tmp/gh-action bind usage in CI container + compose runner
* Switch OpenResty volume sources to container-local absolute paths (/var/www, /etc/nginx)
* Ensure docker engine installation also runs when DOCKER_IN_CONTAINER is true
* Add container-only fuse-overlayfs setup and write /etc/docker/daemon.json storage-driver config

https://chatgpt.com/share/69528062-ff74-800f-ae11-a658b5917307
2025-12-29 17:47:03 +01:00
kevinveenbirkenbach 8dbe1d66c5 refactor(user-backup): rename sys-bkp-provider-user role and clean up SSH handling
https://chatgpt.com/share/69518670-34b0-800f-84f9-70d001b1e030
2025-12-28 20:35:07 +01:00
kevinveenbirkenbach 5a74f80e85 Executed ruff 2025-12-28 15:23:07 +01:00