Commit Graph
230 Commits
Author SHA1 Message Date
kevinveenbirkenbach 673dab59d2 ci(deploy): prevent docker compose builds in GitHub Actions
Switch CI detection to GITHUB_ACTIONS and disable compose builds via
INFINITO_NO_BUILD=1. The deploy workflows now rely on pulling the prebuilt
CI image instead of triggering local Docker builds.

- Remove legacy UP_BUILD handling
- Enable --no-build in compose when running in real CI
- Keep local/act builds working as before

https://chatgpt.com/share/69666e0b-f08c-800f-93fd-611eced4b100
2026-01-13 17:08:34 +01:00
kevinveenbirkenbach dac111ccb7 ci(deploy): simplify per-app deploy workflow and Makefile act targets
- Rename _deploy-matrix.yml -> _deploy-per-app.yml and drop unused inputs
  (tested_lifecycles, only_distro, final_exclude_re).
- Keep one runner per app (matrix.app) and run all distros inside that job.
- Makefile: run deploy tests through act workflow_dispatch (no serial script).
- Add scripts/ci/discover.sh and scripts/ci/deploy-one-app.sh as small SRP helpers.
- Update reusable workflow callers to use the renamed file.

https://chatgpt.com/share/69659ff3-1414-800f-8652-5298fcf55398
2026-01-13 03:41:25 +01:00
kevinveenbirkenbach 3329fbdfd5 ci(act): unify deploy matrix + force serial execution with max-parallel
- Add reusable _deploy-matrix workflow with discovery + per-app deploy across distros
- Enforce serial execution via strategy.max-parallel=1 and act --concurrent-jobs 1
- Support filters: mode (server|workstation|rest), only_app, only_distro, include/exclude regex
- Refactor test runner: remove discovery, require --app, deploy single app with deps only
- Generalize discover script to discover-apps.sh with INCLUDE_RE/EXCLUDE_RE + lifecycle gating
- Update Makefile targets to drive workflow_dispatch inputs for local act runs

https://chatgpt.com/share/69659ff3-1414-800f-8652-5298fcf55398
2026-01-13 03:06:56 +01:00
kevinveenbirkenbach 8bf2b4c1e1 Ruff 2026-01-12 17:50:43 +01:00
kevinveenbirkenbach 198a2fe3cf fix(ansible): enforce system python interpreter and install python-ldap via OS packages
- Stop auto-injecting ansible_python_interpreter from PYTHON env for localhost
- Set ansible_python_interpreter globally to /usr/bin/python3
- Add interpreter to sample inventory
- Remove role-local interpreter overrides
- Switch python-ldap installation to distro package (python-ldap / python3-ldap)
- Drop pip-based python-ldap installation to avoid venv/libldap mismatches

https://chatgpt.com/share/6964335a-29cc-800f-ae65-6a35c7cd2559
2026-01-12 14:48:53 +01:00
kevinveenbirkenbach e5f8067db2 fix(deploy-test): apply lifecycle gating to workstation and server deploy tests
Apply TESTED_LIFECYCLES filtering directly in the deploy test runner so roles
with unsupported lifecycles (e.g. pre-alpha) are excluded consistently for
both workstation and server workflows.

This prevents roles like desk-copyq (lifecycle: pre-alpha) from being deployed
in CI unless explicitly enabled via TESTED_LIFECYCLES.

https://chatgpt.com/share/6963e316-0a50-800f-a895-8828269a0ca3
2026-01-11 18:51:09 +01:00
kevinveenbirkenbach 66d50e2dde feat(inventory): resolve shared providers transitively and support schema-less roles
- Allow credentials generator to run for roles without schema/main.yml so
  transitive config-based generation still works.
- Make InventoryManager resilient to missing YAML by defaulting to {} and
  only loading schema when schema/main.yml exists.
- Add recursive shared-provider include resolution based on docker.services
  config and apply provider schemas/special rules transitively.
- Require database_password injection only when database.enabled=true and shared=true.
- Update unit/integration tests to create required files for .exists() checks and
  validate transitive provider role credential generation.

https://chatgpt.com/share/69639d93-4c10-800f-bc6c-8e5b9fb75088
2026-01-11 13:54:37 +01:00
kevinveenbirkenbach e4baa6d7ab Ruff 2026-01-11 01:01:44 +01:00
kevinveenbirkenbach 39f7f8fbc7 feat(create-credentials): generate shared provider app credentials from role schemas
Extend credentials generator to include schemas for shared provider roles
(OpenLDAP, Keycloak, Matomo, and svc-db-<type>) based on docker.services.*.shared
flags in role config. Credentials are now emitted/added for all involved
applications while preserving existing inventory formatting, and CLI overrides
can target specific apps via applications.<app>.credentials.<key>.
2026-01-11 00:00:29 +01:00
kevinveenbirkenbach 2d7a904f2b ruff optimation 2026-01-10 20:06:53 +01:00
kevinveenbirkenbach c0b03bb95a refactor(up): route make up through python orchestrator and entry.sh init
- Replace direct docker compose up with python-based orchestrator (Compose.up)
- Centralize stack startup, systemd readiness wait, and init logic in Compose.up
- Run container init via scripts/docker/entry.sh instead of make target
- Remove duplicate systemd wait in deploy test runner
- Add cli.deploy.test.up entrypoint for make up integration

https://chatgpt.com/share/6962a2c8-0f50-800f-8bfb-2efd16adaf40
2026-01-10 20:04:33 +01:00
kevinveenbirkenbach 9d95eea3b0 fix(inventory): pin ansible_python_interpreter for localhost to avoid auto-discovery issues
Ensure that localhost inventories always define ansible_python_interpreter when using
ansible_connection=local. This prevents Ansible from falling back to interpreter
auto-discovery (interpreter_python=auto_silent), which could select unexpected Python
binaries inside containers (e.g. /usr/sbin/python3.13) and break module execution.

The interpreter is now pinned to $PYTHON if set, otherwise to sys.executable.

https://chatgpt.com/share/69628603-3e18-800f-a6f9-2b367728a9ff
2026-01-10 18:01:20 +01:00
kevinveenbirkenbach 1ca0fd05b0 fix(deploy-test): capture run_after resolver output for dependency resolution
Ensure stdout is captured when calling run_after_resolution inside the infinito
container so declared galaxy_info.run_after deps (e.g. matomo, mailu) are
correctly detected and added to deploy_ids.

https://chatgpt.com/share/69617fbb-5af4-800f-848d-edeb2c1125a8
2026-01-09 23:22:41 +01:00
kevinveenbirkenbach 4146535404 Adapted path 2026-01-09 22:35:06 +01:00
kevinveenbirkenbach 2d25928333 test(deploy): simplify runner by removing lifecycle gating and using include-based inventory
- remove TESTED_LIFECYCLES handling from runner
- build CI inventory via --include instead of large exclude lists
- deploy app together with its run_after dependencies

https://chatgpt.com/share/69616d7d-4af0-800f-8c68-c4ed78f0fbf7
2026-01-09 22:04:50 +01:00
kevinveenbirkenbach 38fcbbc451 fix(deploy-test): do not let lifecycle blacklist override explicit run_after deps
Ensure explicitly included applications (run_after dependencies) are not excluded
again by lifecycle blacklist filtering. Also log lifecycle hits for easier CI
debugging.

https://chatgpt.com/share/6960fe98-683c-800f-9419-0078dc67f195
2026-01-09 14:11:31 +01:00
kevinveenbirkenbach 4390e3e847 Changed to fail fast strategy for workstations to improve test time 2026-01-09 01:03:42 +01:00
kevinveenbirkenbach d7f13d30c6 fix(ci): wait for systemd readiness after compose up
Add a systemd readiness gate to the CI compose wrapper to avoid a race
where systemctl is called before DBus/systemd is fully ready.
This prevents sporadic failures like:
"System has not been booted with systemd as init system (PID 1)".

No Dockerfile or Ansible role changes required.

https://chatgpt.com/share/696045b5-bd08-800f-acd9-e3a4e2e23e52
2026-01-09 01:02:54 +01:00
kevinveenbirkenbach f0e777dca1 refactor(ci-deploy): run run_after resolution inside compose container
Resolve run_after dependencies via Compose.exec inside the infinito
CI container instead of invoking the resolver on the host.

This ensures:
- consistent runtime environment (PYTHONPATH, installed code)
- correct resolution when local and container sources differ
- alignment with other compose-based CI execution paths

https://chatgpt.com/share/69603903-d158-800f-815d-c506e3a3a44a
2026-01-09 00:08:39 +01:00
kevinveenbirkenbach c88419c487 fix(deploy-test): do not swallow exit code when keeping stack on failure
Avoid returning from finally in the test runner, which previously
overrode non-zero exit codes when --keep-stack-on-failure was set.
The compose stack is now conditionally kept without affecting the
actual test result.

https://chatgpt.com/share/696032fd-57d8-800f-bcc8-a0edf7b829a4
2026-01-08 23:43:05 +01:00
kevinveenbirkenbach d99c7e49a5 fix(ci): reintroduce lifecycle gating in deploy test runner
- Restore lifecycle-based role exclusion via cli.meta.roles.lifecycle_filter
- Make tested lifecycles configurable through TESTED_LIFECYCLES env
- Apply lifecycle gating consistently for workstation and server tests
- Fix minor bug in invokable parsing (avoid shadowing built-in list)
- Improve logging with lifecycle exclusion details

https://chatgpt.com/share/695fbcf4-2148-800f-8685-56ee73a9bae0
2026-01-08 15:19:26 +01:00
kevinveenbirkenbach 3135d2377a Solved missing imports 2026-01-08 14:51:02 +01:00
kevinveenbirkenbach 5a9f1f1dfb feat(deploy): dump full inventory on invalid application_id
- Print full raw inventory file when application_id validation fails
- Fix invalid reason detection to use in_roles / in_inventory flags
- Simplify CI debugging for per-app dedicated deploy tests

https://chatgpt.com/share/695fb5d2-801c-800f-a1c8-06c2bcace450
2026-01-08 14:48:45 +01:00
kevinveenbirkenbach b30335903f Adapted default file 2026-01-08 14:21:01 +01:00
kevinveenbirkenbach d6b0fb3be7 refactor(ci): run server deploy per app across all distros in single job
* Avoid GitHub Actions matrix job limits by looping distros inside the per-app job
* Simplify discover output emission and pin checkout/upload-artifact to stable versions
* Tidy long log lines and minor formatting in deploy/test helpers

https://chatgpt.com/share/695fa9b9-4460-800f-bcc3-d566b83649b2
2026-01-08 13:57:22 +01:00
kevinveenbirkenbach dba121ecf4 refactor(ci): move deploy test runner from bash to python orchestrator
* Add `cli.deploy.test` module to orchestrate compose-based CI deploy tests
* Support server per-app runs (including run_after dependencies) and workstation selection
* Add optional image build flags (--no-cache, --missing) and stack retention on failure
* Simplify `scripts/tests/deploy.sh` to a thin wrapper that forwards args to the Python runner

https://chatgpt.com/share/695f9e22-6e9c-800f-9a50-65f553619af2
2026-01-08 13:07:57 +01:00
kevinveenbirkenbach 83985163e2 feat(cli): add run_after resolution command with cycle-safe traversal
- Introduce CLI tool to resolve galaxy_info.run_after dependencies transitively
- Detect and report circular run_after dependencies explicitly
- Validate role existence and run_after schema strictly
- Provide whitespace-separated output for easy shell usage
- Add comprehensive unittest coverage including CLI behavior and error cases

https://chatgpt.com/share/695f8e09-54f4-800f-9f5f-ed9c2fdd664b
2026-01-08 11:59:17 +01:00
kevinveenbirkenbach 8d3e8d0b73 feat(email): unify SYSTEM_EMAIL_* vars and improve msmtp/postfix local delivery
- Replace DEFAULT_SYSTEM_EMAIL merge with flat SYSTEM_EMAIL_* variables
- Update templates across apps to new variable names
- Fix msmtp health check to use -t and local username fallback
- Improve postfix localhost config (localdomain, mydestination, timeouts)
- Extend sys-hostname to ensure /etc/hosts mapping once

https://chatgpt.com/share/695e8f39-0694-800f-b91c-a74f4b78d7c8
2026-01-07 17:52:04 +01:00
kevinveenbirkenbach 29483a6415 cli: add lifecycle_filter command and fix help forwarding
- Introduce meta roles lifecycle_filter CLI to select roles by lifecycle
- Support whitelist/blacklist modes, selection filtering, and auto-detected roles dir
- Fix blacklist handling for roles with missing lifecycle
- Forward --help correctly to subcommand argparse
- Add unit tests for lifecycle filtering logic

https://chatgpt.com/share/695d09d2-c864-800f-9e23-b9db0c6f3537
2026-01-06 14:10:33 +01:00
kevinveenbirkenbach 0f7add0f1a Unify Mailu token handling via users[*].tokens and clean up inventory validation
- Migrate Mailu credentials from legacy mailu_token to tokens['web-app-mailu']
- Update all affected application templates to use the new token structure
- Adjust inventory validation to no longer special-case mailu_token
- Minor typing and formatting cleanups in inventory creation and tests

https://chatgpt.com/share/69554915-87c8-800f-afd5-12118dbb1107
2025-12-31 17:02:20 +01:00
kevinveenbirkenbach 533a1235fe Implement SPOT-based authorized_keys management for users
* Move SSH key handling to users.<name>.authorized_keys (list-only) and write keys via shared user role
* Enforce at least one SSH key for administrator user
* Wrap backup user keys with restricted command wrapper before writing authorized_keys
* Remove legacy inventory file copy and CLI flag/function for administrator authorized_keys
* Extend user setup generator to include authorized_keys field
* Rename administrator scripts path to root scripts path and update dependent variables
* Update inventory sample and docs to reflect new SSH key workflow

https://chatgpt.com/share/695510b8-23f4-800f-b5db-877b07a1469b
2025-12-31 13:01:45 +01:00
kevinveenbirkenbach ea04035e12 feat(config): rename SSL_ENABLED to TLS_ENABLED across stack
* Replace global `SSL_ENABLED` flag with `TLS_ENABLED`
* Update web protocol/ports (http/https, ws/wss) to use `TLS_ENABLED`
* Switch cert-related roles and nginx TLS header gating to `TLS_ENABLED`
* Update app env templates (BookWyrm, Confluence, Jira, Magento, OpenProject, WordPress, etc.) to use `TLS_ENABLED`
* Adjust inventory generator to write `TLS_ENABLED` and update CLI help text accordingly
* Update unit tests for inventory vars overrides to expect `TLS_ENABLED`

https://chatgpt.com/share/695306ef-a798-800f-b118-e07b525e1811
2025-12-29 23:55:20 +01:00
kevinveenbirkenbach 5175aefaf8 Renamed desktop to workstation 2025-12-29 22:44:20 +01:00
kevinveenbirkenbach 1e5ebbb05f feat(inventory): add --vars-file support for YAML-based host_vars overrides
- add --vars-file CLI option to load host_vars overrides from YAML
- merge vars-file overrides before JSON --vars (JSON takes precedence)
- implement deep-merge logic for YAML vars files
- update CI deploy script to use vars file instead of inline JSON
- add unit test for apply_vars_overrides_from_file
- remove obsolete MASK_CREDENTIALS_IN_LOGS TODO entry

https://chatgpt.com/share/6952e51d-6c50-800f-8764-31dc8c609cdb
2025-12-29 21:31:05 +01:00
kevinveenbirkenbach bf4af7c823 Renamed domain variables 2025-12-29 20:37:47 +01:00
kevinveenbirkenbach a9e1a84b71 feat(ci): run Docker-in-Docker in deploy container via fuse-overlayfs
* Remove host Docker socket (/var/run/docker.sock) and DOCKER_HOST wiring for CI container
* Drop /tmp/gh-action bind usage in CI container + compose runner
* Switch OpenResty volume sources to container-local absolute paths (/var/www, /etc/nginx)
* Ensure docker engine installation also runs when DOCKER_IN_CONTAINER is true
* Add container-only fuse-overlayfs setup and write /etc/docker/daemon.json storage-driver config

https://chatgpt.com/share/69528062-ff74-800f-ae11-a658b5917307
2025-12-29 17:47:03 +01:00
kevinveenbirkenbach 8dbe1d66c5 refactor(user-backup): rename sys-bkp-provider-user role and clean up SSH handling
https://chatgpt.com/share/69518670-34b0-800f-84f9-70d001b1e030
2025-12-28 20:35:07 +01:00
kevinveenbirkenbach 5a74f80e85 Executed ruff 2025-12-28 15:23:07 +01:00
kevinveenbirkenbach f3714b00e7 feat(users,token-store): add user token defaults and unify token variable naming
- add `tokens` support to generated default_users
- normalize sys-token-store output variable to `sys_token_store_token`
- align Matomo role to new token-store interface
- reorder Matomo task files for clearer execution flow

https://chatgpt.com/share/69513d00-9a7c-800f-8b1f-5c31e0e21b75
2025-12-28 15:21:35 +01:00
kevinveenbirkenbach 2b5ff60a6f infra(docker): drop host networking and normalize port binding logic
This change removes --network=host usage, introduces DOCKER_IN_CONTAINER
for reliable environment detection, and replaces hardcoded localhost
bindings with an explicit DOCKER_BIND_HOST abstraction to avoid nftables
issues in CI/DiD environments.

https://chatgpt.com/share/69497616-7b00-800f-8fa3-ddc6eb9c5e98
2025-12-22 17:46:08 +01:00
kevinveenbirkenbach ffd071c64e ci(openresty): share gh-action runtime and scope nginx/www volumes per CI run
- Mount /tmp/gh-action into the CI container so Docker can access run-scoped paths
- Scope OpenResty www and nginx volumes to GITHUB_RUN_ID when running in container
- Avoid leaking host /etc/nginx and /var/www into CI runs
- Remove redundant preflight

https://chatgpt.com/share/694918d0-53b4-800f-93a7-ec82946f3e24
2025-12-22 11:09:12 +01:00
kevinveenbirkenbach 1fdf5ae169 fix(openresty): make restart handler safe with include_tasks and assertions
- Move OpenResty restart logic into included handler tasks
- Use block/rescue to dump container logs on failure
- Tolerate restarting state during pre-flight validation
- Wait up to 2 minutes for container to reach running state
- Add explicit change detection for docker restart
- Enable assert mode in container deploy CLI

https://chatgpt.com/share/6948fa05-1970-800f-a010-59c8c3588713
2025-12-22 09:40:08 +01:00
kevinveenbirkenbach d9824bb506 test(deploy): mock backup authorized_keys in CI inventory
Create a dummy authorized_keys file inside the CI container inventory
after inventory generation and before ansible-playbook execution.

This prevents sys-bkp-provider-user from failing on missing file lookups
while keeping the role logic unchanged and the deploy tests deterministic.
2025-12-21 23:39:20 +01:00
kevinveenbirkenbach d2e88b6a8c feat(cli): eliminate runpy warnings for inventory + container entrypoints
* Remove legacy **main** re-export wrappers to avoid `runpy` "already in sys.modules" warnings
* Introduce `cli.deploy.container.command` as the stable implementation module
* Reduce `cli.deploy.container.__main__` to a minimal entrypoint delegating to `command.main`
* Re-export container helpers from package `__init__` without importing `__main__`
* Update unit tests to import/patch the new command module and keep CLI dispatch stable

https://chatgpt.com/share/69484a22-60dc-800f-a23d-068d7c45fe37
2025-12-21 20:27:26 +01:00
kevinveenbirkenbachandCopilot Autofix powered by AI d0a2890a08 Potential fix for code scanning alert no. 125: Use of the return value of a procedure
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-12-21 15:51:36 +01:00
kevinveenbirkenbach 1fe0a420fa fix(cli): avoid runpy warnings by moving logic into command modules
Stop importing __main__ from package __init__.py to prevent runpy RuntimeWarning when using `python -m`.

- Move CLI logic into command.py modules
- Keep __main__.py as thin wrappers
- Update unit tests to import command modules directly

https://chatgpt.com/share/6947fb8e-550c-800f-8323-e36f92cb5bba
2025-12-21 14:52:09 +01:00
kevinveenbirkenbach f9573460e9 fix(deploy): force local CLI build for invokable resolution
- Drop legacy compatibility wrapper in cli.meta.applications.invokable
- Resolve repository root explicitly via pathlib
- Ensure filter_plugins are importable outside Ansible
- Run invokable discovery with INSTALL_LOCAL_BUILD=1 in deploy tests
- Make deploy.sh fail fast when invokable resolution is unavailable
- Normalize sorting and exclusion logic (drv-* roles, locale-safe sort)
2025-12-21 14:27:21 +01:00
kevinveenbirkenbach 79785ea531 Merge branch 'master' of github.com:kevinveenbirkenbach/infinito-nexus 2025-12-21 11:53:55 +01:00
kevinveenbirkenbach ce19fed69a Increased verbosity 2025-12-21 11:52:04 +01:00
kevinveenbirkenbachandCopilot Autofix powered by AI d3020ce1a4 Potential fix for code scanning alert no. 120: Empty except
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-12-21 11:42:31 +01:00