Commit Graph
257 Commits
Author SHA1 Message Date
kevinveenbirkenbach c92aff464b feat(ci): add min_storage validation and CI filtering with warnings
- Add sufficient_storage CLI for validating docker.services.<entity>.min_storage
- Introduce role_resource_validation utility with human-friendly size parsing
- Filter CI app discovery by available runner storage (default: 12GB)
- Emit GitHub Actions warnings for:
  - missing min_storage metadata (treated as 0GB)
  - roles exceeding available runner storage
- Add BigBlueButton min_storage metadata (50GB)
- Add unit tests for validation logic and CLI behavior
- Add humanfriendly dependency for size parsing

https://chatgpt.com/share/696e097b-1038-800f-b8fc-23e9dadfa021
2026-01-19 11:37:28 +01:00
kevinveenbirkenbach daab131b71 ci: auto-detect storage constraints and enable aggressive docker cleanup
- add storage detection utility for CI (DockerRootDir free space check)
- automatically set STORAGE_CONSTRAINED in cli.deploy.test when disk < 100 GiB
- forward STORAGE_CONSTRAINED into generated CI inventory via --vars
- introduce global STORAGE_CONSTRAINED default in group_vars
- run aggressive docker prune handler only in storage constrained mode
- mark CI inventory environment as development

https://chatgpt.com/share/696d6a11-2d90-800f-83e8-a30f6229e2e0
2026-01-19 00:17:31 +01:00
kevinveenbirkenbach 72929e83fc feat(shopware,credentials): add strong OpenSearch admin password generation
- add strong_password generator with special characters
- introduce opensearch_initial_admin_password credential
- wire OpenSearch admin password via env template
- decouple OpenSearch admin password from Shopware admin user
- minor formatting cleanup in service resolver
2026-01-18 23:33:19 +01:00
kevinveenbirkenbach 160a3a3233 feat(services): resolve shared onlyoffice provider role
- add onlyoffice enabled+shared → web-svc-onlyoffice service resolution
- enable shared onlyoffice for Nextcloud
- fix missing trailing newlines

https://chatgpt.com/share/696d54a1-9dbc-800f-8507-088bad56bc48
2026-01-18 22:50:45 +01:00
kevinveenbirkenbach e11dd60714 feat(services): resolve coturn as shared provider and refactor BBB coturn handling
- add coturn enabled+shared → web-svc-coturn service resolution
- refactor BigBlueButton coturn logic to distinguish local vs shared coturn
- introduce BBB_COTURN_LOCAL_ENABLED flag
- split TURN secrets into local and shared variants
- update dependencies, env and compose templates accordingly
- enable shared coturn in BigBlueButton and Nextcloud configs

https://chatgpt.com/share/696d54a1-9dbc-800f-8507-088bad56bc48
2026-01-18 22:46:02 +01:00
kevinveenbirkenbach 7a1fb02554 test(cli): add unit tests for combined + services resolution (services edges, desktop, cycles)
- add role_config_path support and services edges in combined resolver/tree
- add services resolver mapping with desktop enabled rule (non-shared)
- add unittest coverage for repo_paths, yaml parsing, services mapping/BFS, and combined tree cycles
- update tests to reflect new stable path expectations and tree output

https://chatgpt.com/share/696cb80f-e8f4-800f-b062-e47985b4b074
2026-01-18 11:37:41 +01:00
kevinveenbirkenbach cf216014e3 refactor(cli): introduce combined role resolution and reorganize meta resolvers
- add cli.meta.applications.resolution.combined to resolve run_after + app-only dependencies
- add optional --tree output for combined resolution
- add cli.meta.applications.resolution.dependencies (app-only) resolver
- move run_after resolver to cli.meta.applications.resolution.run_after
- update test deploy dependency resolver to use combined module
- add unit tests for combined/dependencies/run_after resolvers
- minor cleanup: remove stray header comment, fix newline in vars, tidy YAML formatting

https://chatgpt.com/share/696ca531-9644-800f-98c0-bae58c0aff41
2026-01-18 10:17:26 +01:00
kevinveenbirkenbach fa422ed7fe cli: allow comma-separated application IDs for --id flag
- Support space-separated, comma-separated and mixed --id values
- Normalize and deduplicate application IDs before validation
- Fix Makefile to pass APP without quoting
- Add unittest coverage for ID normalization

https://chatgpt.com/share/696be0c2-816c-800f-8b07-3c5cd0304ac5
2026-01-17 20:19:13 +01:00
kevinveenbirkenbach 357a9bda83 fix(users): normalize reserved flag handling and align CLI/tests
- Always emit reserved flag in generated users (default: false)
- Do not override existing user definitions via --reserved-usernames
- Normalize description default to empty string
- Align unit tests with new reserved semantics
- Fix YAML quoting and trailing newline issues
- Clean up minor formatting and config inconsistencies

https://chatgpt.com/share/696bb761-0048-800f-9cf7-aebd3fda75be
2026-01-17 17:22:50 +01:00
kevinveenbirkenbach fd983bc69c refactor(ci,network): make DNS, IP and domain configuration fully env-driven
- Introduce env.ci as single source of truth for CI networking
- Make docker-compose, CoreDNS and Docker daemon fully env-driven
- Render CoreDNS Corefile from template via envsubst
- Replace hardcoded IPs/domains with environment variables
- Integrate env-based networking into Ansible defaults
- Remove static Corefile and daemon.json

This makes DNS and networking deterministic and reproducible across
host, CoreDNS, infinito runner and Docker-in-Docker.

https://chatgpt.com/share/696a3dda-5c6c-800f-8a16-47c0c604e1e3
2026-01-16 14:30:14 +01:00
kevinveenbirkenbach a06f395640 Replaced -i with --id 2026-01-16 00:47:34 +01:00
kevinveenbirkenbach 505af3e2ad cli: make --log require explicit directory and create parents automatically
- Change --log flag to require explicit log directory (--log <LOG_DIR>)
- Automatically create log directory with parents=True
- Harden permissions best-effort (0700)
- Update CLI help text accordingly
- Adjust Makefile test targets to pass explicit log directory
- Ignore logs/ via .gitignore and .dockerignore
- Refactor flag parsing to correctly detect deploy with --log
- Add dedicated logging unit tests
- Update existing tests for new log semantics
- Remove deprecated -i alias for --id in deploy command

https://chatgpt.com/share/6969680a-dbf8-800f-aafb-3d32d6212ba0
2026-01-15 23:19:31 +01:00
kevinveenbirkenbach e782f41d8e cli(deploy): extend --help with ansible-playbook help and passthrough usage
Add a combined help output that includes:
- Wrapper CLI help
- Full `ansible-playbook --help`
- Explanation and examples for native Ansible flag passthrough

This makes it clear how to use --tags, --check, -e, --forks, and other
ansible-playbook options directly via the infinito deploy wrapper.

https://chatgpt.com/share/69694e02-de58-800f-a9ea-2fb788ff5077
2026-01-15 21:28:40 +01:00
kevinveenbirkenbach ea63818216 refactor(cli): simplify dedicated deploy wrapper and add ansible passthrough support
- Remove unused cli_root and logs handling from dedicated deploy wrapper
- Add parse_known_args() to forward native ansible-playbook flags
- Append passthrough args at the end for proper override behavior
- Clean up MODE_LOGS usage
- Update runner signature and cleanup phase logic
- Adapt unit tests to new wrapper contract
- Add unit test for ansible flag passthrough in command.main()

https://chatgpt.com/share/69694ad5-453c-800f-8047-cf210f60460f
2026-01-15 21:15:06 +01:00
kevinveenbirkenbach bab0bceb1d feat(local-dns): add dnsmasq-based infinito.example wildcard + set CI primary domain
https://chatgpt.com/share/6969261a-8d34-800f-9a61-725e90544dea
2026-01-15 18:38:15 +01:00
kevinveenbirkenbach 49df004a5f General py optimations 2026-01-15 05:02:55 +01:00
kevinveenbirkenbach ef4a6611d7 ci(deploy): log effective compose env without leaking secrets
Print selected INFINITO_* and GITHUB_SHA variables before starting the CI
compose stack and only indicate whether NIX_CONFIG is set, avoiding secret
exposure while making the effective image/tag visible in logs.

https://chatgpt.com/share/69684776-86c0-800f-b089-7e549a83e87f
2026-01-15 02:48:29 +01:00
kevinveenbirkenbach 334c4b70c7 fix: stabilize rapid deploy env, improve entry.sh debugging and harden contact config
- Fix Makefile rapid deploy target to properly export INFINITO_DISTRO
- Capture stdout/stderr from entry.sh init and raise explicit error on failure
- Add fallback contact addresses for demo environments (email, matrix, mastodon, etc.)
- Quote mailto URLs and identifiers to ensure valid YAML output
- Replace placeholder legal contact with demo user

This prevents YAML parser errors (e.g. `url: mailto:`), improves CI diagnostics,
and ensures desktop contact menu is always rendered with valid defaults.

https://chatgpt.com/share/69679c78-f338-800f-be86-54b3195d19b7
2026-01-14 14:38:42 +01:00
kevinveenbirkenbach 3f667aadda ci: switch CI images to latest tag and skip compose build in deploy
- Build CI images workflow now always publishes :latest instead of ci-<sha>
- Removed obsolete tag input and CI tag computation logic
- Deploy in real GitHub Actions now pulls ghcr.io/.../infinito-<distro>:latest
- Skip docker compose build when INFINITO_NO_BUILD=1
- Respect INFINITO_IMAGE for --missing image existence check

This ensures deploy jobs use the already published production image and never rebuild in CI.

https://chatgpt.com/share/69678e2c-7d7c-800f-b422-15e98c8e4b6e
2026-01-14 13:37:54 +01:00
kevinveenbirkenbach 4a008d8b97 ci: centralize INSTALL_LOCAL_BUILD handling across Makefile, compose and runner
- Control INSTALL_LOCAL_BUILD centrally via Makefile depending on CI context:
  - real GitHub Actions runner: disable local rebuild
  - act and local runs: enable local rebuild
- Pass INSTALL_LOCAL_BUILD through docker-compose using environment expansion
- Remove hardcoded INSTALL_LOCAL_BUILD=1 from CI container runner
- Keep Dockerfile local-build during image build for CI images

This ensures consistent local/CI behavior and avoids unnecessary rebuilds on real CI runners.

https://chatgpt.com/share/6967876f-e400-800f-90ec-fe870e565bf3
2026-01-14 13:09:13 +01:00
kevinveenbirkenbach 517ccf44c0 ci: replace rest deploy mode with universal and add universal stage
- Rename CI workflow and MODE from "rest" to "universal"
- Extend CLI and test runner to support universal deploy type
- Add tasks/stages/02_universal.yml for universal host type
- Update discover logic to use universal app selection
- Remove legacy rest->server mapping and obsolete select helper

https://chatgpt.com/share/6967625e-81c8-800f-8592-5d5b8edec1e2
2026-01-14 10:31:06 +01:00
kevinveenbirkenbach 638d9f2dfa Docker optimations 2026-01-14 00:37:48 +01:00
kevinveenbirkenbach 3cf7d20d86 feat(test-deploy): add debug toggle and rapid deploy target
- add --debug / --no-debug flag to deploy test CLI and runner
- make Ansible debug mode optional instead of always enabled
- extend deploy.sh with debug flag forwarding
- add make target test-deploy-rapid for fast single-app deploys

https://chatgpt.com/share/6966c80c-661c-800f-8fb9-769659da290d
2026-01-13 23:32:37 +01:00
kevinveenbirkenbach 585fcb4b53 Used correct relative path 2026-01-13 21:32:25 +01:00
kevinveenbirkenbach 6b7b67681e fix(ci): unblock systemd startup and stabilize Docker healthcheck
Ensure systemd is started immediately as PID 1 in the infinito container
to avoid blocking on pkgmgr during boot. Increase compose runner timeout
to match healthcheck window and make healthcheck script executable in the
image.

This fixes containers getting stuck in "starting" state and makes CI
startup deterministic.

https://chatgpt.com/share/696683a2-9350-800f-a0cf-587d4013cb29
2026-01-13 19:02:43 +01:00
kevinveenbirkenbach 8c82db3556 feat(ci): add Docker healthcheck and wait for healthy container in compose runner
Add a systemd-based Docker HEALTHCHECK that verifies:
- systemd readiness
- infinito CLI availability and executability
- infinito CLI smoke test (--help)

Update CI compose runner to wait for Docker health status instead of
manual systemd polling. The runner now:
- resolves the infinito container via docker compose
- waits for healthy state
- dumps journalctl and docker logs on timeout

This removes boot race conditions and makes container readiness deterministic.

https://chatgpt.com/share/696683a2-9350-800f-a0cf-587d4013cb29
2026-01-13 18:40:33 +01:00
kevinveenbirkenbach 75de900549 test(compose): dump last 200 log lines when systemd wait times out
Extend wait_for_systemd_ready() to automatically print the last 200 log
lines on timeout for easier CI debugging.

- Adds journalctl output (preferred for systemd containers)
- Adds docker logs fallback
- Improves timeout error diagnostics

https://chatgpt.com/share/69667eca-088c-800f-af12-5682d0038351
2026-01-13 18:19:37 +01:00
kevinveenbirkenbach 673dab59d2 ci(deploy): prevent docker compose builds in GitHub Actions
Switch CI detection to GITHUB_ACTIONS and disable compose builds via
INFINITO_NO_BUILD=1. The deploy workflows now rely on pulling the prebuilt
CI image instead of triggering local Docker builds.

- Remove legacy UP_BUILD handling
- Enable --no-build in compose when running in real CI
- Keep local/act builds working as before

https://chatgpt.com/share/69666e0b-f08c-800f-93fd-611eced4b100
2026-01-13 17:08:34 +01:00
kevinveenbirkenbach dac111ccb7 ci(deploy): simplify per-app deploy workflow and Makefile act targets
- Rename _deploy-matrix.yml -> _deploy-per-app.yml and drop unused inputs
  (tested_lifecycles, only_distro, final_exclude_re).
- Keep one runner per app (matrix.app) and run all distros inside that job.
- Makefile: run deploy tests through act workflow_dispatch (no serial script).
- Add scripts/ci/discover.sh and scripts/ci/deploy-one-app.sh as small SRP helpers.
- Update reusable workflow callers to use the renamed file.

https://chatgpt.com/share/69659ff3-1414-800f-8652-5298fcf55398
2026-01-13 03:41:25 +01:00
kevinveenbirkenbach 3329fbdfd5 ci(act): unify deploy matrix + force serial execution with max-parallel
- Add reusable _deploy-matrix workflow with discovery + per-app deploy across distros
- Enforce serial execution via strategy.max-parallel=1 and act --concurrent-jobs 1
- Support filters: mode (server|workstation|rest), only_app, only_distro, include/exclude regex
- Refactor test runner: remove discovery, require --app, deploy single app with deps only
- Generalize discover script to discover-apps.sh with INCLUDE_RE/EXCLUDE_RE + lifecycle gating
- Update Makefile targets to drive workflow_dispatch inputs for local act runs

https://chatgpt.com/share/69659ff3-1414-800f-8652-5298fcf55398
2026-01-13 03:06:56 +01:00
kevinveenbirkenbach 8bf2b4c1e1 Ruff 2026-01-12 17:50:43 +01:00
kevinveenbirkenbach 198a2fe3cf fix(ansible): enforce system python interpreter and install python-ldap via OS packages
- Stop auto-injecting ansible_python_interpreter from PYTHON env for localhost
- Set ansible_python_interpreter globally to /usr/bin/python3
- Add interpreter to sample inventory
- Remove role-local interpreter overrides
- Switch python-ldap installation to distro package (python-ldap / python3-ldap)
- Drop pip-based python-ldap installation to avoid venv/libldap mismatches

https://chatgpt.com/share/6964335a-29cc-800f-ae65-6a35c7cd2559
2026-01-12 14:48:53 +01:00
kevinveenbirkenbach e5f8067db2 fix(deploy-test): apply lifecycle gating to workstation and server deploy tests
Apply TESTED_LIFECYCLES filtering directly in the deploy test runner so roles
with unsupported lifecycles (e.g. pre-alpha) are excluded consistently for
both workstation and server workflows.

This prevents roles like desk-copyq (lifecycle: pre-alpha) from being deployed
in CI unless explicitly enabled via TESTED_LIFECYCLES.

https://chatgpt.com/share/6963e316-0a50-800f-a895-8828269a0ca3
2026-01-11 18:51:09 +01:00
kevinveenbirkenbach 66d50e2dde feat(inventory): resolve shared providers transitively and support schema-less roles
- Allow credentials generator to run for roles without schema/main.yml so
  transitive config-based generation still works.
- Make InventoryManager resilient to missing YAML by defaulting to {} and
  only loading schema when schema/main.yml exists.
- Add recursive shared-provider include resolution based on docker.services
  config and apply provider schemas/special rules transitively.
- Require database_password injection only when database.enabled=true and shared=true.
- Update unit/integration tests to create required files for .exists() checks and
  validate transitive provider role credential generation.

https://chatgpt.com/share/69639d93-4c10-800f-bc6c-8e5b9fb75088
2026-01-11 13:54:37 +01:00
kevinveenbirkenbach e4baa6d7ab Ruff 2026-01-11 01:01:44 +01:00
kevinveenbirkenbach 39f7f8fbc7 feat(create-credentials): generate shared provider app credentials from role schemas
Extend credentials generator to include schemas for shared provider roles
(OpenLDAP, Keycloak, Matomo, and svc-db-<type>) based on docker.services.*.shared
flags in role config. Credentials are now emitted/added for all involved
applications while preserving existing inventory formatting, and CLI overrides
can target specific apps via applications.<app>.credentials.<key>.
2026-01-11 00:00:29 +01:00
kevinveenbirkenbach 2d7a904f2b ruff optimation 2026-01-10 20:06:53 +01:00
kevinveenbirkenbach c0b03bb95a refactor(up): route make up through python orchestrator and entry.sh init
- Replace direct docker compose up with python-based orchestrator (Compose.up)
- Centralize stack startup, systemd readiness wait, and init logic in Compose.up
- Run container init via scripts/docker/entry.sh instead of make target
- Remove duplicate systemd wait in deploy test runner
- Add cli.deploy.test.up entrypoint for make up integration

https://chatgpt.com/share/6962a2c8-0f50-800f-8bfb-2efd16adaf40
2026-01-10 20:04:33 +01:00
kevinveenbirkenbach 9d95eea3b0 fix(inventory): pin ansible_python_interpreter for localhost to avoid auto-discovery issues
Ensure that localhost inventories always define ansible_python_interpreter when using
ansible_connection=local. This prevents Ansible from falling back to interpreter
auto-discovery (interpreter_python=auto_silent), which could select unexpected Python
binaries inside containers (e.g. /usr/sbin/python3.13) and break module execution.

The interpreter is now pinned to $PYTHON if set, otherwise to sys.executable.

https://chatgpt.com/share/69628603-3e18-800f-a6f9-2b367728a9ff
2026-01-10 18:01:20 +01:00
kevinveenbirkenbach 1ca0fd05b0 fix(deploy-test): capture run_after resolver output for dependency resolution
Ensure stdout is captured when calling run_after_resolution inside the infinito
container so declared galaxy_info.run_after deps (e.g. matomo, mailu) are
correctly detected and added to deploy_ids.

https://chatgpt.com/share/69617fbb-5af4-800f-848d-edeb2c1125a8
2026-01-09 23:22:41 +01:00
kevinveenbirkenbach 4146535404 Adapted path 2026-01-09 22:35:06 +01:00
kevinveenbirkenbach 2d25928333 test(deploy): simplify runner by removing lifecycle gating and using include-based inventory
- remove TESTED_LIFECYCLES handling from runner
- build CI inventory via --include instead of large exclude lists
- deploy app together with its run_after dependencies

https://chatgpt.com/share/69616d7d-4af0-800f-8c68-c4ed78f0fbf7
2026-01-09 22:04:50 +01:00
kevinveenbirkenbach 38fcbbc451 fix(deploy-test): do not let lifecycle blacklist override explicit run_after deps
Ensure explicitly included applications (run_after dependencies) are not excluded
again by lifecycle blacklist filtering. Also log lifecycle hits for easier CI
debugging.

https://chatgpt.com/share/6960fe98-683c-800f-9419-0078dc67f195
2026-01-09 14:11:31 +01:00
kevinveenbirkenbach 4390e3e847 Changed to fail fast strategy for workstations to improve test time 2026-01-09 01:03:42 +01:00
kevinveenbirkenbach d7f13d30c6 fix(ci): wait for systemd readiness after compose up
Add a systemd readiness gate to the CI compose wrapper to avoid a race
where systemctl is called before DBus/systemd is fully ready.
This prevents sporadic failures like:
"System has not been booted with systemd as init system (PID 1)".

No Dockerfile or Ansible role changes required.

https://chatgpt.com/share/696045b5-bd08-800f-acd9-e3a4e2e23e52
2026-01-09 01:02:54 +01:00
kevinveenbirkenbach f0e777dca1 refactor(ci-deploy): run run_after resolution inside compose container
Resolve run_after dependencies via Compose.exec inside the infinito
CI container instead of invoking the resolver on the host.

This ensures:
- consistent runtime environment (PYTHONPATH, installed code)
- correct resolution when local and container sources differ
- alignment with other compose-based CI execution paths

https://chatgpt.com/share/69603903-d158-800f-815d-c506e3a3a44a
2026-01-09 00:08:39 +01:00
kevinveenbirkenbach c88419c487 fix(deploy-test): do not swallow exit code when keeping stack on failure
Avoid returning from finally in the test runner, which previously
overrode non-zero exit codes when --keep-stack-on-failure was set.
The compose stack is now conditionally kept without affecting the
actual test result.

https://chatgpt.com/share/696032fd-57d8-800f-bcc8-a0edf7b829a4
2026-01-08 23:43:05 +01:00
kevinveenbirkenbach d99c7e49a5 fix(ci): reintroduce lifecycle gating in deploy test runner
- Restore lifecycle-based role exclusion via cli.meta.roles.lifecycle_filter
- Make tested lifecycles configurable through TESTED_LIFECYCLES env
- Apply lifecycle gating consistently for workstation and server tests
- Fix minor bug in invokable parsing (avoid shadowing built-in list)
- Improve logging with lifecycle exclusion details

https://chatgpt.com/share/695fbcf4-2148-800f-8685-56ee73a9bae0
2026-01-08 15:19:26 +01:00
kevinveenbirkenbach 3135d2377a Solved missing imports 2026-01-08 14:51:02 +01:00
kevinveenbirkenbach 5a9f1f1dfb feat(deploy): dump full inventory on invalid application_id
- Print full raw inventory file when application_id validation fails
- Fix invalid reason detection to use in_roles / in_inventory flags
- Simplify CI debugging for per-app dedicated deploy tests

https://chatgpt.com/share/695fb5d2-801c-800f-a1c8-06c2bcace450
2026-01-08 14:48:45 +01:00