Commit Graph
918 Commits
Author SHA1 Message Date
kevinveenbirkenbach a565344ed5 Solved shellcheck 2026-01-21 00:25:35 +01:00
kevinveenbirkenbach f08993049c feat(inventory): add dynamic inventory resolver and propagate INVENTORY_DIR everywhere
- add scripts/inventory/resolve.sh to compute INVENTORY_DIR based on runtime context (act/github/local)
- resolve INVENTORY_DIR dynamically in Makefile via resolve.sh
- enforce INVENTORY_DIR usage across all local deploy/test scripts
- propagate INVENTORY_DIR into container and development CLI commands
- remove hardcoded /etc/inventories paths

https://chatgpt.com/share/69700d69-b270-800f-ad50-d9ff486378b9
2026-01-21 00:18:57 +01:00
kevinveenbirkenbach 0a4040d14d Lint 2026-01-20 22:00:49 +01:00
kevinveenbirkenbach d22dc4748b **refactor(cli): switch development dispatcher to argparse subcommands**
* Replace runpy-based nested CLI dispatch with argparse subcommand hub
* Auto-register development commands via `add_parser()` and execute `_handler`
* Fix `python -m cli.deploy.development <cmd>` resolution (e.g. `up`, `down`, `run`)

https://chatgpt.com/share/696fdea6-b208-800f-8b55-c4205c567aa2
2026-01-20 20:59:24 +01:00
kevinveenbirkenbach 50816243ed Solved linting hints 2026-01-20 20:50:50 +01:00
kevinveenbirkenbach d35e932e0c refactor(cli): switch development CLI to dynamic dispatcher and fix test paths
- Replace static argparse-based CLI in cli.deploy.development with dynamic
  module dispatcher compatible with nested command discovery.
- Ensure cli.deploy.development is executable via python -m and root dispatcher.
- Fix unit test path resolution to work locally and in CI.
- Make docker-* Makefile targets depend on install.
- Simplify test runner script and keep Nix flake config compatibility.

This aligns the CLI architecture with integration test discovery
(cli/**/__main__.py) and removes the need for manual subcommand registration.

https://chatgpt.com/share/696fdbbc-b6d8-800f-a8b5-067aa7e38706
2026-01-20 20:46:20 +01:00
kevinveenbirkenbach 4e0d09e773 refactor(dev-orchestrator): centralize image build and add up --when-down SPOT
- Introduce cli.deploy.development build subcommand as single build SPOT
- Remove legacy compose.build_infinito implementation
- Make up use build-missing logic via build handler
- Add --when-down flag to up (skip startup if stack already running)
- Simplify restart/run to reuse up handler
- Ensure tests use up --when-down to avoid stack interference
- Make test targets depend on build-missing

https://chatgpt.com/share/696fd3c9-99f0-800f-a0ab-33a91058419a
2026-01-20 20:13:02 +01:00
kevinveenbirkenbach bbea43f669 refactor(ci,tests): migrate discovery and local deploy to development compose stack
- Replace legacy ci scripts with meta build-test-matrix and resolve-invokable-roles
- Switch log tailing to scripts/meta/log-tail.sh
- Rename local DNS helpers to scripts/dns/*
- Run ci-deploy-discover via development docker-up
- Replace deploy-local-full with development compose orchestrator
- Ensure INFINITO_DISTRO is passed explicitly for local full deploy

https://chatgpt.com/share/696f9ccd-e4d8-800f-9c67-9c071188ecd0
https://chatgpt.com/share/696facf5-b2fc-800f-8a33-3352dd193c54
2026-01-20 17:27:09 +01:00
kevinveenbirkenbach f52a012829 feat(cli): replace test deploy runner with development CLI (up/down/stop/restart/run)
https://chatgpt.com/share/696f9ccd-e4d8-800f-9c67-9c071188ecd0
2026-01-20 16:18:31 +01:00
kevinveenbirkenbach 811501d85f refactor(ci): move compose down logic into dedicated down module with safe /mnt/docker cleanup
https://chatgpt.com/share/696f4386-fbbc-800f-bdde-f1e6c4145c84
2026-01-20 12:54:54 +01:00
kevinveenbirkenbach 5cfa73941d **fix(cli): remove roles_root argument and resolve roles path internally**
https://chatgpt.com/share/696e3d20-276c-800f-87a7-a7052989ffe4
2026-01-19 19:10:20 +01:00
kevinveenbirkenbach c92aff464b feat(ci): add min_storage validation and CI filtering with warnings
- Add sufficient_storage CLI for validating docker.services.<entity>.min_storage
- Introduce role_resource_validation utility with human-friendly size parsing
- Filter CI app discovery by available runner storage (default: 12GB)
- Emit GitHub Actions warnings for:
  - missing min_storage metadata (treated as 0GB)
  - roles exceeding available runner storage
- Add BigBlueButton min_storage metadata (50GB)
- Add unit tests for validation logic and CLI behavior
- Add humanfriendly dependency for size parsing

https://chatgpt.com/share/696e097b-1038-800f-b8fc-23e9dadfa021
2026-01-19 11:37:28 +01:00
kevinveenbirkenbach daab131b71 ci: auto-detect storage constraints and enable aggressive docker cleanup
- add storage detection utility for CI (DockerRootDir free space check)
- automatically set STORAGE_CONSTRAINED in cli.deploy.test when disk < 100 GiB
- forward STORAGE_CONSTRAINED into generated CI inventory via --vars
- introduce global STORAGE_CONSTRAINED default in group_vars
- run aggressive docker prune handler only in storage constrained mode
- mark CI inventory environment as development

https://chatgpt.com/share/696d6a11-2d90-800f-83e8-a30f6229e2e0
2026-01-19 00:17:31 +01:00
kevinveenbirkenbach 72929e83fc feat(shopware,credentials): add strong OpenSearch admin password generation
- add strong_password generator with special characters
- introduce opensearch_initial_admin_password credential
- wire OpenSearch admin password via env template
- decouple OpenSearch admin password from Shopware admin user
- minor formatting cleanup in service resolver
2026-01-18 23:33:19 +01:00
kevinveenbirkenbach 160a3a3233 feat(services): resolve shared onlyoffice provider role
- add onlyoffice enabled+shared → web-svc-onlyoffice service resolution
- enable shared onlyoffice for Nextcloud
- fix missing trailing newlines

https://chatgpt.com/share/696d54a1-9dbc-800f-8507-088bad56bc48
2026-01-18 22:50:45 +01:00
kevinveenbirkenbach e11dd60714 feat(services): resolve coturn as shared provider and refactor BBB coturn handling
- add coturn enabled+shared → web-svc-coturn service resolution
- refactor BigBlueButton coturn logic to distinguish local vs shared coturn
- introduce BBB_COTURN_LOCAL_ENABLED flag
- split TURN secrets into local and shared variants
- update dependencies, env and compose templates accordingly
- enable shared coturn in BigBlueButton and Nextcloud configs

https://chatgpt.com/share/696d54a1-9dbc-800f-8507-088bad56bc48
2026-01-18 22:46:02 +01:00
kevinveenbirkenbach 7a1fb02554 test(cli): add unit tests for combined + services resolution (services edges, desktop, cycles)
- add role_config_path support and services edges in combined resolver/tree
- add services resolver mapping with desktop enabled rule (non-shared)
- add unittest coverage for repo_paths, yaml parsing, services mapping/BFS, and combined tree cycles
- update tests to reflect new stable path expectations and tree output

https://chatgpt.com/share/696cb80f-e8f4-800f-b062-e47985b4b074
2026-01-18 11:37:41 +01:00
kevinveenbirkenbach cf216014e3 refactor(cli): introduce combined role resolution and reorganize meta resolvers
- add cli.meta.applications.resolution.combined to resolve run_after + app-only dependencies
- add optional --tree output for combined resolution
- add cli.meta.applications.resolution.dependencies (app-only) resolver
- move run_after resolver to cli.meta.applications.resolution.run_after
- update test deploy dependency resolver to use combined module
- add unit tests for combined/dependencies/run_after resolvers
- minor cleanup: remove stray header comment, fix newline in vars, tidy YAML formatting

https://chatgpt.com/share/696ca531-9644-800f-98c0-bae58c0aff41
2026-01-18 10:17:26 +01:00
kevinveenbirkenbach fa422ed7fe cli: allow comma-separated application IDs for --id flag
- Support space-separated, comma-separated and mixed --id values
- Normalize and deduplicate application IDs before validation
- Fix Makefile to pass APP without quoting
- Add unittest coverage for ID normalization

https://chatgpt.com/share/696be0c2-816c-800f-8b07-3c5cd0304ac5
2026-01-17 20:19:13 +01:00
kevinveenbirkenbach 357a9bda83 fix(users): normalize reserved flag handling and align CLI/tests
- Always emit reserved flag in generated users (default: false)
- Do not override existing user definitions via --reserved-usernames
- Normalize description default to empty string
- Align unit tests with new reserved semantics
- Fix YAML quoting and trailing newline issues
- Clean up minor formatting and config inconsistencies

https://chatgpt.com/share/696bb761-0048-800f-9cf7-aebd3fda75be
2026-01-17 17:22:50 +01:00
kevinveenbirkenbach fd983bc69c refactor(ci,network): make DNS, IP and domain configuration fully env-driven
- Introduce env.ci as single source of truth for CI networking
- Make docker-compose, CoreDNS and Docker daemon fully env-driven
- Render CoreDNS Corefile from template via envsubst
- Replace hardcoded IPs/domains with environment variables
- Integrate env-based networking into Ansible defaults
- Remove static Corefile and daemon.json

This makes DNS and networking deterministic and reproducible across
host, CoreDNS, infinito runner and Docker-in-Docker.

https://chatgpt.com/share/696a3dda-5c6c-800f-8a16-47c0c604e1e3
2026-01-16 14:30:14 +01:00
kevinveenbirkenbach a06f395640 Replaced -i with --id 2026-01-16 00:47:34 +01:00
kevinveenbirkenbach 505af3e2ad cli: make --log require explicit directory and create parents automatically
- Change --log flag to require explicit log directory (--log <LOG_DIR>)
- Automatically create log directory with parents=True
- Harden permissions best-effort (0700)
- Update CLI help text accordingly
- Adjust Makefile test targets to pass explicit log directory
- Ignore logs/ via .gitignore and .dockerignore
- Refactor flag parsing to correctly detect deploy with --log
- Add dedicated logging unit tests
- Update existing tests for new log semantics
- Remove deprecated -i alias for --id in deploy command

https://chatgpt.com/share/6969680a-dbf8-800f-aafb-3d32d6212ba0
2026-01-15 23:19:31 +01:00
kevinveenbirkenbach e782f41d8e cli(deploy): extend --help with ansible-playbook help and passthrough usage
Add a combined help output that includes:
- Wrapper CLI help
- Full `ansible-playbook --help`
- Explanation and examples for native Ansible flag passthrough

This makes it clear how to use --tags, --check, -e, --forks, and other
ansible-playbook options directly via the infinito deploy wrapper.

https://chatgpt.com/share/69694e02-de58-800f-a9ea-2fb788ff5077
2026-01-15 21:28:40 +01:00
kevinveenbirkenbach ea63818216 refactor(cli): simplify dedicated deploy wrapper and add ansible passthrough support
- Remove unused cli_root and logs handling from dedicated deploy wrapper
- Add parse_known_args() to forward native ansible-playbook flags
- Append passthrough args at the end for proper override behavior
- Clean up MODE_LOGS usage
- Update runner signature and cleanup phase logic
- Adapt unit tests to new wrapper contract
- Add unit test for ansible flag passthrough in command.main()

https://chatgpt.com/share/69694ad5-453c-800f-8047-cf210f60460f
2026-01-15 21:15:06 +01:00
kevinveenbirkenbach bab0bceb1d feat(local-dns): add dnsmasq-based infinito.example wildcard + set CI primary domain
https://chatgpt.com/share/6969261a-8d34-800f-9a61-725e90544dea
2026-01-15 18:38:15 +01:00
kevinveenbirkenbach 49df004a5f General py optimations 2026-01-15 05:02:55 +01:00
kevinveenbirkenbach ef4a6611d7 ci(deploy): log effective compose env without leaking secrets
Print selected INFINITO_* and GITHUB_SHA variables before starting the CI
compose stack and only indicate whether NIX_CONFIG is set, avoiding secret
exposure while making the effective image/tag visible in logs.

https://chatgpt.com/share/69684776-86c0-800f-b089-7e549a83e87f
2026-01-15 02:48:29 +01:00
kevinveenbirkenbach 334c4b70c7 fix: stabilize rapid deploy env, improve entry.sh debugging and harden contact config
- Fix Makefile rapid deploy target to properly export INFINITO_DISTRO
- Capture stdout/stderr from entry.sh init and raise explicit error on failure
- Add fallback contact addresses for demo environments (email, matrix, mastodon, etc.)
- Quote mailto URLs and identifiers to ensure valid YAML output
- Replace placeholder legal contact with demo user

This prevents YAML parser errors (e.g. `url: mailto:`), improves CI diagnostics,
and ensures desktop contact menu is always rendered with valid defaults.

https://chatgpt.com/share/69679c78-f338-800f-be86-54b3195d19b7
2026-01-14 14:38:42 +01:00
kevinveenbirkenbach 3f667aadda ci: switch CI images to latest tag and skip compose build in deploy
- Build CI images workflow now always publishes :latest instead of ci-<sha>
- Removed obsolete tag input and CI tag computation logic
- Deploy in real GitHub Actions now pulls ghcr.io/.../infinito-<distro>:latest
- Skip docker compose build when INFINITO_NO_BUILD=1
- Respect INFINITO_IMAGE for --missing image existence check

This ensures deploy jobs use the already published production image and never rebuild in CI.

https://chatgpt.com/share/69678e2c-7d7c-800f-b422-15e98c8e4b6e
2026-01-14 13:37:54 +01:00
kevinveenbirkenbach 4a008d8b97 ci: centralize INSTALL_LOCAL_BUILD handling across Makefile, compose and runner
- Control INSTALL_LOCAL_BUILD centrally via Makefile depending on CI context:
  - real GitHub Actions runner: disable local rebuild
  - act and local runs: enable local rebuild
- Pass INSTALL_LOCAL_BUILD through docker-compose using environment expansion
- Remove hardcoded INSTALL_LOCAL_BUILD=1 from CI container runner
- Keep Dockerfile local-build during image build for CI images

This ensures consistent local/CI behavior and avoids unnecessary rebuilds on real CI runners.

https://chatgpt.com/share/6967876f-e400-800f-90ec-fe870e565bf3
2026-01-14 13:09:13 +01:00
kevinveenbirkenbach 517ccf44c0 ci: replace rest deploy mode with universal and add universal stage
- Rename CI workflow and MODE from "rest" to "universal"
- Extend CLI and test runner to support universal deploy type
- Add tasks/stages/02_universal.yml for universal host type
- Update discover logic to use universal app selection
- Remove legacy rest->server mapping and obsolete select helper

https://chatgpt.com/share/6967625e-81c8-800f-8592-5d5b8edec1e2
2026-01-14 10:31:06 +01:00
kevinveenbirkenbach 638d9f2dfa Docker optimations 2026-01-14 00:37:48 +01:00
kevinveenbirkenbach 3cf7d20d86 feat(test-deploy): add debug toggle and rapid deploy target
- add --debug / --no-debug flag to deploy test CLI and runner
- make Ansible debug mode optional instead of always enabled
- extend deploy.sh with debug flag forwarding
- add make target test-deploy-rapid for fast single-app deploys

https://chatgpt.com/share/6966c80c-661c-800f-8fb9-769659da290d
2026-01-13 23:32:37 +01:00
kevinveenbirkenbach 585fcb4b53 Used correct relative path 2026-01-13 21:32:25 +01:00
kevinveenbirkenbach 6b7b67681e fix(ci): unblock systemd startup and stabilize Docker healthcheck
Ensure systemd is started immediately as PID 1 in the infinito container
to avoid blocking on pkgmgr during boot. Increase compose runner timeout
to match healthcheck window and make healthcheck script executable in the
image.

This fixes containers getting stuck in "starting" state and makes CI
startup deterministic.

https://chatgpt.com/share/696683a2-9350-800f-a0cf-587d4013cb29
2026-01-13 19:02:43 +01:00
kevinveenbirkenbach 8c82db3556 feat(ci): add Docker healthcheck and wait for healthy container in compose runner
Add a systemd-based Docker HEALTHCHECK that verifies:
- systemd readiness
- infinito CLI availability and executability
- infinito CLI smoke test (--help)

Update CI compose runner to wait for Docker health status instead of
manual systemd polling. The runner now:
- resolves the infinito container via docker compose
- waits for healthy state
- dumps journalctl and docker logs on timeout

This removes boot race conditions and makes container readiness deterministic.

https://chatgpt.com/share/696683a2-9350-800f-a0cf-587d4013cb29
2026-01-13 18:40:33 +01:00
kevinveenbirkenbach 75de900549 test(compose): dump last 200 log lines when systemd wait times out
Extend wait_for_systemd_ready() to automatically print the last 200 log
lines on timeout for easier CI debugging.

- Adds journalctl output (preferred for systemd containers)
- Adds docker logs fallback
- Improves timeout error diagnostics

https://chatgpt.com/share/69667eca-088c-800f-af12-5682d0038351
2026-01-13 18:19:37 +01:00
kevinveenbirkenbach 673dab59d2 ci(deploy): prevent docker compose builds in GitHub Actions
Switch CI detection to GITHUB_ACTIONS and disable compose builds via
INFINITO_NO_BUILD=1. The deploy workflows now rely on pulling the prebuilt
CI image instead of triggering local Docker builds.

- Remove legacy UP_BUILD handling
- Enable --no-build in compose when running in real CI
- Keep local/act builds working as before

https://chatgpt.com/share/69666e0b-f08c-800f-93fd-611eced4b100
2026-01-13 17:08:34 +01:00
kevinveenbirkenbach dac111ccb7 ci(deploy): simplify per-app deploy workflow and Makefile act targets
- Rename _deploy-matrix.yml -> _deploy-per-app.yml and drop unused inputs
  (tested_lifecycles, only_distro, final_exclude_re).
- Keep one runner per app (matrix.app) and run all distros inside that job.
- Makefile: run deploy tests through act workflow_dispatch (no serial script).
- Add scripts/ci/discover.sh and scripts/ci/deploy-one-app.sh as small SRP helpers.
- Update reusable workflow callers to use the renamed file.

https://chatgpt.com/share/69659ff3-1414-800f-8652-5298fcf55398
2026-01-13 03:41:25 +01:00
kevinveenbirkenbach 3329fbdfd5 ci(act): unify deploy matrix + force serial execution with max-parallel
- Add reusable _deploy-matrix workflow with discovery + per-app deploy across distros
- Enforce serial execution via strategy.max-parallel=1 and act --concurrent-jobs 1
- Support filters: mode (server|workstation|rest), only_app, only_distro, include/exclude regex
- Refactor test runner: remove discovery, require --app, deploy single app with deps only
- Generalize discover script to discover-apps.sh with INCLUDE_RE/EXCLUDE_RE + lifecycle gating
- Update Makefile targets to drive workflow_dispatch inputs for local act runs

https://chatgpt.com/share/69659ff3-1414-800f-8652-5298fcf55398
2026-01-13 03:06:56 +01:00
kevinveenbirkenbach 8bf2b4c1e1 Ruff 2026-01-12 17:50:43 +01:00
kevinveenbirkenbach 198a2fe3cf fix(ansible): enforce system python interpreter and install python-ldap via OS packages
- Stop auto-injecting ansible_python_interpreter from PYTHON env for localhost
- Set ansible_python_interpreter globally to /usr/bin/python3
- Add interpreter to sample inventory
- Remove role-local interpreter overrides
- Switch python-ldap installation to distro package (python-ldap / python3-ldap)
- Drop pip-based python-ldap installation to avoid venv/libldap mismatches

https://chatgpt.com/share/6964335a-29cc-800f-ae65-6a35c7cd2559
2026-01-12 14:48:53 +01:00
kevinveenbirkenbach e5f8067db2 fix(deploy-test): apply lifecycle gating to workstation and server deploy tests
Apply TESTED_LIFECYCLES filtering directly in the deploy test runner so roles
with unsupported lifecycles (e.g. pre-alpha) are excluded consistently for
both workstation and server workflows.

This prevents roles like desk-copyq (lifecycle: pre-alpha) from being deployed
in CI unless explicitly enabled via TESTED_LIFECYCLES.

https://chatgpt.com/share/6963e316-0a50-800f-a895-8828269a0ca3
2026-01-11 18:51:09 +01:00
kevinveenbirkenbach 66d50e2dde feat(inventory): resolve shared providers transitively and support schema-less roles
- Allow credentials generator to run for roles without schema/main.yml so
  transitive config-based generation still works.
- Make InventoryManager resilient to missing YAML by defaulting to {} and
  only loading schema when schema/main.yml exists.
- Add recursive shared-provider include resolution based on docker.services
  config and apply provider schemas/special rules transitively.
- Require database_password injection only when database.enabled=true and shared=true.
- Update unit/integration tests to create required files for .exists() checks and
  validate transitive provider role credential generation.

https://chatgpt.com/share/69639d93-4c10-800f-bc6c-8e5b9fb75088
2026-01-11 13:54:37 +01:00
kevinveenbirkenbach e4baa6d7ab Ruff 2026-01-11 01:01:44 +01:00
kevinveenbirkenbach 39f7f8fbc7 feat(create-credentials): generate shared provider app credentials from role schemas
Extend credentials generator to include schemas for shared provider roles
(OpenLDAP, Keycloak, Matomo, and svc-db-<type>) based on docker.services.*.shared
flags in role config. Credentials are now emitted/added for all involved
applications while preserving existing inventory formatting, and CLI overrides
can target specific apps via applications.<app>.credentials.<key>.
2026-01-11 00:00:29 +01:00
kevinveenbirkenbach 2d7a904f2b ruff optimation 2026-01-10 20:06:53 +01:00
kevinveenbirkenbach c0b03bb95a refactor(up): route make up through python orchestrator and entry.sh init
- Replace direct docker compose up with python-based orchestrator (Compose.up)
- Centralize stack startup, systemd readiness wait, and init logic in Compose.up
- Run container init via scripts/docker/entry.sh instead of make target
- Remove duplicate systemd wait in deploy test runner
- Add cli.deploy.test.up entrypoint for make up integration

https://chatgpt.com/share/6962a2c8-0f50-800f-8bfb-2efd16adaf40
2026-01-10 20:04:33 +01:00
kevinveenbirkenbach 9d95eea3b0 fix(inventory): pin ansible_python_interpreter for localhost to avoid auto-discovery issues
Ensure that localhost inventories always define ansible_python_interpreter when using
ansible_connection=local. This prevents Ansible from falling back to interpreter
auto-discovery (interpreter_python=auto_silent), which could select unexpected Python
binaries inside containers (e.g. /usr/sbin/python3.13) and break module execution.

The interpreter is now pinned to $PYTHON if set, otherwise to sys.executable.

https://chatgpt.com/share/69628603-3e18-800f-a6f9-2b367728a9ff
2026-01-10 18:01:20 +01:00