Commit Graph
115 Commits
Author SHA1 Message Date
kevinveenbirkenbach a89f74fac0 Included extra modul parameter 2026-01-21 16:02:18 +01:00
kevinveenbirkenbach 249103bb56 feat(ci): unify local act workflow with CI discovery logic and strict input resolver
- Use docker-up + ci-discover-output for local discover job (same as CI)
- Add strict input resolver (scripts/meta/resolve/input.sh) with enforced env contract
- Ensure ONLY_APP is always defined (may be empty, but never unset)
- Expose resolved inputs as workflow outputs for linear matrix deploy
- Enforce linear matrix execution via max-parallel: 1
- Align act helper scripts with linear matrix semantics

https://chatgpt.com/share/6970ba64-8c2c-800f-bb90-7edbe9168582
2026-01-21 12:37:03 +01:00
kevinveenbirkenbach 4cf54418ca Removed --missing 2026-01-21 03:34:06 +01:00
kevinveenbirkenbach 7bf8070ef7 fix(ci): remove unsupported --no-cache/--missing flags from distros.sh
The CI distros runner now only passes arguments supported by
cli.deploy.development run. This avoids argparse errors caused by
forwarding build-only flags that the orchestrator does not accept.

https://chatgpt.com/share/697030c4-f60c-800f-94b6-9128f87da57e
2026-01-21 02:49:48 +01:00
kevinveenbirkenbach 370938c71b fix(venv): use bootstrap python to create virtualenv
Avoid using the venv interpreter before it exists.
Create the venv with system python3 and only rely on venv/bin/python afterwards.

https://chatgpt.com/share/69702711-790c-800f-bbe0-812afdf9d975
2026-01-21 02:08:23 +01:00
kevinveenbirkenbach 18acafa9dc ci(images): build & push full+slim CI images
- Build matrix now includes variants: full + slim
- Tag slim images with -slim suffix and use Dockerfile target selection
- Use dedicated gha cache scopes per distro+variant
- Update release checks/retag/promote scripts to handle slim images too

https://chatgpt.com/share/69701a4e-b000-800f-be7e-162dcb93b1d2
2026-01-21 01:46:52 +01:00
kevinveenbirkenbach 0928022731 More python calls from correct context 2026-01-21 01:09:01 +01:00
kevinveenbirkenbach 91003615ea fix(setup): enforce venv python3 via Makefile and fail if missing
scripts/setup.sh no longer falls back to system python3.
The setup now requires PYTHON to be provided by the Makefile (venv)
and verifies that the interpreter exists and is Python 3.

https://chatgpt.com/share/6970174d-0c18-800f-8267-33818611521c
2026-01-21 01:01:11 +01:00
kevinveenbirkenbach a565344ed5 Solved shellcheck 2026-01-21 00:25:35 +01:00
kevinveenbirkenbach f08993049c feat(inventory): add dynamic inventory resolver and propagate INVENTORY_DIR everywhere
- add scripts/inventory/resolve.sh to compute INVENTORY_DIR based on runtime context (act/github/local)
- resolve INVENTORY_DIR dynamically in Makefile via resolve.sh
- enforce INVENTORY_DIR usage across all local deploy/test scripts
- propagate INVENTORY_DIR into container and development CLI commands
- remove hardcoded /etc/inventories paths

https://chatgpt.com/share/69700d69-b270-800f-ad50-d9ff486378b9
2026-01-21 00:18:57 +01:00
kevinveenbirkenbach 6c84807778 feat(test-local): add all-app inventory init and run targets
Add two local test helpers to run full-stack deployments for all discovered apps:

- make test-local-inventory-init-all: bring up dev stack and create inventory
  (/etc/inventories/local-full-<type>/servers.yml) including all apps
- make test-local-run-all: run deploy using the pre-initialized inventory

Also mark various scripts as executable (chmod +x) to ensure consistent
behavior across environments.

https://chatgpt.com/share/6970018e-1244-800f-9345-ca1596061729
2026-01-20 23:43:10 +01:00
kevinveenbirkenbach d5801f35b0 refactor(ci): make deploy env vars strict and export pythonpath
- Make TEST_DEPLOY_TYPE strictly required in build-test-matrix
- Enforce required env vars in all act/ci/local deploy scripts
- Add deploy type validation everywhere
- Restore and export PYTHONPATH in Makefile
- Remove unused test directory variables

https://chatgpt.com/share/69700042-3fcc-800f-8bf9-b429d5bb7e82
2026-01-20 23:22:32 +01:00
kevinveenbirkenbach fa5779aab1 refactor(setup): embed setup configuration directly into setup.sh
- Remove setup-related variables from Makefile
- Hard-code paths and scripts inside setup.sh
- Add validation and structured logging
- Compute reserved usernames dynamically

https://chatgpt.com/share/696ffc6d-dd00-800f-8f40-2112155fbeac
2026-01-20 23:06:31 +01:00
kevinveenbirkenbach 2344a04958 refactor(make): move install and setup logic into scripts
- extract Ansible collection install into scripts/install/ansible.sh
- extract venv bootstrap into scripts/install/venv.sh
- extract python deps install into scripts/install/python.sh
- extract project setup logic into scripts/setup.sh
- simplify Makefile targets to delegate to scripts
- add bootstrap target for full environment setup

https://chatgpt.com/share/696ff8cd-32d4-800f-bdd5-a34159aac1c0
2026-01-20 22:50:02 +01:00
kevinveenbirkenbach 12704e9fa3 refactor(test-act): extract act deploy logic into dedicated scripts
- Move test-act target logic into scripts/tests/deploy/act/all.sh
- Move test-act-app target logic into scripts/tests/deploy/act/app.sh
- Keep Makefile targets as thin delegators
- Preserve existing flags and behavior

This keeps the Makefile minimal and makes act workflows reusable.

https://chatgpt.com/share/696ff4de-3abc-800f-a206-2f3dd5dec170
2026-01-20 22:34:14 +01:00
kevinveenbirkenbach adfc5123fc refactor(test-local): extract cleanup and rapid deploy into dedicated scripts
- Move test-local-cleanup logic into scripts/tests/deploy/local/cleanup.sh
- Move test-local-rapid logic into scripts/tests/deploy/local/rapid.sh
- Simplify Makefile targets to delegate into scripts
- Keep behavior and flags fully backward compatible

This improves maintainability, reuse in CI, and keeps the Makefile thin.
2026-01-20 22:32:00 +01:00
kevinveenbirkenbach 5ce577bba2 refactor(tests): run unit tests directly via docker compose exec
- execute tests inside running infinito container (no CLI wrapper)
- require INFINITO_DISTRO explicitly and show it in banner
- keep Nix accept-flake-config behavior inside container
- pass TEST_PATTERN/TEST_TYPE as container env and set workdir
- fall back to python3 if PYTHON is unset

https://chatgpt.com/share/696ff1d0-aa84-800f-82cb-05f2b1618d05
2026-01-20 22:21:09 +01:00
kevinveenbirkenbach 50816243ed Solved linting hints 2026-01-20 20:50:50 +01:00
kevinveenbirkenbach d35e932e0c refactor(cli): switch development CLI to dynamic dispatcher and fix test paths
- Replace static argparse-based CLI in cli.deploy.development with dynamic
  module dispatcher compatible with nested command discovery.
- Ensure cli.deploy.development is executable via python -m and root dispatcher.
- Fix unit test path resolution to work locally and in CI.
- Make docker-* Makefile targets depend on install.
- Simplify test runner script and keep Nix flake config compatibility.

This aligns the CLI architecture with integration test discovery
(cli/**/__main__.py) and removes the need for manual subcommand registration.

https://chatgpt.com/share/696fdbbc-b6d8-800f-a8b5-067aa7e38706
2026-01-20 20:46:20 +01:00
kevinveenbirkenbach 4e0d09e773 refactor(dev-orchestrator): centralize image build and add up --when-down SPOT
- Introduce cli.deploy.development build subcommand as single build SPOT
- Remove legacy compose.build_infinito implementation
- Make up use build-missing logic via build handler
- Add --when-down flag to up (skip startup if stack already running)
- Simplify restart/run to reuse up handler
- Ensure tests use up --when-down to avoid stack interference
- Make test targets depend on build-missing

https://chatgpt.com/share/696fd3c9-99f0-800f-a0ab-33a91058419a
2026-01-20 20:13:02 +01:00
kevinveenbirkenbach bbea43f669 refactor(ci,tests): migrate discovery and local deploy to development compose stack
- Replace legacy ci scripts with meta build-test-matrix and resolve-invokable-roles
- Switch log tailing to scripts/meta/log-tail.sh
- Rename local DNS helpers to scripts/dns/*
- Run ci-deploy-discover via development docker-up
- Replace deploy-local-full with development compose orchestrator
- Ensure INFINITO_DISTRO is passed explicitly for local full deploy

https://chatgpt.com/share/696f9ccd-e4d8-800f-9c67-9c071188ecd0
https://chatgpt.com/share/696facf5-b2fc-800f-8a33-3352dd193c54
2026-01-20 17:27:09 +01:00
kevinveenbirkenbach a87c0375ef refactor(tests): run unit tests via development compose stack and exec wrapper
- Replace docker compose run with cli.deploy.development up/exec workflow
- Reuse existing stack (coredns + infinito) with healthcheck and entry init
- Preserve NIX flake non-interactive behavior inside container
- Ensure TEST_TYPE and TEST_PATTERN are validated and injected correctly

https://chatgpt.com/share/696f9ccd-e4d8-800f-9c67-9c071188ecd0
2026-01-20 16:39:12 +01:00
kevinveenbirkenbach f52a012829 feat(cli): replace test deploy runner with development CLI (up/down/stop/restart/run)
https://chatgpt.com/share/696f9ccd-e4d8-800f-9c67-9c071188ecd0
2026-01-20 16:18:31 +01:00
kevinveenbirkenbach aaf9667154 Removed legacy code 2026-01-20 13:02:02 +01:00
kevinveenbirkenbach a367fae1c1 ci(docker): ensure deterministic CI by wiping Docker data-root on /mnt
- Stop docker and clean /mnt/docker before restart
- Avoid volume/password drift and flaky Postgres init on GitHub runners
- Move purge_entity.sh into CI administration scope

https://chatgpt.com/share/696f4386-fbbc-800f-bdde-f1e6c4145c84
2026-01-20 11:52:22 +01:00
kevinveenbirkenbach b92e592336 Removed migrate marker 2026-01-20 09:59:04 +01:00
kevinveenbirkenbach e10a000291 fix(ci,postgres): stabilize GitHub runner env, move docker to /mnt, add deep debug for role login
- Introduce deterministic RUNNING_ON_GITHUB / RUNNING_ON_ACT detection
- Move Docker data-root to /mnt on real GitHub runners only
- Switch POSTGRES_PASSWORD schema from bcrypt to alphanumeric (avoid $ interpolation)
- Add robust rescue/debug block for ALTER ROLE login step
  (docker inspect, logs, pg_isready, psql auth, hba, activity, roles)
- Make docker-on-mnt deterministic (no migration, root-dir assertion)

Fixes flaky CI runs and intermittent PostgreSQL authentication failures.

https://chatgpt.com/share/696f4386-fbbc-800f-bdde-f1e6c4145c84
2026-01-20 09:57:32 +01:00
kevinveenbirkenbach 2baa03176a fix(ci): remove unused loop variable in move-docker-to-mnt.sh 2026-01-20 03:46:31 +01:00
kevinveenbirkenbach cb7b878c85 feat(ci): move Docker data-root to /mnt on GitHub runners and raise CI storage threshold
https://chatgpt.com/share/696eeb85-9d78-800f-9844-bfdebe07a0fa
2026-01-20 03:41:55 +01:00
kevinveenbirkenbach d8c50973bd ci: make discover.sh self-contained by resolving repo root internally
Move repo root detection into scripts/ci/discover.sh so it can be executed
from any working directory. Simplify ci-discover-output to just call the
script and rely on its internal path resolution.

https://chatgpt.com/share/696e3d20-276c-800f-87a7-a7052989ffe4
2026-01-19 17:45:38 +01:00
kevinveenbirkenbach d2420ab49c ci: run discovery on host and move storage filter into infinito container
Restore discover.sh execution on the host to avoid running Docker Compose
from inside the container, and execute only the storage validation logic
inside the infinito CI container.

This fixes "--profile" errors caused by compose-in-compose and keeps the
Python dependency stack isolated inside the container.

- Run scripts/ci/discover.sh directly on host
- Execute sufficient_storage filter via docker compose exec
- Keep CI output and ONLY_APP override unchanged

https://chatgpt.com/share/696e3d20-276c-800f-87a7-a7052989ffe4
2026-01-19 17:02:52 +01:00
kevinveenbirkenbach 0fa4454599 fix(admin): purge_entity.sh also removes docker volumes
Use `docker compose down -v` to ensure named and anonymous volumes
belonging to the stack are removed during purge.

https://chatgpt.com/share/696e2523-77d8-800f-bc3d-b81d628e1e2a
2026-01-19 13:35:40 +01:00
kevinveenbirkenbach c92aff464b feat(ci): add min_storage validation and CI filtering with warnings
- Add sufficient_storage CLI for validating docker.services.<entity>.min_storage
- Introduce role_resource_validation utility with human-friendly size parsing
- Filter CI app discovery by available runner storage (default: 12GB)
- Emit GitHub Actions warnings for:
  - missing min_storage metadata (treated as 0GB)
  - roles exceeding available runner storage
- Add BigBlueButton min_storage metadata (50GB)
- Add unit tests for validation logic and CLI behavior
- Add humanfriendly dependency for size parsing

https://chatgpt.com/share/696e097b-1038-800f-b8fc-23e9dadfa021
2026-01-19 11:37:28 +01:00
kevinveenbirkenbach 63bf2f7c62 administration: make DB drop best-effort and allow purging postgres/mariadb stacks safely
- Add best-effort helpers for Postgres and MariaDB database drops
- Avoid connecting to the target DB when dropping Postgres (use template1 for 'postgres')
- Ignore DB drop errors and continue with purge (compose down + rm -rf)
- Protect MariaDB system databases from being dropped
- Update script header to reflect best-effort behavior
2026-01-18 17:46:46 +01:00
kevinveenbirkenbach 0d8bf44c01 scripts/administration: add purge_entity helper to remove stacks + optional DB drop
- Support purging multiple stacks in one run
- Best-effort detect postgres/mariadb from .env and drop DB if found
- Always run compose down and delete /opt/docker/<stack>/volumes + stack dir
- Skip DB drop with warning when no backend is detected

https://chatgpt.com/share/696cfcd5-2b84-800f-a386-b7d1d38d68b7
2026-01-18 16:31:25 +01:00
kevinveenbirkenbach 39fc7b8a1a Added missing debugging flag 2026-01-18 14:24:12 +01:00
kevinveenbirkenbach df31230871 diff --git a/scripts/tests/deploy-local-full.sh b/scripts/tests/deploy-local-full.sh
index 9bcfc96c8..2c5b6d930 100755
--- a/scripts/tests/deploy-local-full.sh
+++ b/scripts/tests/deploy-local-full.sh
@@ -66,54 +66,53 @@ echo
 # ---------------------------------------------------------------------------
 # 2) Run entry.sh + create inventory + deploy INSIDE container
 # ---------------------------------------------------------------------------
-docker exec -t "${INFINITO_CONTAINER}" bash -lc "
+docker exec -t "${INFINITO_CONTAINER}" bash -lc '
   set -euo pipefail
   cd /opt/src/infinito

-  echo '>>> Running entry.sh bootstrap'
+  echo ">>> Running entry.sh bootstrap"
   ./scripts/docker/entry.sh true

-  inv_dir='${INVENTORY_BASE_DIR}/local-full-${TEST_DEPLOY_TYPE}'
-  inv_file=\"\${inv_dir}/servers.yml\"
-  pw_file=\"\${inv_dir}/.password\"
+  inv_dir="'"${INVENTORY_BASE_DIR}"'/local-full-'"${TEST_DEPLOY_TYPE}"'"
+  inv_file="${inv_dir}/servers.yml"
+  pw_file="${inv_dir}/.password"

-  mkdir -p \"\${inv_dir}\"
+  mkdir -p "${inv_dir}"

-  if [[ ! -f \"\${pw_file}\" ]]; then
-    printf '%s\n' 'local-vault-password' > \"\${pw_file}\"
-    chmod 600 \"\${pw_file}\" || true
+  if [[ ! -f "${pw_file}" ]]; then
+    printf "%s\n" "local-vault-password" > "${pw_file}"
+    chmod 600 "${pw_file}" || true
   fi

-  echo '>>> Creating inventory at' \"\${inv_file}\"
-  echo '>>> Include apps count: ${apps_count}'
-  # Avoid printing the full CSV (it is huge); show first few instead:
-  echo '>>> Include sample:' \"\$(python3 - <<'PY'
-import sys
fix(tests): stabilize deploy-local-full.sh quoting and sample generation

- Fix broken heredoc usage inside docker exec bash -lc block
- Prevent apps CSV from being interpreted as a shell command
- Make sample generation robust using python -c
- Ensure --include argument is expanded correctly
- Improve errexit reliability by avoiding command substitution inside echo
- Normalize quoting for host/container variable expansion

https://chatgpt.com/share/696c928c-38a4-800f-a45b-d5f3bba908ff
2026-01-18 08:57:59 +01:00
kevinveenbirkenbach 68157002a2 ci(dns): add AAAA sanity check to prevent CoreDNS SERVFAIL regressions
- Add host-level AAAA DNS sanity test against CoreDNS
- Fail hard on SERVFAIL (breaks Node/getaddrinfo with EAI_AGAIN)
- Allow empty AAAA responses (IPv4-only setup is valid)
- Keep A-record validation unchanged

This prevents silent IPv6 resolver regressions that break CSP checks.

https://chatgpt.com/share/696abc27-19dc-800f-9ead-6598c2f3c302
2026-01-16 23:30:33 +01:00
kevinveenbirkenbach b058377e33 test(dns): make dns test script POSIX-sh compatible for Debian/Ubuntu
- Replace bash-specific [[ ... ]] with POSIX [ ... ] tests
- Use '=' instead of '==' for portable string comparisons
- Fix /bin/sh compatibility (dash) inside docker exec on Debian/Ubuntu
- Prevent DNS test failures caused by non-bash shells

https://chatgpt.com/share/696a9cb2-58e8-800f-8a88-5973d0e1bc40
2026-01-16 21:16:43 +01:00
kevinveenbirkenbach 0ff511293e Add a dedicated Docker-in-Docker DNS self-test and integrate it into the
sys-svc-docker role as a post-restart validation handler.

This change moves the inner-container DNS validation out of the CI test
suite and into the Docker service lifecycle itself, ensuring that DNS is
verified exactly where it matters: inside the Docker-in-Docker daemon.

Key changes:
- Add roles/sys-svc-docker/files/test-dns.sh:
  - Runs inside the DinD container
  - Waits for dockerd readiness
  - Executes a BusyBox test container with explicit --dns
  - Verifies DOMAIN and subdomain resolve to the expected IP
  - Fails hard if DNS is broken

- Register DNS self-test as a docker restart handler:
  - Runs once per deployment
  - Only executed when DOCKER_IN_CONTAINER and MODE_DEBUG are enabled
  - Uses ansible.builtin.script with role_path for correct resolution

- Remove inner Docker DNS test from scripts/tests/dns.sh:
  - CI test suite now only validates host → CoreDNS → outer container chain
  - DinD DNS validation is now part of the Docker service health lifecycle

This ensures that:
- DNS is validated after every Docker restart
- DinD networking failures are detected immediately
- CI and production share the same DNS validation logic
- Docker-in-Docker DNS becomes a first-class health invariant

https://chatgpt.com/share/696a3dda-5c6c-800f-8a16-47c0c604e1e3
2026-01-16 15:07:17 +01:00
kevinveenbirkenbach 35bfbf7d9c fix(dns-tests): make dns.sh shellcheck-clean and harden env validation
- Quote dig server argument to avoid word splitting (SC2086)
- Replace &&/|| pseudo-if with proper if/else blocks (SC2015)
- Validate INFINITO_CONTAINER early via required env guard
- Keep DNS test logic unchanged, only harden correctness and linting

This makes scripts/tests/dns.sh fully shellcheck -x compliant and more robust.

https://chatgpt.com/share/696a4053-93f8-800f-959b-f99e2d7d12e2
2026-01-16 14:42:34 +01:00
kevinveenbirkenbach 26cde20899 Test DNS for all distros 2026-01-16 14:38:22 +01:00
kevinveenbirkenbach c7de80fa70 Removed double domain 2026-01-16 14:33:21 +01:00
kevinveenbirkenbach fd983bc69c refactor(ci,network): make DNS, IP and domain configuration fully env-driven
- Introduce env.ci as single source of truth for CI networking
- Make docker-compose, CoreDNS and Docker daemon fully env-driven
- Render CoreDNS Corefile from template via envsubst
- Replace hardcoded IPs/domains with environment variables
- Integrate env-based networking into Ansible defaults
- Remove static Corefile and daemon.json

This makes DNS and networking deterministic and reproducible across
host, CoreDNS, infinito runner and Docker-in-Docker.

https://chatgpt.com/share/696a3dda-5c6c-800f-8a16-47c0c604e1e3
2026-01-16 14:30:14 +01:00
kevinveenbirkenbach 3a496bcfff Optimized deploy parameters 2026-01-16 01:27:09 +01:00
kevinveenbirkenbach ea0ad4a6f7 fix(release): resolve ShellCheck SC2259 by piping JSON into python -c
Replace here-doc Python invocation with python -c so stdin from gh api is preserved and ShellCheck no longer reports SC2259.

https://chatgpt.com/share/696968ff-5704-800f-b222-2aff06da09ed
2026-01-15 23:23:53 +01:00
kevinveenbirkenbach bcc638fe0a dev(local): add full local deploy helper and rename act/rapid targets
- Rename act-based targets for clarity:
  - test-deploy -> test-act
  - test-deploy-app -> test-act-app
- Rename rapid local deploy target:
  - test-deploy-rapid -> test-local-rapid
- Add test-local-full target to run a full local deploy against a generated
  inventory containing all apps of the selected TEST_DEPLOY_TYPE
- Add scripts/tests/deploy-local-full.sh:
  - Discover apps on host via make ci-deploy-discover
  - Bootstrap container via scripts/docker/entry.sh
  - Generate dedicated inventory with --include list
  - Deploy against that inventory

https://chatgpt.com/share/69692ef1-095c-800f-83cb-924fadfaa554
2026-01-15 20:05:08 +01:00
kevinveenbirkenbach bab0bceb1d feat(local-dns): add dnsmasq-based infinito.example wildcard + set CI primary domain
https://chatgpt.com/share/6969261a-8d34-800f-9a61-725e90544dea
2026-01-15 18:38:15 +01:00
kevinveenbirkenbach c5437b21e3 ci(release): use code-analyses workflow as stable promotion gate
Promote-stable now validates version tags against the real quality gate
(code-analyses.yml) instead of the nightly scheduler wrapper.

This ensures the stable tag only points to versions that passed the full
CI matrix (lint, security, build, deploy, install, tests).

https://chatgpt.com/share/69691ebf-a380-800f-80fc-bd44d59e90dd
2026-01-15 18:07:08 +01:00
kevinveenbirkenbach 97cfdf3f9f ci(release): introduce daily stable promotion and unify release concurrency
- Add daily stable promotion workflow with CI-gated highest-version selection
- Ensure GHCR images exist before moving stable tag
- Retag version images to :stable without rebuild
- Unify release workflows under infinito-release concurrency group
- Remove legacy highest-version/stable-move scripts and inline logic

https://chatgpt.com/share/696909c1-781c-800f-b52d-d9f967e9698c
2026-01-15 18:03:01 +01:00