Commit Graph
97 Commits
Author SHA1 Message Date
kevinveenbirkenbach 51239968fd refactor: rename docker-compose roles to sys-svc-compose and update paths
Renamed docker-compose and docker-compose-ca roles to sys-svc-compose and sys-svc-compose-ca, updated template includes, handlers, lookups, tests, and variables, and moved compose instances to /opt/compose.

https://chatgpt.com/share/6985ab2c-8344-800f-9e9a-768f562d138d
2026-02-06 09:49:42 +01:00
kevinveenbirkenbach 88924b4fdd Removed update mode verywhere to keep behaviour more deterministic 2026-02-05 11:43:14 +01:00
kevinveenbirkenbach f9f0501f1a Refresh matomo 2026-02-04 14:03:38 +01:00
kevinveenbirkenbach a5bbba1eb3 Added inventory to cleanup 2026-02-04 10:07:02 +01:00
kevinveenbirkenbach b6fa8c1a31 Build inventory at beginning 2026-02-04 01:16:10 +01:00
kevinveenbirkenbach fe40b48cab tests(local): add non-destructive dedicated deploy and refactor local utils
- Add local dedicated_distro.sh implementing the same two-pass deploy logic
  as CI (ASYNC_ENABLED=false/true) without destructive cleanup
- Switch test-local-dedicated to pure ENV-based interface (TEST_DEPLOY_TYPE, APP)
- Refactor local deploy helpers into utils/ structure:
  - move reset.sh to utils/reset.sh
  - split cleanup into targeted purge helpers (entity, web, lib)
- Update Makefile targets to reflect new structure and workflows

https://chatgpt.com/share/6982804e-1f60-800f-809e-629d0fe62de3
2026-02-04 00:10:00 +01:00
kevinveenbirkenbach 4066dbfeed Adapted script path 2026-02-03 22:51:34 +01:00
kevinveenbirkenbach 0ca8448b30 ci: hard-clean Docker state between distro deploys
- remove host-mounted INFINITO_DOCKER_VOLUME between runs
- recreate docker data dir to avoid cross-distro state leakage
- keep images while fully resetting containers, networks and volumes
- ensure clean CI environment for each distro deploy
2026-02-03 20:06:36 +01:00
kevinveenbirkenbach 0c9c8277d5 feat(ci): shuffle distros before deploy to improve fairness under time budget
- Randomize distro execution order in all_distros.sh
- Log final execution order for easier debugging

https://chatgpt.com/share/6981f7e4-590c-800f-a6df-15557ce73a86
2026-02-03 14:37:45 +01:00
kevinveenbirkenbach c75a823391 feat(ci): enforce global time budget for multi-distro deploys
- Track runtime per distro during CI deploy runs
- Add MAX_TOTAL_SECONDS env to limit total execution time
- Skip remaining distros if remaining budget is smaller than the slowest prior run
- Print warnings and a detailed timing summary at the end
- Wire time budget via Makefile for ci-deploy-app

https://chatgpt.com/share/6981f7e4-590c-800f-a6df-15557ce73a86
2026-02-03 14:27:59 +01:00
kevinveenbirkenbach 9344a11c18 fix(ci-cleanup): use array-safe container removal to satisfy ShellCheck
- Replace word-splitting container ID handling with bash array
- Avoid SC2086 by passing container IDs as "${ids[@]}"
- Preserve correct behavior for multiple containers in CI cleanup

https://chatgpt.com/share/6981c7d4-642c-800f-8438-9e2349ac53fb
2026-02-03 11:05:15 +01:00
kevinveenbirkenbach bc38347798 refactor(ci-deploy): split per-distro deploy, add two-pass async test and hard cleanup
- Replace legacy app.sh/distros.sh orchestration with explicit all_distros.sh + dedicated_distro.sh
- Deploy each app sequentially across distros against a fresh stack
- Run two deploy passes per distro:
  - Pass 1 with ASYNC_ENABLED=false
  - Pass 2 with ASYNC_ENABLED=true against the same stack
- Remove deprecated cli.deploy.development run orchestrator
- Extend init command with --vars to inject inventory overrides
- Enforce debug mode for CI deploys
- Add hard Docker cleanup after each distro (containers, networks, volumes, build cache; keep images)
- Emit disk and docker usage before and after deploy for observability

https://chatgpt.com/share/6981c7d4-642c-800f-8438-9e2349ac53fb
2026-02-03 11:02:51 +01:00
kevinveenbirkenbach b9bb4a95fd Included other helper command 2026-01-29 14:41:17 +01:00
kevinveenbirkenbach ecb0d6071c On Local deploy reduce repeat times for failed systemctl runenrs to debug faster 2026-01-29 11:26:59 +01:00
kevinveenbirkenbach 52ba09d165 fix(local-deploy): use TEST_DEPLOY_TYPE inventory file and fix shell expansion
https://chatgpt.com/share/697a73ea-1b34-800f-8318-be15afab2d08
2026-01-28 21:38:47 +01:00
kevinveenbirkenbach 3d3d5480d7 fix(admin): make purge_entity.sh TTY-safe and support multi-entity cleanup
- Isolate each stack purge in a subshell to prevent early termination
- Detach all docker/docker-compose calls from STDIN to avoid TTY issues inside containers
- Harden MariaDB drop/truncate logic against nested docker exec behavior
- Allow cleanup.sh to accept multiple APP values (whitespace or comma-separated)
- Deduplicate derived entities before purging and iterate deterministically

https://chatgpt.com/share/69774b3d-611c-800f-8cf0-422d11d50c1c
2026-01-26 12:08:36 +01:00
kevinveenbirkenbach f1b4436d57 Removed " to allow the test deploy of multiple apps in one step 2026-01-24 20:30:31 +01:00
kevinveenbirkenbach b2eea91fa4 Activated debug 2026-01-24 16:28:20 +01:00
kevinveenbirkenbach 51e01d0bec Introduced correct order 2026-01-24 14:10:43 +01:00
kevinveenbirkenbach 8bdf43ee87 Minor improvements 2026-01-23 11:17:53 +01:00
kevinveenbirkenbach 49246fb3d8 test(local): reset inventory before init
Rename test-local-init to test-local-reset and wipe inventory directory
before recreating it to ensure clean local test runs.

https://chatgpt.com/share/69729492-8afc-800f-99f1-823512d8604e
2026-01-22 22:20:13 +01:00
kevinveenbirkenbach 0eb7071a49 fix(local-tests): force python3 for local inventory init and stabilize runtime vars
- Force PYTHON=python3 for test-local-init to avoid missing venv python
- Rename test-local-inventory-init-all target to test-local-init
- Fix JSON quoting for --vars in local deploy scripts
- Add RUNTIME default to group_vars
- Improve debug output for initial host vars
- Update user hint in local run script

https://chatgpt.com/share/697222e1-5edc-800f-b9ab-511d8afc3142
2026-01-22 14:14:50 +01:00
kevinveenbirkenbach d2694566e5 feat(runtime): add development runtime marker and unified runtime detection
- introduce env.development marker for local dev runtime
- add cli.meta.runtime module with library API and CLI entrypoint
- detect runtime via ACT/GitHub/marker with explicit RUNTIME override
- use env.development as docker-compose env override
- pass RUNTIME=dev into development inventory generation
- add make setup-development helper

https://chatgpt.com/share/69720fd6-a1c8-800f-b1e2-6909aa63643a
2026-01-22 12:53:42 +01:00
kevinveenbirkenbach c9e05dcb6b test(local): print full app list in single line instead of sample
https://chatgpt.com/share/69710189-7df4-800f-a291-2979678ea198
2026-01-21 17:39:49 +01:00
kevinveenbirkenbach 249103bb56 feat(ci): unify local act workflow with CI discovery logic and strict input resolver
- Use docker-up + ci-discover-output for local discover job (same as CI)
- Add strict input resolver (scripts/meta/resolve/input.sh) with enforced env contract
- Ensure ONLY_APP is always defined (may be empty, but never unset)
- Expose resolved inputs as workflow outputs for linear matrix deploy
- Enforce linear matrix execution via max-parallel: 1
- Align act helper scripts with linear matrix semantics

https://chatgpt.com/share/6970ba64-8c2c-800f-bb90-7edbe9168582
2026-01-21 12:37:03 +01:00
kevinveenbirkenbach 4cf54418ca Removed --missing 2026-01-21 03:34:06 +01:00
kevinveenbirkenbach 7bf8070ef7 fix(ci): remove unsupported --no-cache/--missing flags from distros.sh
The CI distros runner now only passes arguments supported by
cli.deploy.development run. This avoids argparse errors caused by
forwarding build-only flags that the orchestrator does not accept.

https://chatgpt.com/share/697030c4-f60c-800f-94b6-9128f87da57e
2026-01-21 02:49:48 +01:00
kevinveenbirkenbach 18acafa9dc ci(images): build & push full+slim CI images
- Build matrix now includes variants: full + slim
- Tag slim images with -slim suffix and use Dockerfile target selection
- Use dedicated gha cache scopes per distro+variant
- Update release checks/retag/promote scripts to handle slim images too

https://chatgpt.com/share/69701a4e-b000-800f-be7e-162dcb93b1d2
2026-01-21 01:46:52 +01:00
kevinveenbirkenbach 0928022731 More python calls from correct context 2026-01-21 01:09:01 +01:00
kevinveenbirkenbach a565344ed5 Solved shellcheck 2026-01-21 00:25:35 +01:00
kevinveenbirkenbach f08993049c feat(inventory): add dynamic inventory resolver and propagate INVENTORY_DIR everywhere
- add scripts/inventory/resolve.sh to compute INVENTORY_DIR based on runtime context (act/github/local)
- resolve INVENTORY_DIR dynamically in Makefile via resolve.sh
- enforce INVENTORY_DIR usage across all local deploy/test scripts
- propagate INVENTORY_DIR into container and development CLI commands
- remove hardcoded /etc/inventories paths

https://chatgpt.com/share/69700d69-b270-800f-ad50-d9ff486378b9
2026-01-21 00:18:57 +01:00
kevinveenbirkenbach 6c84807778 feat(test-local): add all-app inventory init and run targets
Add two local test helpers to run full-stack deployments for all discovered apps:

- make test-local-inventory-init-all: bring up dev stack and create inventory
  (/etc/inventories/local-full-<type>/servers.yml) including all apps
- make test-local-run-all: run deploy using the pre-initialized inventory

Also mark various scripts as executable (chmod +x) to ensure consistent
behavior across environments.

https://chatgpt.com/share/6970018e-1244-800f-9345-ca1596061729
2026-01-20 23:43:10 +01:00
kevinveenbirkenbach d5801f35b0 refactor(ci): make deploy env vars strict and export pythonpath
- Make TEST_DEPLOY_TYPE strictly required in build-test-matrix
- Enforce required env vars in all act/ci/local deploy scripts
- Add deploy type validation everywhere
- Restore and export PYTHONPATH in Makefile
- Remove unused test directory variables

https://chatgpt.com/share/69700042-3fcc-800f-8bf9-b429d5bb7e82
2026-01-20 23:22:32 +01:00
kevinveenbirkenbach 12704e9fa3 refactor(test-act): extract act deploy logic into dedicated scripts
- Move test-act target logic into scripts/tests/deploy/act/all.sh
- Move test-act-app target logic into scripts/tests/deploy/act/app.sh
- Keep Makefile targets as thin delegators
- Preserve existing flags and behavior

This keeps the Makefile minimal and makes act workflows reusable.

https://chatgpt.com/share/696ff4de-3abc-800f-a206-2f3dd5dec170
2026-01-20 22:34:14 +01:00
kevinveenbirkenbach adfc5123fc refactor(test-local): extract cleanup and rapid deploy into dedicated scripts
- Move test-local-cleanup logic into scripts/tests/deploy/local/cleanup.sh
- Move test-local-rapid logic into scripts/tests/deploy/local/rapid.sh
- Simplify Makefile targets to delegate into scripts
- Keep behavior and flags fully backward compatible

This improves maintainability, reuse in CI, and keeps the Makefile thin.
2026-01-20 22:32:00 +01:00
kevinveenbirkenbach 5ce577bba2 refactor(tests): run unit tests directly via docker compose exec
- execute tests inside running infinito container (no CLI wrapper)
- require INFINITO_DISTRO explicitly and show it in banner
- keep Nix accept-flake-config behavior inside container
- pass TEST_PATTERN/TEST_TYPE as container env and set workdir
- fall back to python3 if PYTHON is unset

https://chatgpt.com/share/696ff1d0-aa84-800f-82cb-05f2b1618d05
2026-01-20 22:21:09 +01:00
kevinveenbirkenbach 50816243ed Solved linting hints 2026-01-20 20:50:50 +01:00
kevinveenbirkenbach d35e932e0c refactor(cli): switch development CLI to dynamic dispatcher and fix test paths
- Replace static argparse-based CLI in cli.deploy.development with dynamic
  module dispatcher compatible with nested command discovery.
- Ensure cli.deploy.development is executable via python -m and root dispatcher.
- Fix unit test path resolution to work locally and in CI.
- Make docker-* Makefile targets depend on install.
- Simplify test runner script and keep Nix flake config compatibility.

This aligns the CLI architecture with integration test discovery
(cli/**/__main__.py) and removes the need for manual subcommand registration.

https://chatgpt.com/share/696fdbbc-b6d8-800f-a8b5-067aa7e38706
2026-01-20 20:46:20 +01:00
kevinveenbirkenbach 4e0d09e773 refactor(dev-orchestrator): centralize image build and add up --when-down SPOT
- Introduce cli.deploy.development build subcommand as single build SPOT
- Remove legacy compose.build_infinito implementation
- Make up use build-missing logic via build handler
- Add --when-down flag to up (skip startup if stack already running)
- Simplify restart/run to reuse up handler
- Ensure tests use up --when-down to avoid stack interference
- Make test targets depend on build-missing

https://chatgpt.com/share/696fd3c9-99f0-800f-a0ab-33a91058419a
2026-01-20 20:13:02 +01:00
kevinveenbirkenbach bbea43f669 refactor(ci,tests): migrate discovery and local deploy to development compose stack
- Replace legacy ci scripts with meta build-test-matrix and resolve-invokable-roles
- Switch log tailing to scripts/meta/log-tail.sh
- Rename local DNS helpers to scripts/dns/*
- Run ci-deploy-discover via development docker-up
- Replace deploy-local-full with development compose orchestrator
- Ensure INFINITO_DISTRO is passed explicitly for local full deploy

https://chatgpt.com/share/696f9ccd-e4d8-800f-9c67-9c071188ecd0
https://chatgpt.com/share/696facf5-b2fc-800f-8a33-3352dd193c54
2026-01-20 17:27:09 +01:00
kevinveenbirkenbach a87c0375ef refactor(tests): run unit tests via development compose stack and exec wrapper
- Replace docker compose run with cli.deploy.development up/exec workflow
- Reuse existing stack (coredns + infinito) with healthcheck and entry init
- Preserve NIX flake non-interactive behavior inside container
- Ensure TEST_TYPE and TEST_PATTERN are validated and injected correctly

https://chatgpt.com/share/696f9ccd-e4d8-800f-9c67-9c071188ecd0
2026-01-20 16:39:12 +01:00
kevinveenbirkenbach f52a012829 feat(cli): replace test deploy runner with development CLI (up/down/stop/restart/run)
https://chatgpt.com/share/696f9ccd-e4d8-800f-9c67-9c071188ecd0
2026-01-20 16:18:31 +01:00
kevinveenbirkenbach df31230871 diff --git a/scripts/tests/deploy-local-full.sh b/scripts/tests/deploy-local-full.sh
index 9bcfc96c8..2c5b6d930 100755
--- a/scripts/tests/deploy-local-full.sh
+++ b/scripts/tests/deploy-local-full.sh
@@ -66,54 +66,53 @@ echo
 # ---------------------------------------------------------------------------
 # 2) Run entry.sh + create inventory + deploy INSIDE container
 # ---------------------------------------------------------------------------
-docker exec -t "${INFINITO_CONTAINER}" bash -lc "
+docker exec -t "${INFINITO_CONTAINER}" bash -lc '
   set -euo pipefail
   cd /opt/src/infinito

-  echo '>>> Running entry.sh bootstrap'
+  echo ">>> Running entry.sh bootstrap"
   ./scripts/docker/entry.sh true

-  inv_dir='${INVENTORY_BASE_DIR}/local-full-${TEST_DEPLOY_TYPE}'
-  inv_file=\"\${inv_dir}/servers.yml\"
-  pw_file=\"\${inv_dir}/.password\"
+  inv_dir="'"${INVENTORY_BASE_DIR}"'/local-full-'"${TEST_DEPLOY_TYPE}"'"
+  inv_file="${inv_dir}/servers.yml"
+  pw_file="${inv_dir}/.password"

-  mkdir -p \"\${inv_dir}\"
+  mkdir -p "${inv_dir}"

-  if [[ ! -f \"\${pw_file}\" ]]; then
-    printf '%s\n' 'local-vault-password' > \"\${pw_file}\"
-    chmod 600 \"\${pw_file}\" || true
+  if [[ ! -f "${pw_file}" ]]; then
+    printf "%s\n" "local-vault-password" > "${pw_file}"
+    chmod 600 "${pw_file}" || true
   fi

-  echo '>>> Creating inventory at' \"\${inv_file}\"
-  echo '>>> Include apps count: ${apps_count}'
-  # Avoid printing the full CSV (it is huge); show first few instead:
-  echo '>>> Include sample:' \"\$(python3 - <<'PY'
-import sys
fix(tests): stabilize deploy-local-full.sh quoting and sample generation

- Fix broken heredoc usage inside docker exec bash -lc block
- Prevent apps CSV from being interpreted as a shell command
- Make sample generation robust using python -c
- Ensure --include argument is expanded correctly
- Improve errexit reliability by avoiding command substitution inside echo
- Normalize quoting for host/container variable expansion

https://chatgpt.com/share/696c928c-38a4-800f-a45b-d5f3bba908ff
2026-01-18 08:57:59 +01:00
kevinveenbirkenbach 68157002a2 ci(dns): add AAAA sanity check to prevent CoreDNS SERVFAIL regressions
- Add host-level AAAA DNS sanity test against CoreDNS
- Fail hard on SERVFAIL (breaks Node/getaddrinfo with EAI_AGAIN)
- Allow empty AAAA responses (IPv4-only setup is valid)
- Keep A-record validation unchanged

This prevents silent IPv6 resolver regressions that break CSP checks.

https://chatgpt.com/share/696abc27-19dc-800f-9ead-6598c2f3c302
2026-01-16 23:30:33 +01:00
kevinveenbirkenbach b058377e33 test(dns): make dns test script POSIX-sh compatible for Debian/Ubuntu
- Replace bash-specific [[ ... ]] with POSIX [ ... ] tests
- Use '=' instead of '==' for portable string comparisons
- Fix /bin/sh compatibility (dash) inside docker exec on Debian/Ubuntu
- Prevent DNS test failures caused by non-bash shells

https://chatgpt.com/share/696a9cb2-58e8-800f-8a88-5973d0e1bc40
2026-01-16 21:16:43 +01:00
kevinveenbirkenbach 0ff511293e Add a dedicated Docker-in-Docker DNS self-test and integrate it into the
sys-svc-docker role as a post-restart validation handler.

This change moves the inner-container DNS validation out of the CI test
suite and into the Docker service lifecycle itself, ensuring that DNS is
verified exactly where it matters: inside the Docker-in-Docker daemon.

Key changes:
- Add roles/sys-svc-docker/files/test-dns.sh:
  - Runs inside the DinD container
  - Waits for dockerd readiness
  - Executes a BusyBox test container with explicit --dns
  - Verifies DOMAIN and subdomain resolve to the expected IP
  - Fails hard if DNS is broken

- Register DNS self-test as a docker restart handler:
  - Runs once per deployment
  - Only executed when DOCKER_IN_CONTAINER and MODE_DEBUG are enabled
  - Uses ansible.builtin.script with role_path for correct resolution

- Remove inner Docker DNS test from scripts/tests/dns.sh:
  - CI test suite now only validates host → CoreDNS → outer container chain
  - DinD DNS validation is now part of the Docker service health lifecycle

This ensures that:
- DNS is validated after every Docker restart
- DinD networking failures are detected immediately
- CI and production share the same DNS validation logic
- Docker-in-Docker DNS becomes a first-class health invariant

https://chatgpt.com/share/696a3dda-5c6c-800f-8a16-47c0c604e1e3
2026-01-16 15:07:17 +01:00
kevinveenbirkenbach 35bfbf7d9c fix(dns-tests): make dns.sh shellcheck-clean and harden env validation
- Quote dig server argument to avoid word splitting (SC2086)
- Replace &&/|| pseudo-if with proper if/else blocks (SC2015)
- Validate INFINITO_CONTAINER early via required env guard
- Keep DNS test logic unchanged, only harden correctness and linting

This makes scripts/tests/dns.sh fully shellcheck -x compliant and more robust.

https://chatgpt.com/share/696a4053-93f8-800f-959b-f99e2d7d12e2
2026-01-16 14:42:34 +01:00
kevinveenbirkenbach c7de80fa70 Removed double domain 2026-01-16 14:33:21 +01:00
kevinveenbirkenbach fd983bc69c refactor(ci,network): make DNS, IP and domain configuration fully env-driven
- Introduce env.ci as single source of truth for CI networking
- Make docker-compose, CoreDNS and Docker daemon fully env-driven
- Render CoreDNS Corefile from template via envsubst
- Replace hardcoded IPs/domains with environment variables
- Integrate env-based networking into Ansible defaults
- Remove static Corefile and daemon.json

This makes DNS and networking deterministic and reproducible across
host, CoreDNS, infinito runner and Docker-in-Docker.

https://chatgpt.com/share/696a3dda-5c6c-800f-8a16-47c0c604e1e3
2026-01-16 14:30:14 +01:00
kevinveenbirkenbach 3a496bcfff Optimized deploy parameters 2026-01-16 01:27:09 +01:00