Commit Graph
99 Commits
Author SHA1 Message Date
kevinveenbirkenbach e8f62dfa7c Solved reset bug 2026-02-06 18:02:24 +01:00
kevinveenbirkenbach b399253884 test(deploy): run dedicated_distro init and deploy inside development container
Move entry.sh, development init, and development deploy execution into the
development container via cli.deploy.development exec.

Deployment logic, arguments, and execution order remain unchanged.

https://chatgpt.com/share/69861572-9c74-800f-9f3f-904202371bbc
2026-02-06 17:22:38 +01:00
kevinveenbirkenbach 51239968fd refactor: rename docker-compose roles to sys-svc-compose and update paths
Renamed docker-compose and docker-compose-ca roles to sys-svc-compose and sys-svc-compose-ca, updated template includes, handlers, lookups, tests, and variables, and moved compose instances to /opt/compose.

https://chatgpt.com/share/6985ab2c-8344-800f-9e9a-768f562d138d
2026-02-06 09:49:42 +01:00
kevinveenbirkenbach 88924b4fdd Removed update mode verywhere to keep behaviour more deterministic 2026-02-05 11:43:14 +01:00
kevinveenbirkenbach f9f0501f1a Refresh matomo 2026-02-04 14:03:38 +01:00
kevinveenbirkenbach a5bbba1eb3 Added inventory to cleanup 2026-02-04 10:07:02 +01:00
kevinveenbirkenbach b6fa8c1a31 Build inventory at beginning 2026-02-04 01:16:10 +01:00
kevinveenbirkenbach fe40b48cab tests(local): add non-destructive dedicated deploy and refactor local utils
- Add local dedicated_distro.sh implementing the same two-pass deploy logic
  as CI (ASYNC_ENABLED=false/true) without destructive cleanup
- Switch test-local-dedicated to pure ENV-based interface (TEST_DEPLOY_TYPE, APP)
- Refactor local deploy helpers into utils/ structure:
  - move reset.sh to utils/reset.sh
  - split cleanup into targeted purge helpers (entity, web, lib)
- Update Makefile targets to reflect new structure and workflows

https://chatgpt.com/share/6982804e-1f60-800f-809e-629d0fe62de3
2026-02-04 00:10:00 +01:00
kevinveenbirkenbach 4066dbfeed Adapted script path 2026-02-03 22:51:34 +01:00
kevinveenbirkenbach 0ca8448b30 ci: hard-clean Docker state between distro deploys
- remove host-mounted INFINITO_DOCKER_VOLUME between runs
- recreate docker data dir to avoid cross-distro state leakage
- keep images while fully resetting containers, networks and volumes
- ensure clean CI environment for each distro deploy
2026-02-03 20:06:36 +01:00
kevinveenbirkenbach 0c9c8277d5 feat(ci): shuffle distros before deploy to improve fairness under time budget
- Randomize distro execution order in all_distros.sh
- Log final execution order for easier debugging

https://chatgpt.com/share/6981f7e4-590c-800f-a6df-15557ce73a86
2026-02-03 14:37:45 +01:00
kevinveenbirkenbach c75a823391 feat(ci): enforce global time budget for multi-distro deploys
- Track runtime per distro during CI deploy runs
- Add MAX_TOTAL_SECONDS env to limit total execution time
- Skip remaining distros if remaining budget is smaller than the slowest prior run
- Print warnings and a detailed timing summary at the end
- Wire time budget via Makefile for ci-deploy-app

https://chatgpt.com/share/6981f7e4-590c-800f-a6df-15557ce73a86
2026-02-03 14:27:59 +01:00
kevinveenbirkenbach 9344a11c18 fix(ci-cleanup): use array-safe container removal to satisfy ShellCheck
- Replace word-splitting container ID handling with bash array
- Avoid SC2086 by passing container IDs as "${ids[@]}"
- Preserve correct behavior for multiple containers in CI cleanup

https://chatgpt.com/share/6981c7d4-642c-800f-8438-9e2349ac53fb
2026-02-03 11:05:15 +01:00
kevinveenbirkenbach bc38347798 refactor(ci-deploy): split per-distro deploy, add two-pass async test and hard cleanup
- Replace legacy app.sh/distros.sh orchestration with explicit all_distros.sh + dedicated_distro.sh
- Deploy each app sequentially across distros against a fresh stack
- Run two deploy passes per distro:
  - Pass 1 with ASYNC_ENABLED=false
  - Pass 2 with ASYNC_ENABLED=true against the same stack
- Remove deprecated cli.deploy.development run orchestrator
- Extend init command with --vars to inject inventory overrides
- Enforce debug mode for CI deploys
- Add hard Docker cleanup after each distro (containers, networks, volumes, build cache; keep images)
- Emit disk and docker usage before and after deploy for observability

https://chatgpt.com/share/6981c7d4-642c-800f-8438-9e2349ac53fb
2026-02-03 11:02:51 +01:00
kevinveenbirkenbach b9bb4a95fd Included other helper command 2026-01-29 14:41:17 +01:00
kevinveenbirkenbach ecb0d6071c On Local deploy reduce repeat times for failed systemctl runenrs to debug faster 2026-01-29 11:26:59 +01:00
kevinveenbirkenbach 52ba09d165 fix(local-deploy): use TEST_DEPLOY_TYPE inventory file and fix shell expansion
https://chatgpt.com/share/697a73ea-1b34-800f-8318-be15afab2d08
2026-01-28 21:38:47 +01:00
kevinveenbirkenbach 3d3d5480d7 fix(admin): make purge_entity.sh TTY-safe and support multi-entity cleanup
- Isolate each stack purge in a subshell to prevent early termination
- Detach all docker/docker-compose calls from STDIN to avoid TTY issues inside containers
- Harden MariaDB drop/truncate logic against nested docker exec behavior
- Allow cleanup.sh to accept multiple APP values (whitespace or comma-separated)
- Deduplicate derived entities before purging and iterate deterministically

https://chatgpt.com/share/69774b3d-611c-800f-8cf0-422d11d50c1c
2026-01-26 12:08:36 +01:00
kevinveenbirkenbach f1b4436d57 Removed " to allow the test deploy of multiple apps in one step 2026-01-24 20:30:31 +01:00
kevinveenbirkenbach b2eea91fa4 Activated debug 2026-01-24 16:28:20 +01:00
kevinveenbirkenbach 51e01d0bec Introduced correct order 2026-01-24 14:10:43 +01:00
kevinveenbirkenbach 8bdf43ee87 Minor improvements 2026-01-23 11:17:53 +01:00
kevinveenbirkenbach 49246fb3d8 test(local): reset inventory before init
Rename test-local-init to test-local-reset and wipe inventory directory
before recreating it to ensure clean local test runs.

https://chatgpt.com/share/69729492-8afc-800f-99f1-823512d8604e
2026-01-22 22:20:13 +01:00
kevinveenbirkenbach 0eb7071a49 fix(local-tests): force python3 for local inventory init and stabilize runtime vars
- Force PYTHON=python3 for test-local-init to avoid missing venv python
- Rename test-local-inventory-init-all target to test-local-init
- Fix JSON quoting for --vars in local deploy scripts
- Add RUNTIME default to group_vars
- Improve debug output for initial host vars
- Update user hint in local run script

https://chatgpt.com/share/697222e1-5edc-800f-b9ab-511d8afc3142
2026-01-22 14:14:50 +01:00
kevinveenbirkenbach d2694566e5 feat(runtime): add development runtime marker and unified runtime detection
- introduce env.development marker for local dev runtime
- add cli.meta.runtime module with library API and CLI entrypoint
- detect runtime via ACT/GitHub/marker with explicit RUNTIME override
- use env.development as docker-compose env override
- pass RUNTIME=dev into development inventory generation
- add make setup-development helper

https://chatgpt.com/share/69720fd6-a1c8-800f-b1e2-6909aa63643a
2026-01-22 12:53:42 +01:00
kevinveenbirkenbach c9e05dcb6b test(local): print full app list in single line instead of sample
https://chatgpt.com/share/69710189-7df4-800f-a291-2979678ea198
2026-01-21 17:39:49 +01:00
kevinveenbirkenbach 249103bb56 feat(ci): unify local act workflow with CI discovery logic and strict input resolver
- Use docker-up + ci-discover-output for local discover job (same as CI)
- Add strict input resolver (scripts/meta/resolve/input.sh) with enforced env contract
- Ensure ONLY_APP is always defined (may be empty, but never unset)
- Expose resolved inputs as workflow outputs for linear matrix deploy
- Enforce linear matrix execution via max-parallel: 1
- Align act helper scripts with linear matrix semantics

https://chatgpt.com/share/6970ba64-8c2c-800f-bb90-7edbe9168582
2026-01-21 12:37:03 +01:00
kevinveenbirkenbach 4cf54418ca Removed --missing 2026-01-21 03:34:06 +01:00
kevinveenbirkenbach 7bf8070ef7 fix(ci): remove unsupported --no-cache/--missing flags from distros.sh
The CI distros runner now only passes arguments supported by
cli.deploy.development run. This avoids argparse errors caused by
forwarding build-only flags that the orchestrator does not accept.

https://chatgpt.com/share/697030c4-f60c-800f-94b6-9128f87da57e
2026-01-21 02:49:48 +01:00
kevinveenbirkenbach 18acafa9dc ci(images): build & push full+slim CI images
- Build matrix now includes variants: full + slim
- Tag slim images with -slim suffix and use Dockerfile target selection
- Use dedicated gha cache scopes per distro+variant
- Update release checks/retag/promote scripts to handle slim images too

https://chatgpt.com/share/69701a4e-b000-800f-be7e-162dcb93b1d2
2026-01-21 01:46:52 +01:00
kevinveenbirkenbach 0928022731 More python calls from correct context 2026-01-21 01:09:01 +01:00
kevinveenbirkenbach a565344ed5 Solved shellcheck 2026-01-21 00:25:35 +01:00
kevinveenbirkenbach f08993049c feat(inventory): add dynamic inventory resolver and propagate INVENTORY_DIR everywhere
- add scripts/inventory/resolve.sh to compute INVENTORY_DIR based on runtime context (act/github/local)
- resolve INVENTORY_DIR dynamically in Makefile via resolve.sh
- enforce INVENTORY_DIR usage across all local deploy/test scripts
- propagate INVENTORY_DIR into container and development CLI commands
- remove hardcoded /etc/inventories paths

https://chatgpt.com/share/69700d69-b270-800f-ad50-d9ff486378b9
2026-01-21 00:18:57 +01:00
kevinveenbirkenbach 6c84807778 feat(test-local): add all-app inventory init and run targets
Add two local test helpers to run full-stack deployments for all discovered apps:

- make test-local-inventory-init-all: bring up dev stack and create inventory
  (/etc/inventories/local-full-<type>/servers.yml) including all apps
- make test-local-run-all: run deploy using the pre-initialized inventory

Also mark various scripts as executable (chmod +x) to ensure consistent
behavior across environments.

https://chatgpt.com/share/6970018e-1244-800f-9345-ca1596061729
2026-01-20 23:43:10 +01:00
kevinveenbirkenbach d5801f35b0 refactor(ci): make deploy env vars strict and export pythonpath
- Make TEST_DEPLOY_TYPE strictly required in build-test-matrix
- Enforce required env vars in all act/ci/local deploy scripts
- Add deploy type validation everywhere
- Restore and export PYTHONPATH in Makefile
- Remove unused test directory variables

https://chatgpt.com/share/69700042-3fcc-800f-8bf9-b429d5bb7e82
2026-01-20 23:22:32 +01:00
kevinveenbirkenbach 12704e9fa3 refactor(test-act): extract act deploy logic into dedicated scripts
- Move test-act target logic into scripts/tests/deploy/act/all.sh
- Move test-act-app target logic into scripts/tests/deploy/act/app.sh
- Keep Makefile targets as thin delegators
- Preserve existing flags and behavior

This keeps the Makefile minimal and makes act workflows reusable.

https://chatgpt.com/share/696ff4de-3abc-800f-a206-2f3dd5dec170
2026-01-20 22:34:14 +01:00
kevinveenbirkenbach adfc5123fc refactor(test-local): extract cleanup and rapid deploy into dedicated scripts
- Move test-local-cleanup logic into scripts/tests/deploy/local/cleanup.sh
- Move test-local-rapid logic into scripts/tests/deploy/local/rapid.sh
- Simplify Makefile targets to delegate into scripts
- Keep behavior and flags fully backward compatible

This improves maintainability, reuse in CI, and keeps the Makefile thin.
2026-01-20 22:32:00 +01:00
kevinveenbirkenbach 5ce577bba2 refactor(tests): run unit tests directly via docker compose exec
- execute tests inside running infinito container (no CLI wrapper)
- require INFINITO_DISTRO explicitly and show it in banner
- keep Nix accept-flake-config behavior inside container
- pass TEST_PATTERN/TEST_TYPE as container env and set workdir
- fall back to python3 if PYTHON is unset

https://chatgpt.com/share/696ff1d0-aa84-800f-82cb-05f2b1618d05
2026-01-20 22:21:09 +01:00
kevinveenbirkenbach 50816243ed Solved linting hints 2026-01-20 20:50:50 +01:00
kevinveenbirkenbach d35e932e0c refactor(cli): switch development CLI to dynamic dispatcher and fix test paths
- Replace static argparse-based CLI in cli.deploy.development with dynamic
  module dispatcher compatible with nested command discovery.
- Ensure cli.deploy.development is executable via python -m and root dispatcher.
- Fix unit test path resolution to work locally and in CI.
- Make docker-* Makefile targets depend on install.
- Simplify test runner script and keep Nix flake config compatibility.

This aligns the CLI architecture with integration test discovery
(cli/**/__main__.py) and removes the need for manual subcommand registration.

https://chatgpt.com/share/696fdbbc-b6d8-800f-a8b5-067aa7e38706
2026-01-20 20:46:20 +01:00
kevinveenbirkenbach 4e0d09e773 refactor(dev-orchestrator): centralize image build and add up --when-down SPOT
- Introduce cli.deploy.development build subcommand as single build SPOT
- Remove legacy compose.build_infinito implementation
- Make up use build-missing logic via build handler
- Add --when-down flag to up (skip startup if stack already running)
- Simplify restart/run to reuse up handler
- Ensure tests use up --when-down to avoid stack interference
- Make test targets depend on build-missing

https://chatgpt.com/share/696fd3c9-99f0-800f-a0ab-33a91058419a
2026-01-20 20:13:02 +01:00
kevinveenbirkenbach bbea43f669 refactor(ci,tests): migrate discovery and local deploy to development compose stack
- Replace legacy ci scripts with meta build-test-matrix and resolve-invokable-roles
- Switch log tailing to scripts/meta/log-tail.sh
- Rename local DNS helpers to scripts/dns/*
- Run ci-deploy-discover via development docker-up
- Replace deploy-local-full with development compose orchestrator
- Ensure INFINITO_DISTRO is passed explicitly for local full deploy

https://chatgpt.com/share/696f9ccd-e4d8-800f-9c67-9c071188ecd0
https://chatgpt.com/share/696facf5-b2fc-800f-8a33-3352dd193c54
2026-01-20 17:27:09 +01:00
kevinveenbirkenbach a87c0375ef refactor(tests): run unit tests via development compose stack and exec wrapper
- Replace docker compose run with cli.deploy.development up/exec workflow
- Reuse existing stack (coredns + infinito) with healthcheck and entry init
- Preserve NIX flake non-interactive behavior inside container
- Ensure TEST_TYPE and TEST_PATTERN are validated and injected correctly

https://chatgpt.com/share/696f9ccd-e4d8-800f-9c67-9c071188ecd0
2026-01-20 16:39:12 +01:00
kevinveenbirkenbach f52a012829 feat(cli): replace test deploy runner with development CLI (up/down/stop/restart/run)
https://chatgpt.com/share/696f9ccd-e4d8-800f-9c67-9c071188ecd0
2026-01-20 16:18:31 +01:00
kevinveenbirkenbach df31230871 diff --git a/scripts/tests/deploy-local-full.sh b/scripts/tests/deploy-local-full.sh
index 9bcfc96c8..2c5b6d930 100755
--- a/scripts/tests/deploy-local-full.sh
+++ b/scripts/tests/deploy-local-full.sh
@@ -66,54 +66,53 @@ echo
 # ---------------------------------------------------------------------------
 # 2) Run entry.sh + create inventory + deploy INSIDE container
 # ---------------------------------------------------------------------------
-docker exec -t "${INFINITO_CONTAINER}" bash -lc "
+docker exec -t "${INFINITO_CONTAINER}" bash -lc '
   set -euo pipefail
   cd /opt/src/infinito

-  echo '>>> Running entry.sh bootstrap'
+  echo ">>> Running entry.sh bootstrap"
   ./scripts/docker/entry.sh true

-  inv_dir='${INVENTORY_BASE_DIR}/local-full-${TEST_DEPLOY_TYPE}'
-  inv_file=\"\${inv_dir}/servers.yml\"
-  pw_file=\"\${inv_dir}/.password\"
+  inv_dir="'"${INVENTORY_BASE_DIR}"'/local-full-'"${TEST_DEPLOY_TYPE}"'"
+  inv_file="${inv_dir}/servers.yml"
+  pw_file="${inv_dir}/.password"

-  mkdir -p \"\${inv_dir}\"
+  mkdir -p "${inv_dir}"

-  if [[ ! -f \"\${pw_file}\" ]]; then
-    printf '%s\n' 'local-vault-password' > \"\${pw_file}\"
-    chmod 600 \"\${pw_file}\" || true
+  if [[ ! -f "${pw_file}" ]]; then
+    printf "%s\n" "local-vault-password" > "${pw_file}"
+    chmod 600 "${pw_file}" || true
   fi

-  echo '>>> Creating inventory at' \"\${inv_file}\"
-  echo '>>> Include apps count: ${apps_count}'
-  # Avoid printing the full CSV (it is huge); show first few instead:
-  echo '>>> Include sample:' \"\$(python3 - <<'PY'
-import sys
fix(tests): stabilize deploy-local-full.sh quoting and sample generation

- Fix broken heredoc usage inside docker exec bash -lc block
- Prevent apps CSV from being interpreted as a shell command
- Make sample generation robust using python -c
- Ensure --include argument is expanded correctly
- Improve errexit reliability by avoiding command substitution inside echo
- Normalize quoting for host/container variable expansion

https://chatgpt.com/share/696c928c-38a4-800f-a45b-d5f3bba908ff
2026-01-18 08:57:59 +01:00
kevinveenbirkenbach 68157002a2 ci(dns): add AAAA sanity check to prevent CoreDNS SERVFAIL regressions
- Add host-level AAAA DNS sanity test against CoreDNS
- Fail hard on SERVFAIL (breaks Node/getaddrinfo with EAI_AGAIN)
- Allow empty AAAA responses (IPv4-only setup is valid)
- Keep A-record validation unchanged

This prevents silent IPv6 resolver regressions that break CSP checks.

https://chatgpt.com/share/696abc27-19dc-800f-9ead-6598c2f3c302
2026-01-16 23:30:33 +01:00
kevinveenbirkenbach b058377e33 test(dns): make dns test script POSIX-sh compatible for Debian/Ubuntu
- Replace bash-specific [[ ... ]] with POSIX [ ... ] tests
- Use '=' instead of '==' for portable string comparisons
- Fix /bin/sh compatibility (dash) inside docker exec on Debian/Ubuntu
- Prevent DNS test failures caused by non-bash shells

https://chatgpt.com/share/696a9cb2-58e8-800f-8a88-5973d0e1bc40
2026-01-16 21:16:43 +01:00
kevinveenbirkenbach 0ff511293e Add a dedicated Docker-in-Docker DNS self-test and integrate it into the
sys-svc-docker role as a post-restart validation handler.

This change moves the inner-container DNS validation out of the CI test
suite and into the Docker service lifecycle itself, ensuring that DNS is
verified exactly where it matters: inside the Docker-in-Docker daemon.

Key changes:
- Add roles/sys-svc-docker/files/test-dns.sh:
  - Runs inside the DinD container
  - Waits for dockerd readiness
  - Executes a BusyBox test container with explicit --dns
  - Verifies DOMAIN and subdomain resolve to the expected IP
  - Fails hard if DNS is broken

- Register DNS self-test as a docker restart handler:
  - Runs once per deployment
  - Only executed when DOCKER_IN_CONTAINER and MODE_DEBUG are enabled
  - Uses ansible.builtin.script with role_path for correct resolution

- Remove inner Docker DNS test from scripts/tests/dns.sh:
  - CI test suite now only validates host → CoreDNS → outer container chain
  - DinD DNS validation is now part of the Docker service health lifecycle

This ensures that:
- DNS is validated after every Docker restart
- DinD networking failures are detected immediately
- CI and production share the same DNS validation logic
- Docker-in-Docker DNS becomes a first-class health invariant

https://chatgpt.com/share/696a3dda-5c6c-800f-8a16-47c0c604e1e3
2026-01-16 15:07:17 +01:00
kevinveenbirkenbach 35bfbf7d9c fix(dns-tests): make dns.sh shellcheck-clean and harden env validation
- Quote dig server argument to avoid word splitting (SC2086)
- Replace &&/|| pseudo-if with proper if/else blocks (SC2015)
- Validate INFINITO_CONTAINER early via required env guard
- Keep DNS test logic unchanged, only harden correctness and linting

This makes scripts/tests/dns.sh fully shellcheck -x compliant and more robust.

https://chatgpt.com/share/696a4053-93f8-800f-959b-f99e2d7d12e2
2026-01-16 14:42:34 +01:00
kevinveenbirkenbach c7de80fa70 Removed double domain 2026-01-16 14:33:21 +01:00