Commit Graph
684 Commits
Author SHA1 Message Date
kevinveenbirkenbach ecb0d6071c On Local deploy reduce repeat times for failed systemctl runenrs to debug faster 2026-01-29 11:26:59 +01:00
kevinveenbirkenbach 52ba09d165 fix(local-deploy): use TEST_DEPLOY_TYPE inventory file and fix shell expansion
https://chatgpt.com/share/697a73ea-1b34-800f-8318-be15afab2d08
2026-01-28 21:38:47 +01:00
kevinveenbirkenbach 3d3d5480d7 fix(admin): make purge_entity.sh TTY-safe and support multi-entity cleanup
- Isolate each stack purge in a subshell to prevent early termination
- Detach all docker/docker-compose calls from STDIN to avoid TTY issues inside containers
- Harden MariaDB drop/truncate logic against nested docker exec behavior
- Allow cleanup.sh to accept multiple APP values (whitespace or comma-separated)
- Deduplicate derived entities before purging and iterate deterministically

https://chatgpt.com/share/69774b3d-611c-800f-8cf0-422d11d50c1c
2026-01-26 12:08:36 +01:00
kevinveenbirkenbach f1b4436d57 Removed " to allow the test deploy of multiple apps in one step 2026-01-24 20:30:31 +01:00
kevinveenbirkenbach b2eea91fa4 Activated debug 2026-01-24 16:28:20 +01:00
kevinveenbirkenbach 51e01d0bec Introduced correct order 2026-01-24 14:10:43 +01:00
kevinveenbirkenbach 8bdf43ee87 Minor improvements 2026-01-23 11:17:53 +01:00
kevinveenbirkenbach 49246fb3d8 test(local): reset inventory before init
Rename test-local-init to test-local-reset and wipe inventory directory
before recreating it to ensure clean local test runs.

https://chatgpt.com/share/69729492-8afc-800f-99f1-823512d8604e
2026-01-22 22:20:13 +01:00
kevinveenbirkenbach 0eb7071a49 fix(local-tests): force python3 for local inventory init and stabilize runtime vars
- Force PYTHON=python3 for test-local-init to avoid missing venv python
- Rename test-local-inventory-init-all target to test-local-init
- Fix JSON quoting for --vars in local deploy scripts
- Add RUNTIME default to group_vars
- Improve debug output for initial host vars
- Update user hint in local run script

https://chatgpt.com/share/697222e1-5edc-800f-b9ab-511d8afc3142
2026-01-22 14:14:50 +01:00
kevinveenbirkenbach d2694566e5 feat(runtime): add development runtime marker and unified runtime detection
- introduce env.development marker for local dev runtime
- add cli.meta.runtime module with library API and CLI entrypoint
- detect runtime via ACT/GitHub/marker with explicit RUNTIME override
- use env.development as docker-compose env override
- pass RUNTIME=dev into development inventory generation
- add make setup-development helper

https://chatgpt.com/share/69720fd6-a1c8-800f-b1e2-6909aa63643a
2026-01-22 12:53:42 +01:00
kevinveenbirkenbach c9e05dcb6b test(local): print full app list in single line instead of sample
https://chatgpt.com/share/69710189-7df4-800f-a291-2979678ea198
2026-01-21 17:39:49 +01:00
kevinveenbirkenbach 249103bb56 feat(ci): unify local act workflow with CI discovery logic and strict input resolver
- Use docker-up + ci-discover-output for local discover job (same as CI)
- Add strict input resolver (scripts/meta/resolve/input.sh) with enforced env contract
- Ensure ONLY_APP is always defined (may be empty, but never unset)
- Expose resolved inputs as workflow outputs for linear matrix deploy
- Enforce linear matrix execution via max-parallel: 1
- Align act helper scripts with linear matrix semantics

https://chatgpt.com/share/6970ba64-8c2c-800f-bb90-7edbe9168582
2026-01-21 12:37:03 +01:00
kevinveenbirkenbach 4cf54418ca Removed --missing 2026-01-21 03:34:06 +01:00
kevinveenbirkenbach 7bf8070ef7 fix(ci): remove unsupported --no-cache/--missing flags from distros.sh
The CI distros runner now only passes arguments supported by
cli.deploy.development run. This avoids argparse errors caused by
forwarding build-only flags that the orchestrator does not accept.

https://chatgpt.com/share/697030c4-f60c-800f-94b6-9128f87da57e
2026-01-21 02:49:48 +01:00
kevinveenbirkenbach 18acafa9dc ci(images): build & push full+slim CI images
- Build matrix now includes variants: full + slim
- Tag slim images with -slim suffix and use Dockerfile target selection
- Use dedicated gha cache scopes per distro+variant
- Update release checks/retag/promote scripts to handle slim images too

https://chatgpt.com/share/69701a4e-b000-800f-be7e-162dcb93b1d2
2026-01-21 01:46:52 +01:00
kevinveenbirkenbach 0928022731 More python calls from correct context 2026-01-21 01:09:01 +01:00
kevinveenbirkenbach a565344ed5 Solved shellcheck 2026-01-21 00:25:35 +01:00
kevinveenbirkenbach f08993049c feat(inventory): add dynamic inventory resolver and propagate INVENTORY_DIR everywhere
- add scripts/inventory/resolve.sh to compute INVENTORY_DIR based on runtime context (act/github/local)
- resolve INVENTORY_DIR dynamically in Makefile via resolve.sh
- enforce INVENTORY_DIR usage across all local deploy/test scripts
- propagate INVENTORY_DIR into container and development CLI commands
- remove hardcoded /etc/inventories paths

https://chatgpt.com/share/69700d69-b270-800f-ad50-d9ff486378b9
2026-01-21 00:18:57 +01:00
kevinveenbirkenbach 6c84807778 feat(test-local): add all-app inventory init and run targets
Add two local test helpers to run full-stack deployments for all discovered apps:

- make test-local-inventory-init-all: bring up dev stack and create inventory
  (/etc/inventories/local-full-<type>/servers.yml) including all apps
- make test-local-run-all: run deploy using the pre-initialized inventory

Also mark various scripts as executable (chmod +x) to ensure consistent
behavior across environments.

https://chatgpt.com/share/6970018e-1244-800f-9345-ca1596061729
2026-01-20 23:43:10 +01:00
kevinveenbirkenbach d5801f35b0 refactor(ci): make deploy env vars strict and export pythonpath
- Make TEST_DEPLOY_TYPE strictly required in build-test-matrix
- Enforce required env vars in all act/ci/local deploy scripts
- Add deploy type validation everywhere
- Restore and export PYTHONPATH in Makefile
- Remove unused test directory variables

https://chatgpt.com/share/69700042-3fcc-800f-8bf9-b429d5bb7e82
2026-01-20 23:22:32 +01:00
kevinveenbirkenbach 12704e9fa3 refactor(test-act): extract act deploy logic into dedicated scripts
- Move test-act target logic into scripts/tests/deploy/act/all.sh
- Move test-act-app target logic into scripts/tests/deploy/act/app.sh
- Keep Makefile targets as thin delegators
- Preserve existing flags and behavior

This keeps the Makefile minimal and makes act workflows reusable.

https://chatgpt.com/share/696ff4de-3abc-800f-a206-2f3dd5dec170
2026-01-20 22:34:14 +01:00
kevinveenbirkenbach adfc5123fc refactor(test-local): extract cleanup and rapid deploy into dedicated scripts
- Move test-local-cleanup logic into scripts/tests/deploy/local/cleanup.sh
- Move test-local-rapid logic into scripts/tests/deploy/local/rapid.sh
- Simplify Makefile targets to delegate into scripts
- Keep behavior and flags fully backward compatible

This improves maintainability, reuse in CI, and keeps the Makefile thin.
2026-01-20 22:32:00 +01:00
kevinveenbirkenbach 5ce577bba2 refactor(tests): run unit tests directly via docker compose exec
- execute tests inside running infinito container (no CLI wrapper)
- require INFINITO_DISTRO explicitly and show it in banner
- keep Nix accept-flake-config behavior inside container
- pass TEST_PATTERN/TEST_TYPE as container env and set workdir
- fall back to python3 if PYTHON is unset

https://chatgpt.com/share/696ff1d0-aa84-800f-82cb-05f2b1618d05
2026-01-20 22:21:09 +01:00
kevinveenbirkenbach 50816243ed Solved linting hints 2026-01-20 20:50:50 +01:00
kevinveenbirkenbach d35e932e0c refactor(cli): switch development CLI to dynamic dispatcher and fix test paths
- Replace static argparse-based CLI in cli.deploy.development with dynamic
  module dispatcher compatible with nested command discovery.
- Ensure cli.deploy.development is executable via python -m and root dispatcher.
- Fix unit test path resolution to work locally and in CI.
- Make docker-* Makefile targets depend on install.
- Simplify test runner script and keep Nix flake config compatibility.

This aligns the CLI architecture with integration test discovery
(cli/**/__main__.py) and removes the need for manual subcommand registration.

https://chatgpt.com/share/696fdbbc-b6d8-800f-a8b5-067aa7e38706
2026-01-20 20:46:20 +01:00
kevinveenbirkenbach 4e0d09e773 refactor(dev-orchestrator): centralize image build and add up --when-down SPOT
- Introduce cli.deploy.development build subcommand as single build SPOT
- Remove legacy compose.build_infinito implementation
- Make up use build-missing logic via build handler
- Add --when-down flag to up (skip startup if stack already running)
- Simplify restart/run to reuse up handler
- Ensure tests use up --when-down to avoid stack interference
- Make test targets depend on build-missing

https://chatgpt.com/share/696fd3c9-99f0-800f-a0ab-33a91058419a
2026-01-20 20:13:02 +01:00
kevinveenbirkenbach bbea43f669 refactor(ci,tests): migrate discovery and local deploy to development compose stack
- Replace legacy ci scripts with meta build-test-matrix and resolve-invokable-roles
- Switch log tailing to scripts/meta/log-tail.sh
- Rename local DNS helpers to scripts/dns/*
- Run ci-deploy-discover via development docker-up
- Replace deploy-local-full with development compose orchestrator
- Ensure INFINITO_DISTRO is passed explicitly for local full deploy

https://chatgpt.com/share/696f9ccd-e4d8-800f-9c67-9c071188ecd0
https://chatgpt.com/share/696facf5-b2fc-800f-8a33-3352dd193c54
2026-01-20 17:27:09 +01:00
kevinveenbirkenbach a87c0375ef refactor(tests): run unit tests via development compose stack and exec wrapper
- Replace docker compose run with cli.deploy.development up/exec workflow
- Reuse existing stack (coredns + infinito) with healthcheck and entry init
- Preserve NIX flake non-interactive behavior inside container
- Ensure TEST_TYPE and TEST_PATTERN are validated and injected correctly

https://chatgpt.com/share/696f9ccd-e4d8-800f-9c67-9c071188ecd0
2026-01-20 16:39:12 +01:00
kevinveenbirkenbach f52a012829 feat(cli): replace test deploy runner with development CLI (up/down/stop/restart/run)
https://chatgpt.com/share/696f9ccd-e4d8-800f-9c67-9c071188ecd0
2026-01-20 16:18:31 +01:00
kevinveenbirkenbach df31230871 diff --git a/scripts/tests/deploy-local-full.sh b/scripts/tests/deploy-local-full.sh
index 9bcfc96c8..2c5b6d930 100755
--- a/scripts/tests/deploy-local-full.sh
+++ b/scripts/tests/deploy-local-full.sh
@@ -66,54 +66,53 @@ echo
 # ---------------------------------------------------------------------------
 # 2) Run entry.sh + create inventory + deploy INSIDE container
 # ---------------------------------------------------------------------------
-docker exec -t "${INFINITO_CONTAINER}" bash -lc "
+docker exec -t "${INFINITO_CONTAINER}" bash -lc '
   set -euo pipefail
   cd /opt/src/infinito

-  echo '>>> Running entry.sh bootstrap'
+  echo ">>> Running entry.sh bootstrap"
   ./scripts/docker/entry.sh true

-  inv_dir='${INVENTORY_BASE_DIR}/local-full-${TEST_DEPLOY_TYPE}'
-  inv_file=\"\${inv_dir}/servers.yml\"
-  pw_file=\"\${inv_dir}/.password\"
+  inv_dir="'"${INVENTORY_BASE_DIR}"'/local-full-'"${TEST_DEPLOY_TYPE}"'"
+  inv_file="${inv_dir}/servers.yml"
+  pw_file="${inv_dir}/.password"

-  mkdir -p \"\${inv_dir}\"
+  mkdir -p "${inv_dir}"

-  if [[ ! -f \"\${pw_file}\" ]]; then
-    printf '%s\n' 'local-vault-password' > \"\${pw_file}\"
-    chmod 600 \"\${pw_file}\" || true
+  if [[ ! -f "${pw_file}" ]]; then
+    printf "%s\n" "local-vault-password" > "${pw_file}"
+    chmod 600 "${pw_file}" || true
   fi

-  echo '>>> Creating inventory at' \"\${inv_file}\"
-  echo '>>> Include apps count: ${apps_count}'
-  # Avoid printing the full CSV (it is huge); show first few instead:
-  echo '>>> Include sample:' \"\$(python3 - <<'PY'
-import sys
fix(tests): stabilize deploy-local-full.sh quoting and sample generation

- Fix broken heredoc usage inside docker exec bash -lc block
- Prevent apps CSV from being interpreted as a shell command
- Make sample generation robust using python -c
- Ensure --include argument is expanded correctly
- Improve errexit reliability by avoiding command substitution inside echo
- Normalize quoting for host/container variable expansion

https://chatgpt.com/share/696c928c-38a4-800f-a45b-d5f3bba908ff
2026-01-18 08:57:59 +01:00
kevinveenbirkenbach 68157002a2 ci(dns): add AAAA sanity check to prevent CoreDNS SERVFAIL regressions
- Add host-level AAAA DNS sanity test against CoreDNS
- Fail hard on SERVFAIL (breaks Node/getaddrinfo with EAI_AGAIN)
- Allow empty AAAA responses (IPv4-only setup is valid)
- Keep A-record validation unchanged

This prevents silent IPv6 resolver regressions that break CSP checks.

https://chatgpt.com/share/696abc27-19dc-800f-9ead-6598c2f3c302
2026-01-16 23:30:33 +01:00
kevinveenbirkenbach b058377e33 test(dns): make dns test script POSIX-sh compatible for Debian/Ubuntu
- Replace bash-specific [[ ... ]] with POSIX [ ... ] tests
- Use '=' instead of '==' for portable string comparisons
- Fix /bin/sh compatibility (dash) inside docker exec on Debian/Ubuntu
- Prevent DNS test failures caused by non-bash shells

https://chatgpt.com/share/696a9cb2-58e8-800f-8a88-5973d0e1bc40
2026-01-16 21:16:43 +01:00
kevinveenbirkenbach 0ff511293e Add a dedicated Docker-in-Docker DNS self-test and integrate it into the
sys-svc-docker role as a post-restart validation handler.

This change moves the inner-container DNS validation out of the CI test
suite and into the Docker service lifecycle itself, ensuring that DNS is
verified exactly where it matters: inside the Docker-in-Docker daemon.

Key changes:
- Add roles/sys-svc-docker/files/test-dns.sh:
  - Runs inside the DinD container
  - Waits for dockerd readiness
  - Executes a BusyBox test container with explicit --dns
  - Verifies DOMAIN and subdomain resolve to the expected IP
  - Fails hard if DNS is broken

- Register DNS self-test as a docker restart handler:
  - Runs once per deployment
  - Only executed when DOCKER_IN_CONTAINER and MODE_DEBUG are enabled
  - Uses ansible.builtin.script with role_path for correct resolution

- Remove inner Docker DNS test from scripts/tests/dns.sh:
  - CI test suite now only validates host → CoreDNS → outer container chain
  - DinD DNS validation is now part of the Docker service health lifecycle

This ensures that:
- DNS is validated after every Docker restart
- DinD networking failures are detected immediately
- CI and production share the same DNS validation logic
- Docker-in-Docker DNS becomes a first-class health invariant

https://chatgpt.com/share/696a3dda-5c6c-800f-8a16-47c0c604e1e3
2026-01-16 15:07:17 +01:00
kevinveenbirkenbach 35bfbf7d9c fix(dns-tests): make dns.sh shellcheck-clean and harden env validation
- Quote dig server argument to avoid word splitting (SC2086)
- Replace &&/|| pseudo-if with proper if/else blocks (SC2015)
- Validate INFINITO_CONTAINER early via required env guard
- Keep DNS test logic unchanged, only harden correctness and linting

This makes scripts/tests/dns.sh fully shellcheck -x compliant and more robust.

https://chatgpt.com/share/696a4053-93f8-800f-959b-f99e2d7d12e2
2026-01-16 14:42:34 +01:00
kevinveenbirkenbach c7de80fa70 Removed double domain 2026-01-16 14:33:21 +01:00
kevinveenbirkenbach fd983bc69c refactor(ci,network): make DNS, IP and domain configuration fully env-driven
- Introduce env.ci as single source of truth for CI networking
- Make docker-compose, CoreDNS and Docker daemon fully env-driven
- Render CoreDNS Corefile from template via envsubst
- Replace hardcoded IPs/domains with environment variables
- Integrate env-based networking into Ansible defaults
- Remove static Corefile and daemon.json

This makes DNS and networking deterministic and reproducible across
host, CoreDNS, infinito runner and Docker-in-Docker.

https://chatgpt.com/share/696a3dda-5c6c-800f-8a16-47c0c604e1e3
2026-01-16 14:30:14 +01:00
kevinveenbirkenbach 3a496bcfff Optimized deploy parameters 2026-01-16 01:27:09 +01:00
kevinveenbirkenbach bcc638fe0a dev(local): add full local deploy helper and rename act/rapid targets
- Rename act-based targets for clarity:
  - test-deploy -> test-act
  - test-deploy-app -> test-act-app
- Rename rapid local deploy target:
  - test-deploy-rapid -> test-local-rapid
- Add test-local-full target to run a full local deploy against a generated
  inventory containing all apps of the selected TEST_DEPLOY_TYPE
- Add scripts/tests/deploy-local-full.sh:
  - Discover apps on host via make ci-deploy-discover
  - Bootstrap container via scripts/docker/entry.sh
  - Generate dedicated inventory with --include list
  - Deploy against that inventory

https://chatgpt.com/share/69692ef1-095c-800f-83cb-924fadfaa554
2026-01-15 20:05:08 +01:00
kevinveenbirkenbach a323f2199d optimized code tests 2026-01-15 02:07:31 +01:00
kevinveenbirkenbach 8ea56548d1 Optimized INFINITO_BUILD_LOCAL 2026-01-15 01:53:32 +01:00
kevinveenbirkenbach 517ccf44c0 ci: replace rest deploy mode with universal and add universal stage
- Rename CI workflow and MODE from "rest" to "universal"
- Extend CLI and test runner to support universal deploy type
- Add tasks/stages/02_universal.yml for universal host type
- Update discover logic to use universal app selection
- Remove legacy rest->server mapping and obsolete select helper

https://chatgpt.com/share/6967625e-81c8-800f-8592-5d5b8edec1e2
2026-01-14 10:31:06 +01:00
kevinveenbirkenbach 3cf7d20d86 feat(test-deploy): add debug toggle and rapid deploy target
- add --debug / --no-debug flag to deploy test CLI and runner
- make Ansible debug mode optional instead of always enabled
- extend deploy.sh with debug flag forwarding
- add make target test-deploy-rapid for fast single-app deploys

https://chatgpt.com/share/6966c80c-661c-800f-8fb9-769659da290d
2026-01-13 23:32:37 +01:00
kevinveenbirkenbach 3329fbdfd5 ci(act): unify deploy matrix + force serial execution with max-parallel
- Add reusable _deploy-matrix workflow with discovery + per-app deploy across distros
- Enforce serial execution via strategy.max-parallel=1 and act --concurrent-jobs 1
- Support filters: mode (server|workstation|rest), only_app, only_distro, include/exclude regex
- Refactor test runner: remove discovery, require --app, deploy single app with deps only
- Generalize discover script to discover-apps.sh with INCLUDE_RE/EXCLUDE_RE + lifecycle gating
- Update Makefile targets to drive workflow_dispatch inputs for local act runs

https://chatgpt.com/share/69659ff3-1414-800f-8652-5298fcf55398
2026-01-13 03:06:56 +01:00
kevinveenbirkenbach 54a027ad8c test(ci): apply lifecycle gating during server app discovery
Move lifecycle filtering into discover-server-apps.sh so only
lifecycle-eligible web-app-* and web-svc-* roles are discovered.

This prevents invalid per-app deploy tests where applications are
excluded later during inventory generation, causing "not present in
inventory" errors.

- Filter invokable roles by server prefixes
- Apply lifecycle_filter blacklist using TESTED_LIFECYCLES
- Always emit valid JSON for CI matrix generation

https://chatgpt.com/share/695fd936-0ee4-800f-a57f-2d7995e15ba0
2026-01-08 17:19:53 +01:00
kevinveenbirkenbach fab08aadf6 refactor(ci): split deploy tests into per-app server matrix and workstation workflow
- Replace shared test-deploy workflow with dedicated server/workstation workflows
- Add discovery step to generate per-app server matrix from invokable roles
- Introduce helper script to output discovered server apps as JSON
- Keep compose stack cleanup robust via trap and always upload per-run logs

https://chatgpt.com/share/695f9e22-6e9c-800f-9a50-65f553619af2
2026-01-08 13:30:07 +01:00
kevinveenbirkenbach dba121ecf4 refactor(ci): move deploy test runner from bash to python orchestrator
* Add `cli.deploy.test` module to orchestrate compose-based CI deploy tests
* Support server per-app runs (including run_after dependencies) and workstation selection
* Add optional image build flags (--no-cache, --missing) and stack retention on failure
* Simplify `scripts/tests/deploy.sh` to a thin wrapper that forwards args to the Python runner

https://chatgpt.com/share/695f9e22-6e9c-800f-9a50-65f553619af2
2026-01-08 13:07:57 +01:00
kevinveenbirkenbach 964a0070c7 fix(ci): fix printf argument mismatch in deploy test script
ShellCheck SC2183: add missing format specifier for ALWAYS_EXCLUDE

https://chatgpt.com/share/695d9599-94a8-800f-8e82-979345be5384
2026-01-07 00:06:56 +01:00
kevinveenbirkenbach 8cd0c941f0 **ci(deploy): gate CI deploys by tested lifecycle**
* Introduce lifecycle gating to CI deploy tests
* Only allow roles with tested lifecycles (alpha, beta, rc, stable)
* Automatically exclude roles with untested or missing lifecycle metadata
* Simplify static BASE_EXCLUDE by relying on lifecycle filtering

https://chatgpt.com/share/695d16d9-e210-800f-92d7-a1a429644be2
2026-01-06 15:06:03 +01:00
kevinveenbirkenbach 5598da06d2 tests: run code tests via docker compose and drop unused deploy keys
https://chatgpt.com/share/69550786-4bfc-800f-b34f-27af9b7adc04
2025-12-31 12:22:41 +01:00
kevinveenbirkenbach 164dd4001b fix(ci,compose): use compose-only runner with host cgroups and clean stack reset
- switch deploy tests to docker compose exec (-T) instead of docker exec
- remove hard dependency on INFINITO_CONTAINER
- ensure clean CI state via compose down -v before deploy tests
- start coredns + infinito exclusively via compose
- enable host cgroup namespace for systemd containers (cgroup: host)

https://chatgpt.com/share/6953e367-b674-800f-bc55-97e8625314cf
2025-12-30 15:36:16 +01:00