Commit Graph
65 Commits
Author SHA1 Message Date
kevinveenbirkenbach a323f2199d optimized code tests 2026-01-15 02:07:31 +01:00
kevinveenbirkenbach 8ea56548d1 Optimized INFINITO_BUILD_LOCAL 2026-01-15 01:53:32 +01:00
kevinveenbirkenbach 97642e949c ci(release): switch to immutable ci-<sha> image tags and retag on release
- Build CI images with immutable ci-<sha> tags instead of latest
- Pass CI image tag into deploy workflows via INFINITO_IMAGE_TAG
- Derive CI_TAG from checked-out release tag commit
- Retag CI images to <version> and latest on release (no rebuild)
- Make release scripts independent from GITHUB_SHA coupling
- Improve robustness and reproducibility of release pipeline

https://chatgpt.com/share/696832f6-35e4-800f-a574-178013d4426e
2026-01-15 01:20:39 +01:00
kevinveenbirkenbach 510197cc1f ci: always keep stack on failure in deploy-app workflow
Remove KEEP_STACK env handling and always pass --keep-stack-on-failure
to deploy.sh to preserve containers and logs for debugging after failed
CI runs.

https://chatgpt.com/share/6967a413-7a24-800f-8837-514133a57072
2026-01-14 15:11:23 +01:00
kevinveenbirkenbach 0cad360d29 ci: refactor deploy workflows and make CI build behaviour explicit
- Replace legacy MODE env with TEST_DEPLOY_TYPE across workflows and scripts
- Route CI deploy and discovery through Makefile targets
- Rename deploy-one-app.sh to deploy-app.sh and update all call sites
- Fix Makefile recursion and ensure scripts are executed directly
- Ensure JSON discovery output is clean via silent make target
- Make MISSING_ONLY default true via Makefile
- Respect INFINITO_NO_BUILD in compose up to avoid implicit rebuilds in CI
- Keep explicit build step before compose up for deterministic CI runs
- Improve deploy logging (test_deploy_type instead of mode)

Result:
- Deterministic CI behaviour
- No accidental docker compose rebuilds on up
- Single source of truth for deploy orchestration
- Cleaner env contract between workflows, make, bash and python

https://chatgpt.com/share/69678024-fe38-800f-b877-add46aa55190
2026-01-14 12:37:41 +01:00
kevinveenbirkenbach 517ccf44c0 ci: replace rest deploy mode with universal and add universal stage
- Rename CI workflow and MODE from "rest" to "universal"
- Extend CLI and test runner to support universal deploy type
- Add tasks/stages/02_universal.yml for universal host type
- Update discover logic to use universal app selection
- Remove legacy rest->server mapping and obsolete select helper

https://chatgpt.com/share/6967625e-81c8-800f-8592-5d5b8edec1e2
2026-01-14 10:31:06 +01:00
kevinveenbirkenbach e63f18a403 ci(deploy): keep stack only when running under act
- Default KEEP_STACK to true only for act runs
- Disable KEEP_STACK in GitHub Actions to prevent runner disk exhaustion
- Remove explicit KEEP_STACK overrides from deploy workflows

https://chatgpt.com/share/696752fb-befc-800f-9cf1-57093178297a
2026-01-14 09:35:01 +01:00
kevinveenbirkenbach 3cf7d20d86 feat(test-deploy): add debug toggle and rapid deploy target
- add --debug / --no-debug flag to deploy test CLI and runner
- make Ansible debug mode optional instead of always enabled
- extend deploy.sh with debug flag forwarding
- add make target test-deploy-rapid for fast single-app deploys

https://chatgpt.com/share/6966c80c-661c-800f-8fb9-769659da290d
2026-01-13 23:32:37 +01:00
kevinveenbirkenbach 6b7b67681e fix(ci): unblock systemd startup and stabilize Docker healthcheck
Ensure systemd is started immediately as PID 1 in the infinito container
to avoid blocking on pkgmgr during boot. Increase compose runner timeout
to match healthcheck window and make healthcheck script executable in the
image.

This fixes containers getting stuck in "starting" state and makes CI
startup deterministic.

https://chatgpt.com/share/696683a2-9350-800f-a0cf-587d4013cb29
2026-01-13 19:02:43 +01:00
kevinveenbirkenbach 8c82db3556 feat(ci): add Docker healthcheck and wait for healthy container in compose runner
Add a systemd-based Docker HEALTHCHECK that verifies:
- systemd readiness
- infinito CLI availability and executability
- infinito CLI smoke test (--help)

Update CI compose runner to wait for Docker health status instead of
manual systemd polling. The runner now:
- resolves the infinito container via docker compose
- waits for healthy state
- dumps journalctl and docker logs on timeout

This removes boot race conditions and makes container readiness deterministic.

https://chatgpt.com/share/696683a2-9350-800f-a0cf-587d4013cb29
2026-01-13 18:40:33 +01:00
kevinveenbirkenbach 1086e573c3 refactor(ci): move release helper scripts into scripts/release and drop legacy publish helpers
- Update release workflow to use new script paths
- Remove deprecated build publish/retag-stable scripts
- Fix minor comment typo

https://chatgpt.com/share/69662b40-6938-800f-8ab5-c85424e13db4
2026-01-13 13:36:17 +01:00
kevinveenbirkenbach 392d15107b ci(release): build CI images, retag on release, gate stable on tests
- Add reusable build-ci-images workflow to build and push ci-<sha> images for all distros
- Make all deploy test workflows use prebuilt CI images (no local rebuilds)
- Replace ci-version workflow with release workflow
- Retag ci images to <version> and latest on tag push (no rebuild)
- Move stable git tag only if all analyses/tests succeed and version is highest
- Make compose image/pull policy configurable via env

https://chatgpt.com/share/69662869-ee1c-800f-9e6c-bbbf47284fad
2026-01-13 12:11:33 +01:00
kevinveenbirkenbach 56d211e60d ci(logs): centralize failure log tailing into reusable script
Replace inline tail logic in deploy workflows with a shared
scripts/ci/log-tail.sh helper that prints the last N lines of all
log files on failure.

This improves debuggability, avoids code duplication, and ensures
consistent log output across server, universal and workstation
pipelines.

https://chatgpt.com/share/69661d0e-81a8-800f-8b98-6c964672d027
2026-01-13 11:23:02 +01:00
kevinveenbirkenbach ea6cccfae7 Solved shellchecker warning 2026-01-13 05:46:54 +01:00
kevinveenbirkenbach bdaac5cc90 ci(deploy): split per-app deploy into server/workstation/rest workflows (env-only defaults)
- Remove _deploy-per-app.yml (inputs-based indirection)
- Add dedicated workflows:
  - test-deploy-server.yml
  - test-deploy-workstation.yml
  - test-deploy-universal.yml (rest)
- Configure via env defaults only (KISS / SRP / SPOT)
- Update Makefile to target the new workflows and pass overrides via env
- Ensure deploy-one-app.sh is executable
- Keep failure debug (last 200 chars) and upload logs only outside act
2026-01-13 05:23:47 +01:00
kevinveenbirkenbach dac111ccb7 ci(deploy): simplify per-app deploy workflow and Makefile act targets
- Rename _deploy-matrix.yml -> _deploy-per-app.yml and drop unused inputs
  (tested_lifecycles, only_distro, final_exclude_re).
- Keep one runner per app (matrix.app) and run all distros inside that job.
- Makefile: run deploy tests through act workflow_dispatch (no serial script).
- Add scripts/ci/discover.sh and scripts/ci/deploy-one-app.sh as small SRP helpers.
- Update reusable workflow callers to use the renamed file.

https://chatgpt.com/share/69659ff3-1414-800f-8652-5298fcf55398
2026-01-13 03:41:25 +01:00
kevinveenbirkenbach 3329fbdfd5 ci(act): unify deploy matrix + force serial execution with max-parallel
- Add reusable _deploy-matrix workflow with discovery + per-app deploy across distros
- Enforce serial execution via strategy.max-parallel=1 and act --concurrent-jobs 1
- Support filters: mode (server|workstation|rest), only_app, only_distro, include/exclude regex
- Refactor test runner: remove discovery, require --app, deploy single app with deps only
- Generalize discover script to discover-apps.sh with INCLUDE_RE/EXCLUDE_RE + lifecycle gating
- Update Makefile targets to drive workflow_dispatch inputs for local act runs

https://chatgpt.com/share/69659ff3-1414-800f-8652-5298fcf55398
2026-01-13 03:06:56 +01:00
kevinveenbirkenbach 54a027ad8c test(ci): apply lifecycle gating during server app discovery
Move lifecycle filtering into discover-server-apps.sh so only
lifecycle-eligible web-app-* and web-svc-* roles are discovered.

This prevents invalid per-app deploy tests where applications are
excluded later during inventory generation, causing "not present in
inventory" errors.

- Filter invokable roles by server prefixes
- Apply lifecycle_filter blacklist using TESTED_LIFECYCLES
- Always emit valid JSON for CI matrix generation

https://chatgpt.com/share/695fd936-0ee4-800f-a57f-2d7995e15ba0
2026-01-08 17:19:53 +01:00
kevinveenbirkenbach fab08aadf6 refactor(ci): split deploy tests into per-app server matrix and workstation workflow
- Replace shared test-deploy workflow with dedicated server/workstation workflows
- Add discovery step to generate per-app server matrix from invokable roles
- Introduce helper script to output discovered server apps as JSON
- Keep compose stack cleanup robust via trap and always upload per-run logs

https://chatgpt.com/share/695f9e22-6e9c-800f-9a50-65f553619af2
2026-01-08 13:30:07 +01:00
kevinveenbirkenbach dba121ecf4 refactor(ci): move deploy test runner from bash to python orchestrator
* Add `cli.deploy.test` module to orchestrate compose-based CI deploy tests
* Support server per-app runs (including run_after dependencies) and workstation selection
* Add optional image build flags (--no-cache, --missing) and stack retention on failure
* Simplify `scripts/tests/deploy.sh` to a thin wrapper that forwards args to the Python runner

https://chatgpt.com/share/695f9e22-6e9c-800f-9a50-65f553619af2
2026-01-08 13:07:57 +01:00
kevinveenbirkenbach 964a0070c7 fix(ci): fix printf argument mismatch in deploy test script
ShellCheck SC2183: add missing format specifier for ALWAYS_EXCLUDE

https://chatgpt.com/share/695d9599-94a8-800f-8e82-979345be5384
2026-01-07 00:06:56 +01:00
kevinveenbirkenbach f3fa8d7680 fix(ci): avoid git ownership issues by syncing source as root
- Replace clean-container with compose down for predictable teardown
- Add explicit up/down targets to Makefile
- Sync bind-mounted source into internal repo using rsync --chown=root:root
- Prevent git dubious ownership errors in CI containers running as root

https://chatgpt.com/share/695d7045-a1f8-800f-83d0-f5872d85cfcf
2026-01-06 21:27:43 +01:00
kevinveenbirkenbach 8cd0c941f0 **ci(deploy): gate CI deploys by tested lifecycle**
* Introduce lifecycle gating to CI deploy tests
* Only allow roles with tested lifecycles (alpha, beta, rc, stable)
* Automatically exclude roles with untested or missing lifecycle metadata
* Simplify static BASE_EXCLUDE by relying on lifecycle filtering

https://chatgpt.com/share/695d16d9-e210-800f-92d7-a1a429644be2
2026-01-06 15:06:03 +01:00
kevinveenbirkenbach 6936d84798 ci: always build & publish images for version tags, gate stable on tests
- Always build and publish Docker images for v* tags (version + latest),
  independent of test results
- Run full code analyses in parallel and only mark Git + container :stable
  when all checks succeed
- Replace rebuild-on-stable with pure imagetools retagging (no rebuild)
- Prevent race conditions by ordering stable tagging after image publish
- Ensure latest always tracks the most recent version image

https://chatgpt.com/share/695bdfa5-dbe0-800f-bb1c-4fe81eba6058
2026-01-05 16:58:21 +01:00
kevinveenbirkenbach 5598da06d2 tests: run code tests via docker compose and drop unused deploy keys
https://chatgpt.com/share/69550786-4bfc-800f-b34f-27af9b7adc04
2025-12-31 12:22:41 +01:00
kevinveenbirkenbach 164dd4001b fix(ci,compose): use compose-only runner with host cgroups and clean stack reset
- switch deploy tests to docker compose exec (-T) instead of docker exec
- remove hard dependency on INFINITO_CONTAINER
- ensure clean CI state via compose down -v before deploy tests
- start coredns + infinito exclusively via compose
- enable host cgroup namespace for systemd containers (cgroup: host)

https://chatgpt.com/share/6953e367-b674-800f-bc55-97e8625314cf
2025-12-30 15:36:16 +01:00
kevinveenbirkenbach 4bd4c11e68 fix(ci): reuse existing deploy test container to avoid name conflicts
* Prevent docker run name conflicts when a previous CI run left the container behind
* Reuse and start the existing deploy test container instead of creating a new one

https://chatgpt.com/share/6953d93c-a24c-800f-9334-40c34f768f9f
2025-12-30 14:51:26 +01:00
kevinveenbirkenbach c825dffbf7 ci(deploy): unify test-deploy entrypoint via Makefile and INFINITO_DISTRO
- switch workflow to `make test-deploy` with matrix-driven INFINITO_DISTRO
- centralize container naming via INFINITO_CONTAINER
- add clean-container helper for reliable CI reruns
- remove --distro CLI flag and rely on env-based distro selection
- pass NIX_CONFIG as docker build arg for rate-limit-safe builds

https://chatgpt.com/share/6953d266-aa80-800f-88b8-0413926a81f0
2025-12-30 14:23:44 +01:00
kevinveenbirkenbach 6bc097f96f refactor(email,domain): unify primary domain and TLS handling
- replace localhost special-casing with valid FQDN (infinito.localhost)
- derive system email DOMAIN and HOST directly from DOMAIN_PRIMARY
- align Mailu TLS_FLAVOR with SYSTEM_EMAIL.TLS (mail vs notls)
- tie OIDC_VERIFY_SSL to TLS_ENABLED for consistent test/CI behavior
- update CI deploy script to use new primary domain

https://chatgpt.com/share/695311c0-fbe8-800f-8058-9d40a06b189a
2025-12-30 00:41:46 +01:00
kevinveenbirkenbach 2f41b400d9 Added --diff to e2e deploy tests to see what gets deployed 2025-12-29 23:24:35 +01:00
kevinveenbirkenbach 5175aefaf8 Renamed desktop to workstation 2025-12-29 22:44:20 +01:00
kevinveenbirkenbach 1e5ebbb05f feat(inventory): add --vars-file support for YAML-based host_vars overrides
- add --vars-file CLI option to load host_vars overrides from YAML
- merge vars-file overrides before JSON --vars (JSON takes precedence)
- implement deep-merge logic for YAML vars files
- update CI deploy script to use vars file instead of inline JSON
- add unit test for apply_vars_overrides_from_file
- remove obsolete MASK_CREDENTIALS_IN_LOGS TODO entry

https://chatgpt.com/share/6952e51d-6c50-800f-8764-31dc8c609cdb
2025-12-29 21:31:05 +01:00
kevinveenbirkenbach d734a15ad6 feat(ci,webserver): add localhost DNS + default vhost 404 assertion
- switch CI DNS to CoreDNS-based *.localhost resolution (A/AAAA → loopback)
- set PRIMARY_DOMAIN to localhost for CI deploys
- add explicit DNS resolution assert for test.localhost
- deploy explicit default_server vhost returning 404 text/plain
- assert default vhost behavior via HTTP before app domains exist

https://chatgpt.com/share/6952cf2a-bbd4-800f-b2b8-c4baba66e301
2025-12-29 19:57:41 +01:00
kevinveenbirkenbach a9e1a84b71 feat(ci): run Docker-in-Docker in deploy container via fuse-overlayfs
* Remove host Docker socket (/var/run/docker.sock) and DOCKER_HOST wiring for CI container
* Drop /tmp/gh-action bind usage in CI container + compose runner
* Switch OpenResty volume sources to container-local absolute paths (/var/www, /etc/nginx)
* Ensure docker engine installation also runs when DOCKER_IN_CONTAINER is true
* Add container-only fuse-overlayfs setup and write /etc/docker/daemon.json storage-driver config

https://chatgpt.com/share/69528062-ff74-800f-ae11-a658b5917307
2025-12-29 17:47:03 +01:00
kevinveenbirkenbach d61f11f521 fix(ci): mount /tmp/gh-action into infinito container for compose bind mounts
- ensure files created by Ansible under /tmp/gh-action are visible to Docker host
- prevent file-vs-directory bind mount errors in GitHub CI
- align container filesystem with docker compose host paths

https://chatgpt.com/share/69528062-ff74-800f-ae11-a658b5917307
2025-12-29 14:49:33 +01:00
kevinveenbirkenbach 789ba5a247 - Make Nix non-interactive by default via NIX_CONFIG (accept-flake-config)
- Restore docker service handling outside containers, validate inside containers
- Simplify entry.sh: keep original rebuild/copy logic, remove systemd branching
- Switch CI deploy runner:
  - start only CoreDNS via docker compose
  - start infinito via docker run with systemd, cgroup host access and bind mounts
- Avoid compose limitations (no cgroupns=host support)
- Ensure deterministic container naming for exec-based CI operations

https://chatgpt.com/share/6950110b-111c-800f-bbcd-83a551d5a3b8
2025-12-27 20:47:50 +01:00
kevinveenbirkenbach b207134472 feat(ci): enable systemd-in-docker and use explicit container detection
- Install systemd and dbus in Docker image for CI service tests
- Run systemd as PID 1 with proper container conventions
- Mark container environment explicitly via ENV container=docker
- Simplify container detection to rely on explicit env flag
- Skip host-only tasks (timezone, systemctl handlers) inside containers
- Adjust docker-compose runner for privileged systemd execution
- Reduce CI parallelism to avoid OOM kills

This makes systemd-based Ansible roles reliable in Docker/CI
and avoids fragile virtualization heuristics.

https://chatgpt.com/share/6950110b-111c-800f-bbcd-83a551d5a3b8
2025-12-27 18:14:28 +01:00
kevinveenbirkenbach abd626b4a7 svc-prx-openresty: add MODE_DEBUG wait loop and always dump logs on failure
- wait for container running/healthy in MODE_DEBUG
- dump container status + logs in always block for easier debugging
- enable --debug in deploy test script

https://chatgpt.com/share/694d897b-1fb8-800f-ae7b-a3100cfb3b90
2025-12-25 19:59:03 +01:00
kevinveenbirkenbach 65b1f3f2cd Set --primary-domain docker.test for test-deploys 2025-12-25 15:26:10 +01:00
kevinveenbirkenbach 3e95649a52 ci(compose): keep infinito container alive for exec-based deploy tests
Switch CI deploy tests from docker run to docker compose with a long-lived
infinito service. The container now uses `tail -f /dev/null` as command to
prevent restart loops caused by the default `infinito --help` CMD, which
previously killed exec processes with exit code 137.

Changes include:
- make infinito a stable exec target in compose (no premature restarts)
- add compose helpers (build/up/exec/down) to deploy.sh
- move inventory creation and dedicated deploy into the running container
- add conditional cleanup via EXIT trap (keep stack on failure for debugging)
- support per-distro container naming via INFINITO_DISTRO

This stabilizes CI runs and makes debugging failed deploys possible.

https://chatgpt.com/share/694d3f99-07e4-800f-bc05-3a60c902064d
2025-12-25 14:43:45 +01:00
kevinveenbirkenbach 54969b1f9d test(deploy): force unmasked logs for local debug runs
Set MASK_CREDENTIALS_IN_LOGS=false explicitly in deploy test script
to ensure bootstrap errors are visible during local debugging.

https://chatgpt.com/share/694bd2bf-353c-800f-a0cc-0e4d9f2e550f
2025-12-24 12:59:34 +01:00
kevinveenbirkenbach 46baaacba9 Enabled svc-db- credentials for servers 2025-12-22 11:57:53 +01:00
kevinveenbirkenbach ffd071c64e ci(openresty): share gh-action runtime and scope nginx/www volumes per CI run
- Mount /tmp/gh-action into the CI container so Docker can access run-scoped paths
- Scope OpenResty www and nginx volumes to GITHUB_RUN_ID when running in container
- Avoid leaking host /etc/nginx and /var/www into CI runs
- Remove redundant preflight

https://chatgpt.com/share/694918d0-53b4-800f-93a7-ec82946f3e24
2025-12-22 11:09:12 +01:00
kevinveenbirkenbach c61db9f9e0 feat(ci): silence INSTALL_LOCAL_BUILD output during invokable discovery
* Refactor docker entrypoint to run local build via a single helper function
* Add INSTALL_LOCAL_BUILD_SILENCE flag to fully suppress DEV build output
* Enable INSTALL_LOCAL_BUILD for invokable resolution without noisy logs
* Keep behavior deterministic and non-redundant while preserving failure semantics

https://chatgpt.com/share/694874d7-508c-800f-aa89-d9cef71128dc
2025-12-21 23:29:39 +01:00
kevinveenbirkenbach d85da02db1 Disabled INSTALL_LOCAL_BUILD due to noise 2025-12-21 23:14:48 +01:00
kevinveenbirkenbach b9cb69be8b fix(shellcheck): resolve SC2206 by safe array population
* Replace unquoted array expansion with mapfile-based parsing
* Prevent word splitting and globbing in compute_exclude_csv
* Keep behavior identical while satisfying ShellCheck

https://chatgpt.com/share/69481b6b-02f8-800f-91c8-bbc6592f97b4
2025-12-21 17:07:48 +01:00
kevinveenbirkenbach 30d557ef0d chore(format): add formatter target and apply shfmt/ruff formatting
- Add `make format` target running shfmt + ruff format + ruff check --fix
- Reformat shell scripts with shfmt
- Apply ruff formatting and autofixes across Python code

https://chatgpt.com/share/69480c5a-b0bc-800f-9a2b-770d98c1b6ff
2025-12-21 16:03:48 +01:00
kevinveenbirkenbach f9573460e9 fix(deploy): force local CLI build for invokable resolution
- Drop legacy compatibility wrapper in cli.meta.applications.invokable
- Resolve repository root explicitly via pathlib
- Ensure filter_plugins are importable outside Ansible
- Run invokable discovery with INSTALL_LOCAL_BUILD=1 in deploy tests
- Make deploy.sh fail fast when invokable resolution is unavailable
- Normalize sorting and exclusion logic (drv-* roles, locale-safe sort)
2025-12-21 14:27:21 +01:00
kevinveenbirkenbach cc9c2dceb8 fix(ci,nix): forward NIX_CONFIG into docker build and runtime to avoid GitHub rate limits
- Export NIX_CONFIG in CI jobs and Makefile
- Pass NIX_CONFIG as docker build-arg
- Use NIX_CONFIG only in relevant Dockerfile RUN steps (no secret persistence)
- Forward NIX_CONFIG to test and deploy containers

https://chatgpt.com/share/6947c681-abc8-800f-ba12-de2ff3a7abad
2025-12-21 11:05:31 +01:00
kevinveenbirkenbach 9ef2d86fd9 test(deploy): add preflight check for entry.sh in deploy image
https://chatgpt.com/share/6946f9a9-bf94-800f-adab-f1baf49e1e12
2025-12-20 20:31:47 +01:00