Commit Graph
14682 Commits
Author SHA1 Message Date
kevinveenbirkenbach e846766b73 backup: switch docker-to-local backups to dump-only and disable redis backups
- run baudolo with explicit --backups-dir and --dump-only
- disable backup for redis services across all web-app roles
- minor formatting cleanup in system service configuration

https://chatgpt.com/share/6951ad3b-b5b8-800f-a84c-e6ca1e98933e
2025-12-28 23:20:38 +01:00
kevinveenbirkenbach 634fc9672d Renamed variables 2025-12-28 21:39:39 +01:00
kevinveenbirkenbach 04073cbf5c feat(sys-service): add deferred service execution via sys-service-runner
Introduce `system_service_run_final` to defer service execution until the end
of the play. Services marked with this flag are collected during setup and
executed later by the new `sys-service-runner` role.

Key changes:
- Add `system_service_run_final` flag to sys-service
- Automatically suppress flush for deferred services
- Collect deferred services into a final run list
- Introduce sys-service-runner role with per-unit rescue diagnostics
- Migrate health check services to deferred execution

https://chatgpt.com/share/6951921d-0c20-800f-8d42-1ff01a1f08ef
2025-12-28 21:24:54 +01:00
kevinveenbirkenbach 735ae59cbb refactor: migrate cleanback installation from pkgmgr to pip
- replace pkgmgr-install with sys-pip-install for cleanback
- install cleanback as a regular pip package
- align roles with pyproject-based distribution
- keep system-wide, non-interactive installation semantics

https://chatgpt.com/share/69517d20-f850-800f-b6ff-6b983247888f
2025-12-28 20:42:12 +01:00
kevinveenbirkenbach 6ba43b963d Optimized sys-bkp-provider and sys-ctl-bkp-docker-2-loc 2025-12-28 20:38:26 +01:00
kevinveenbirkenbach 8dbe1d66c5 refactor(user-backup): rename sys-bkp-provider-user role and clean up SSH handling
https://chatgpt.com/share/69518670-34b0-800f-84f9-70d001b1e030
2025-12-28 20:35:07 +01:00
kevinveenbirkenbach c7851645b4 refactor(backup): move backups under /var/lib/infinito and parameterize pull script
- switch compose volumes from /Backups to /var/lib/infinito
- add PATH_INFINITO_BACKUP_DIR and drop BACKUPS_FOLDER_PATH
- update backup provider permissions + cleanup service to use new path
- make pull-specific-host accept --folder and raise if no remote backups
- adjust unit tests accordingly

https://chatgpt.com/share/69515eed-001c-800f-b1da-aee8d8683e63
2025-12-28 19:15:29 +01:00
kevinveenbirkenbach e53e7c8f23 refactor(sys-ctl-hlth-csp): switch CSP health check to docker-based csp-checker
- replace legacy checkcsp CLI invocation with docker run
- add configurable CSP checker image and optional pull behavior
- update health script to build and execute docker commands
- pre-pull image during deploy when MODE_UPDATE is enabled
- clean up service ExecStart arguments and handle empty ignore lists

https://chatgpt.com/share/69514fa3-7eec-800f-8deb-ed07b6b6e1b6
2025-12-28 17:28:54 +01:00
kevinveenbirkenbach cfceee7dbe Solved integration tests 2025-12-28 15:38:34 +01:00
kevinveenbirkenbach 5a74f80e85 Executed ruff 2025-12-28 15:23:07 +01:00
kevinveenbirkenbach f3714b00e7 feat(users,token-store): add user token defaults and unify token variable naming
- add `tokens` support to generated default_users
- normalize sys-token-store output variable to `sys_token_store_token`
- align Matomo role to new token-store interface
- reorder Matomo task files for clearer execution flow

https://chatgpt.com/share/69513d00-9a7c-800f-8b1f-5c31e0e21b75
2025-12-28 15:21:35 +01:00
kevinveenbirkenbach f067166095 refactor(secrets): centralize infinito secrets paths and remove role-local csv vars
- introduce global PATH_INFINITO_* secrets paths in group_vars
- replace BKP_DOCKER_2_LOC_DATABASES_CSV with PATH_INFINITO_DATABASE_SECRETS
- align sys-ctl-bkp-docker-2-loc and sys-token-store to shared secrets layout
- remove role-local defaults in favor of global path definitions
- drop outdated Nextcloud backup recovery docs

https://chatgpt.com/share/695132e1-1768-800f-b7e6-b72927f14b81
2025-12-28 14:38:25 +01:00
kevinveenbirkenbach a50adc558a fix(sys-ctl-bkp-docker-2-loc): ensure databases.csv exists and fix loop var collision
- Ensure secrets directory and databases.csv are created before service setup
- Remove permission handling on absent files
- Fix Ansible loop variable collision in baudolo command assertions

https://chatgpt.com/share/69512ac8-6054-800f-a5e3-103b8ab53bba
2025-12-28 14:04:01 +01:00
kevinveenbirkenbach 5686a5ba29 fix(sys-pip-install,sys-service): avoid venv pip installs and handle oneshot services correctly
- Force sys-pip-install to use system python (/usr/bin/python3) instead of active venv
- Add explicit PATH for system-wide pip installs
- Add diagnostics task to verify installation context with system python
- Replace systemctl is-active check with Result/ExecMainStatus to correctly handle oneshot units
- Prevent false failures when oneshot services exit successfully

https://chatgpt.com/share/695123ba-8da4-800f-ae0d-2f09a5efb1e7
2025-12-28 13:33:53 +01:00
kevinveenbirkenbach 880d73e2ff refactor: replace pkgmgr with system pip installation roles
- Introduce sys-pip and sys-pip-install roles
- Install Python CLI tools system-wide via python3 -m pip
- Migrate cleanup and backup roles from pkgmgr to pip-based installs
- Update documentation to reflect Python-based installation
- Rename dev-python-pip to sys-pip for clearer responsibility

https://chatgpt.com/share/695107c4-f320-800f-b4ce-da953de9bb86
2025-12-28 12:41:59 +01:00
kevinveenbirkenbach 160642fe43 Removed unnecessary warnings 2025-12-28 11:06:21 +01:00
kevinveenbirkenbach 426e0c2489 Moved Corefile 2025-12-28 11:03:56 +01:00
kevinveenbirkenbach d65749157f chore(docker): silence hadolint version pinning warnings for multi-distro CI install
Ignore DL3008, DL3033 and DL3041 for the systemd/dbus installation step.
This image is used for multi-distro CI testing, where package version
pinning is neither portable nor desirable.

https://chatgpt.com/share/6950ffb3-1970-800f-9533-a115905ef6b7
2025-12-28 10:59:54 +01:00
kevinveenbirkenbach fc1388a8e4 tests: recognize ansible.builtin.set_fact in run_once validation
The run_once integration test only detected short-form `set_fact`
and failed to recognize fully qualified module usage
(`ansible.builtin.set_fact`).

This caused false negatives for roles correctly using FQCN-compliant
Ansible syntax.

Fix:
- Accept both `set_fact` and `ansible.builtin.set_fact` when collecting
  global run_once_<suffix> definitions
- Accept both forms when detecting per-role run_once definitions
- Remove redundant prefilter token, as structural detection is sufficient

No production logic is affected.

https://chatgpt.com/share/6950fe2c-4b24-800f-9f40-85b2fed3bfba
2025-12-28 10:53:36 +01:00
kevinveenbirkenbach dcd0bed303 refactor(sys-ctl-bkp-docker-2-loc): move databases.csv permission handling to core
- Move databases.csv permission task from seed phase to core tasks
- Ensure root-only permissions are enforced early and unconditionally
- Remove custom run-once logic and related state tracking
- Simplify role logic and improve determinism
- Update run-once integration test documentation and expectations
- Add TODO to verify Docker-in-Docker behavior
2025-12-28 10:41:42 +01:00
kevinveenbirkenbach 01f87b029e Removed unused SYS_TOKEN_STORE_DEFAULT_TOKEN_LENGTH 2025-12-28 10:21:07 +01:00
kevinveenbirkenbach 27b50d4a3c refactor(sys-service): move systemctl handler logic to include and add rc-based diagnostics
- Refactor sys-service handler to use include_tasks for better structure
- Add robust systemctl restart diagnostics based on exit code (is-active)
- Ensure systemctl status and journalctl output are shown before failing
- Prevent premature handler failure by making systemd restart non-fatal
- Minor style cleanup in filter plugin (PEP8 / quoting)

https://chatgpt.com/share/6950f59e-d6cc-800f-9981-f9af4cbb5590
2025-12-28 10:17:14 +01:00
kevinveenbirkenbach 789ba5a247 - Make Nix non-interactive by default via NIX_CONFIG (accept-flake-config)
- Restore docker service handling outside containers, validate inside containers
- Simplify entry.sh: keep original rebuild/copy logic, remove systemd branching
- Switch CI deploy runner:
  - start only CoreDNS via docker compose
  - start infinito via docker run with systemd, cgroup host access and bind mounts
- Avoid compose limitations (no cgroupns=host support)
- Ensure deterministic container naming for exec-based CI operations

https://chatgpt.com/share/6950110b-111c-800f-bbcd-83a551d5a3b8
2025-12-27 20:47:50 +01:00
kevinveenbirkenbach b207134472 feat(ci): enable systemd-in-docker and use explicit container detection
- Install systemd and dbus in Docker image for CI service tests
- Run systemd as PID 1 with proper container conventions
- Mark container environment explicitly via ENV container=docker
- Simplify container detection to rely on explicit env flag
- Skip host-only tasks (timezone, systemctl handlers) inside containers
- Adjust docker-compose runner for privileged systemd execution
- Reduce CI parallelism to avoid OOM kills

This makes systemd-based Ansible roles reliable in Docker/CI
and avoids fragile virtualization heuristics.

https://chatgpt.com/share/6950110b-111c-800f-bbcd-83a551d5a3b8
2025-12-27 18:14:28 +01:00
kevinveenbirkenbach feffa0844d refactor: switch backup-docker-to-local seeding to system-wide baudolo CLI
- replace nix/pkgmgr-based installation with system-wide pip install
- add cross-distro GCC runtime installation (gcc-libs / libstdc++)
- migrate database seeding to baudolo-seed binary
- standardize databases.csv path to /var/lib/infinito/secrets/databases.csv
- remove legacy database_entry_seeder usage
- clean up filter plugin formatting and task naming

https://chatgpt.com/share/694fd81d-0c80-800f-b785-c6b9f26bf9cf
2025-12-27 13:59:05 +01:00
kevinveenbirkenbach ff3feb3862 gitea: limit database connection pool via env vars
Limit Gitea's database connection pool by setting
MAX_OPEN_CONNS, MAX_IDLE_CONNS and CONN_MAX_LIFETIME
using GITEA__database__* environment variables.

This prevents MariaDB connection exhaustion caused by
unbounded idle connections under heavy browsing or
crawler-like access patterns.

https://chatgpt.com/share/694e2f80-f1e0-800f-b0b5-d42d4121f367
2025-12-26 07:47:20 +01:00
kevinveenbirkenbach 25460ebda9 Removed failing during debugging 2025-12-25 20:52:26 +01:00
kevinveenbirkenbach 22c049f6f2 Deactivated sys-daemon handlers in container 2025-12-25 20:23:42 +01:00
kevinveenbirkenbach abd626b4a7 svc-prx-openresty: add MODE_DEBUG wait loop and always dump logs on failure
- wait for container running/healthy in MODE_DEBUG
- dump container status + logs in always block for easier debugging
- enable --debug in deploy test script

https://chatgpt.com/share/694d897b-1fb8-800f-ae7b-a3100cfb3b90
2025-12-25 19:59:03 +01:00
kevinveenbirkenbach fadde240d8 refactor(tokens): unify token handling via sys-token-store
- Introduce sys-token-store lookup.yml (users → store → empty)
- Introduce sys-token-store write.yml (persist + inject into runtime users)
- Remove all implicit token generation and legacy token tasks
- Migrate Matomo to users.<user>.tokens[application_id]
- Remove redundant Matomo token.value and credentials schema
- Fix Matomo bootstrap env: use MATOMO_ADMIN_USER consistently
- Harden Matomo site verification error handling
- Align sys-front-inj-matomo with unified token model

https://chatgpt.com/share/694d7d55-28a0-800f-8c91-cd09cb9ded7f
2025-12-25 19:07:08 +01:00
kevinveenbirkenbach 65b1f3f2cd Set --primary-domain docker.test for test-deploys 2025-12-25 15:26:10 +01:00
kevinveenbirkenbach 3e95649a52 ci(compose): keep infinito container alive for exec-based deploy tests
Switch CI deploy tests from docker run to docker compose with a long-lived
infinito service. The container now uses `tail -f /dev/null` as command to
prevent restart loops caused by the default `infinito --help` CMD, which
previously killed exec processes with exit code 137.

Changes include:
- make infinito a stable exec target in compose (no premature restarts)
- add compose helpers (build/up/exec/down) to deploy.sh
- move inventory creation and dedicated deploy into the running container
- add conditional cleanup via EXIT trap (keep stack on failure for debugging)
- support per-distro container naming via INFINITO_DISTRO

This stabilizes CI runs and makes debugging failed deploys possible.

https://chatgpt.com/share/694d3f99-07e4-800f-bc05-3a60c902064d
2025-12-25 14:43:45 +01:00
kevinveenbirkenbach 347bb175ab Removed non defined variable 2025-12-25 12:46:42 +01:00
kevinveenbirkenbach 1efab4011a ci(docker-compose): add CoreDNS service and CI-specific DNS wiring
- introduce CoreDNS service behind CI profile
- route CI containers via fixed DNS (172.30.0.53)
- add CoreDNS Corefile for docker.test and external forwarding
- make infinito image distro-selectable via INFINITO_DISTRO
- align volumes, env, and networking for CI runner

https://chatgpt.com/share/694d1f08-9794-800f-b1ff-548e9e6c500f
2025-12-25 12:24:51 +01:00
kevinveenbirkenbach 6b5c1a73af Merge pull request #43 from kevinveenbirkenbach/dependabot/github_actions/docker/setup-buildx-action-3.12.0
build(deps): bump docker/setup-buildx-action from 3.11.1 to 3.12.0
2025-12-25 06:05:04 +01:00
dependabot[bot] 14c4ca76f5 build(deps): bump docker/setup-buildx-action from 3.11.1 to 3.12.0
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3.11.1 to 3.12.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/e468171a9de216ec08956ac3ada2f0791b6bd435...8d2750c68a42422c14e847fe6c8ac0403b4cbd6f)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: 3.12.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-25 01:04:51 +00:00
kevinveenbirkenbach 9fc1906774 Optimized newlines 2025-12-24 20:41:03 +01:00
kevinveenbirkenbach 95e3dcd303 sys-daemon: make systemd-analyze optional and refactor core task flow
- Introduce 01_core.yml to centralize sys-daemon execution logic
- Install systemd tools only when required (for systemd-analyze)
- Keep reset/defaults sequencing intact after task renaming
- Normalize task structure and EOF newlines
- Avoid hard dependency on systemd in minimal/container environments

https://chatgpt.com/share/694c417d-94b4-800f-b4a2-e78f4cd37aaf
2025-12-24 20:39:32 +01:00
kevinveenbirkenbach 7c9e4fffdb refactor: extract SSH client installation into sys-svc-ssh role
Move distribution-aware OpenSSH client installation into a dedicated
sys-svc-ssh system service role. Replace inline package installation
in pkgmgr and user-root with role inclusion and shared run-once logic
to improve reuse, consistency, and maintainability.
2025-12-24 20:25:42 +01:00
kevinveenbirkenbach 71604e199e refactor(sys-svc-docker): split docker CLI (container) from engine (host)
- move distro-aware docker CLI installation into shared install-cli.sh
- use install-cli.sh only in Dockerfile (container context)
- install docker engine and compose only on hosts via Ansible
- guard engine installation with DOCKER_IN_CONTAINER
- avoid mixing Docker Inc. repo with Debian docker.io packages

https://chatgpt.com/share/694c3985-f228-800f-ac5c-b0ae41558596
2025-12-24 20:05:24 +01:00
kevinveenbirkenbach 62d1fb6a2f **Fix Matomo trusted hosts + use local API URL**
* Set Matomo `trusted_hosts[]` to include both `{{ domain }}` and `{{ DOCKER_REACH_HOST }}:{{ http_port }}` (idempotent)
* Add `http_port` var for reuse
* Switch `MATOMO_INDEX_PHP_URL` to local reach URL to make API calls reliable on localhost

https://chatgpt.com/share/694c1e06-fcac-800f-9c03-d92a150a4c73
2025-12-24 18:08:12 +01:00
kevinveenbirkenbach 17cba11c99 Optimized administrator integration 2025-12-24 16:47:51 +01:00
kevinveenbirkenbach 1f29519624 centralize system email domain and remove hardcoded admin emails
- derive internal email domain with .localdomain fallback for localhost
- ensure valid email domains for local installations
- remove hardcoded administrator emails from all roles
- prepare roles to consume centrally defined system email settings

https://chatgpt.com/share/694c05d5-e8c0-800f-8d86-602aaa82be21
2025-12-24 16:25:06 +01:00
kevinveenbirkenbach 70bcb5ef18 web-app-matomo: add bootstrap failure debug dump for compose and env files
Wrap Matomo bootstrap in block/rescue to capture failures and
dump docker-compose.yml, effective compose config, and env files.
Env files are only shown when MASK_CREDENTIALS_IN_LOGS is false.

https://chatgpt.com/share/694bde08-a72c-800f-9ac3-a8aa9aef3bde
2025-12-24 13:35:14 +01:00
kevinveenbirkenbach 54969b1f9d test(deploy): force unmasked logs for local debug runs
Set MASK_CREDENTIALS_IN_LOGS=false explicitly in deploy test script
to ensure bootstrap errors are visible during local debugging.

https://chatgpt.com/share/694bd2bf-353c-800f-a0cc-0e4d9f2e550f
2025-12-24 12:59:34 +01:00
kevinveenbirkenbach b3ff44ea08 fix(matomo): use fully qualified service and site URLs for bootstrap
- Use service-based MATOMO_URL inside compose network
- Provide fully qualified MATOMO_SITE_URL (incl. scheme) for installer
- Align vars with dynamic service naming
- Prevent Playwright installer failures due to invalid URLs

https://chatgpt.com/share/694bd2bf-353c-800f-a0cc-0e4d9f2e550f
2025-12-24 12:46:56 +01:00
kevinveenbirkenbach 2c4463879a refactor(matomo): replace nix installer with docker-compose bootstrap profile
* add token configuration and bootstrap service to role config
* introduce dedicated bootstrap env file wired via env_file override
* replace nix/playwright-based installer with one-shot compose bootstrap
* determine bootstrap need based on config.ini.php and token presence
* move post-install API configuration into separate task file

https://chatgpt.com/share/694bcac4-1a00-800f-9aff-dbcd9a2003dc
2025-12-24 12:12:32 +01:00
kevinveenbirkenbach 34350892f0 harden docker-compose file permissions
- set Dockerfile, .env, docker-compose.yml and override files to 0640
- remove executable permissions from config and env files
- align file modes with security best practices

https://chatgpt.com/share/694bc971-37bc-800f-b663-532516e8880d
2025-12-24 12:07:21 +01:00
kevinveenbirkenbach 7b4deaf062 docker-container: allow explicit env_file override in base template
Enable docker_compose_env to explicitly override the env_file used by
docker-container base template. Falls no override is provided, the
existing application_id-based env resolution remains unchanged.

https://chatgpt.com/share/694bc569-1778-800f-8039-9da09fcd438b
2025-12-24 11:50:08 +01:00
kevinveenbirkenbach d3c9380e6b feat(sys-token-store): introduce centralized per-user token store
Add new sys-token-store role to manage per-user, per-app tokens in a single
on-host YAML store.

- Store tokens under /var/lib/infinito/secrets/tokens.yml
- Map users to multiple app tokens (users → tokens → app)
- Ensure secure directories and file permissions
- Load and persist token store server-side (slurp + copy)
- Idempotent token creation and merging
- Generate tokens when missing
- Export resolved token for downstream roles

Constants are defined uppercase, runtime variables lowercase.

https://chatgpt.com/share/694bb78b-45e4-800f-879c-82c11a1b8d0b
2025-12-24 10:50:58 +01:00