Commit Graph
20 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 3c743736a8 feat(sys-ctl-hlth-runtime): alert on EOL or unpatched PHP and missing PHP hardening
Answering "which deployed containers run a vulnerable PHP?" after
CVE-2026-109618/109619 took a manual inventory of registry metadata, and
nothing noticed a hardening ini that an image never loads.

- New role sys-ctl-hlth-runtime, routed through sys-svc-compose next to
  sys-ctl-hlth-container. A daily timer runs `php` in every running
  container and fails, alerting through sys-ctl-alm-compose, when PHP is
  below the floor of its line or on a line older than every floor (7.4,
  8.1). Every service that mounts php_hardening in meta/volumes.yml fails
  while expose_php or session.upload_progress.enabled is on; other PHP
  containers of such a stack are listed as info. Compose and swarm
  containers are matched by their project/stack and service labels. The
  check is not run at deploy time, so a finding never fails a deploy.
- The floors (8.2.35, 8.3.36, 8.4.27, 8.5.12) live once, in
  HEALTH_RUNTIME_FLOORS of the role's vars/main.yml.
- `make security-runtimes` lists the PHP of every pinned role image
  without pulling: Docker Hub tags through the Hub API, which no pull rate
  limit meters, other registries through the OCI config blob (new
  registry.config_blob, which the healthcheck probe now reuses).
  PHP_VERSION, distro PHP packages and a PHP_BASE build argument count. It
  exits 1 on an image below its floor or one the registry did not answer.
  Today it flags nfrastack/fusiondirectory:1.5 (line 7.4) and
  shopware/docker-base:8.3 (8.3.25).

The role TODO.md keeps the follow-ups: further runtimes, the split web
SAPI of web-app-lam, a scheduled CI run of the report and the CVE scan.

Refs #1127

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 00:12:25 +01:00
kevinveenbirkenbachandClaude Opus 4.8 4039313266 refactor(tasks): enforce and apply a single task-file numbering scheme
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.

Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:31 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f76bfcf531 refactor(svc-bkp-volume-2-local): rename role from svc-bkp-container-2-local
Rename the role, its test tree, the primary entity key
(volume-2-local), the derived vars (BKP_VOLUME_2_LOCAL_*,
SYS_SERVICE_BACKUP_VOLUME_2_LOCAL), the run-once fact, and every
group_names consumer flag, include path, doc, and lint reference. The
container_backup service key and the on-disk backup-docker-to-local
tree keep their names: both are contracts with existing consumer
entries, snapshot trees, and the baudolo pull tooling.

Operators must rename the svc-bkp-container-2-local inventory group and
disable the previous versioned units on already-provisioned hosts; the
pre-deploy snapshot glob derives its base from the new role name and
will not find them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 18:52:02 +02:00
kevinveenbirkenbachandClaude Opus 4.8 cddb0dc59b feat(lint): require emoji-prefixed task/step/job names + convert existing
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.

- emit emoji names at the source: cli/build/include generator now writes
  "🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
  tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
  task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
  🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 22:45:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 fdf0213f43 fix(swarm-test): wipe the NFS export in the harness, not in the role
A role must not delete its own data, and the per-volume wipe added inside sys-svc-compose's nfs_prep additionally no-op'd because a selectattr filter emptied its loop. Revert nfs_prep and main.yml to the pre-create-only original, and move the clean-state guarantee into 01_start_nfs.sh: it wipes the NFS export dir before the server starts, so every swarm test run begins from empty state and stale per-app volume data cannot bleed across runs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 15:30:16 +02:00
kevinveenbirkenbachandClaude Opus 4.8 fa48411ae5 fix(sys-svc-compose): wipe stale NFS volume state on dev/act swarm redeploy
Swarm reuses NFS-backed state across cluster rebuilds. A role then reads config persisted from a prior cluster while its database (a node-local volume) is fresh, causing install drift and stale-credential loops. Wipe each NFS-backed volume subdir before pre-create, gated to dev/act via RUNTIME so production state is never touched.

Hoist the shared swarm and NFS guards to the include's when, and fold the per-item mapping check into the loop via selectattr, so the two tasks no longer repeat identical conditions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 14:09:12 +02:00
kevinveenbirkenbach 8618bb3b53 refactor(sys-svc-compose): extract stack-host prep tasks behind single IS_STACK_HOST gate
Four consecutive tasks in tasks/main.yml each carried an inline
"when: IS_STACK_HOST | bool" gate. Move them into a dedicated
02_stack_host_prep.yml and include that file once with the
IS_STACK_HOST gate on the include, so the condition lives in one
place. nfs_prep.yml stays inline because it intentionally runs on
every host (workers need their NFS mount points prepared too) and
up.yml stays inline because extracting a single-task file would not
reduce the duplication that the refactor is targeting.

Ansible's include_tasks `when` semantics mean a False gate on the
parent include skips the file entirely, while inner per-task `when`
clauses ANDed with the parent gate produce the same condition the
inline form expressed. The compose_repository_pull, has_domain and
TLS-mode conditions remain on their respective tasks. The
pre-existing TODO comment on the directories task ("This could lead
to problems in sys-svc-compose directories which are based on a git
repository") was an untracked planning note and is dropped per
CLAUDE.md's comment policy.
2026-06-09 23:40:15 +02:00
kevinveenbirkenbach 77e4f288c0 refactor: rename SWARM_MANAGER_ACTIVE to IS_STACK_HOST
The old name suggested "is the swarm manager up?" but the variable
actually answers "is this host the one that holds the compose stack?".
It is true on the single host in compose mode and on every swarm
manager in swarm mode. The SWARM_ prefix is also misleading in compose
mode where no swarm exists.

Replace every reference across group_vars and the gating call sites
(sys-svc-compose, web-app-espocrm, web-app-bigbluebutton). Definition
itself moves to group_vars/all/18_swarm.yml line 6; the body is
unchanged so the truth value at runtime stays identical for both modes.
2026-06-05 11:12:13 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3b7bbfa71d refactor(swarm): SPOT-define SWARM_MANAGER_ACTIVE + SWARM_PRIMARY_MANAGER
Collapse two repeated swarm-vs-compose expressions into single-source-of-truth
group vars and apply across sys-svc-compose, web-app-mediawiki:

- SWARM_MANAGER_ACTIVE: true when DEPLOYMENT_MODE \!= 'swarm' or this host is
  in svc-docker-swarm-manager. Used as the gate for "execute manager-only
  deploy work here" (dir creation, file render, compose up, handler tasks).

- SWARM_PRIMARY_MANAGER: first svc-docker-swarm-manager or self under
  compose-mode fallback. Used as delegate_to: for cluster-wide ops
  (compose pull/build/up, swarm stack deploy, post-deploy DB health probe,
  network create, MediaWiki one-shot install/admin).

Pure refactor — no semantic change. Subsequent edits to either expression
now live in one file.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-04 17:11:07 +02:00
kevinveenbirkenbachandClaude Opus 4.7 5adf3011fe refactor(swarm-nfs): generalize swarm+NFS chain across all roles
Migrate the mediawiki-specific swarm patterns into shared sys-svc-compose
helpers so every role inherits swarm + NFS portability:

- meta/volumes.yml: dict schema with mandatory name|path + optional nfs
  block (uid/gid/mode); 47 roles migrated, 48 lookup consumers updated to
  volumes.<key>.{name,path}
- sys-svc-compose helpers: finalize.yml (flush + post_deploy),
  post_deploy.yml (swarm-resolve + DB-healthy-wait), nfs_prep.yml
  (pre-create NFS subdirs), swarm/resolve_self.yml (canonical
  <ENTITY>_CONTAINER fact), render_replicated_templates.yml +
  _render_one_template.yml (template render + swarm-peer replication,
  per-entry notify, fileglob support)
- 15 roles migrated to render_replicated_templates.yml helper
  (mediawiki, prometheus, kix, friendica, dashboard, nextcloud,
  suitecrm, matrix, flowise, openldap)
- 5 new lint tests pin the architecture: volumes schema, compose chain,
  volume-constants-are-lookups, template-render-centralization,
  meta-no-role-var-refs
- ROLE_FILE_TEMPL_COMPOSE SPOT constant + 7 tests migrated to use it
- Handler test scanner extended to detect nested notify in
  vars: role_templates: lists

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-02 18:57:35 +02:00
kevinveenbirkenbachandClaude Opus 4.7 bfe1d6fd79 refactor(sys-service-loader): rename + post-load queue mechanism
Renames the shared-services loader role from sys-utils-service-loader to
sys-service-loader (shorter, no orphaned "utils" prefix; entity name in
meta/services.yml realigns to service-loader per get_entity_name).

Introduces an explicit two-phase contract in the loader: after the
ordered preload pass, flush a `_service_loader_post_run` queue that
service roles register into during preload via the new helper
`list_or_shoot.yml` (list during preload, shoot directly outside it).

Replaces the `when: not _service_loader_preloading` guards in
sys-svc-compose and sys-svc-webserver-core with helper calls. Closes
the gap where non-web deploys would never load these post-load
dependencies; in normal web deploys behavior stays the same thanks to
each sub-role's `run_once_<role>` gate.

Docs, comments, test docstring, and constructor-stage invocation
follow the rename. Historical changelogs (CHANGELOG.md, packaging/*)
intentionally left at the original name.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 06:48:14 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a08db7fa11 refactor(roles/svc-bkp-*): rename backup roles + make container/local-2-remote invokable
Consolidates the backup-role topology around four directional roles
under the existing svc.bkp category:

  sys-ctl-bkp-docker-2-loc  →  svc-bkp-container-2-local
  svc-bkp-loc-2-usb         →  svc-bkp-local-2-device
  svc-bkp-rmt-2-loc         →  svc-bkp-remote-2-local
  sys-bkp-provider          →  deleted (purely a bundler with no own logic;
                                the two pieces it pulled in — user-backup
                                and sys-timer-cln-bkps — are now declared
                                directly by callers that need them)

Every renamed role moves into svc.bkp (already invokable per
roles/categories.yml), gets an `application_id` in vars/main.yml, and
declares the per-app primary-entity service flags + resource limits the
invokable-role lint requires.

Companion changes:

  - svc-bkp-container-2-local: meta/variants.yml declares three matrix
    rounds (gitea+keycloak, postgres+mariadb, openldap) that pin the
    companion topology via service-key flags the CombinedResolver
    translates into provider-role includes.

  - sys-svc-compose: container-2-local is no longer in the docker-mgmt
    include loop; it joins via a separate `'svc-bkp-container-2-local'
    in group_names` gate so the 3 backup roles are deployable
    independently (each addresses its own failure mode).

  - The three `04_seed-database-to-backup.yml` includes (web-app-matrix,
    web-app-bigbluebutton, sys-svc-rdbms) are gated by the same
    group_names check so consumer apps only seed when the container
    backup is actually in scope.

  - web-app-gitea picks up `enabled: true, shared: true, provides:
    gitea` on its primary entity so the service registry can pull it in
    via a `services.gitea.enabled+shared` flag (needed for the
    container-backup variant 0).

  - group_vars constants and the pyproject ruff per-file-ignore + the
    .ansible-lint-ignore paths follow the role-id rename.

  - The remote-2-local script picks up the snake_case rename
    (`pull-specific-host.py` → `pull_specific_host.py`) and the
    `os.makedirs` → `Path.mkdir` conversion so the role no longer needs
    the PTH/N999 ruff escape hatch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 04:23:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 10f320df98 fix(roles): close preload-context coverage gaps for compose+inj-all
Two roles dropped their post-preload coverage because work that
needed to re-evaluate after the preload-context flag flipped was
trapped behind a once-gate from the first (preload) invocation:

sys-svc-compose/01_core.yml: the docker-mgmt include loop
(cln-docker, bkp-docker-2-loc, hlth-container, hlth-volumes,
rpr-container-hard) lived inside 01_core, which is itself gated by
`run_once_sys_svc_compose`. First call ran during preload, set the
once-flag, and the loop's `not _service_loader_preloading` gate
fired False; subsequent compose invocations skipped 01_core entirely
so the loop never re-fired. Move the loop into main.yml outside the
once-gate so the preload-gate is the only thing left in front of
it.

sys-front-inj-all/main.yml: the `sys-svc-webserver-core` include
sat inside an outer `block: when: run_once_sys_front_inj_all is not
defined`. Preload set that flag and every later invocation skipped
the block, so webserver-core was never re-entered outside preload
and its own hlth-include retry never saw `preload=false`. Drop the
outer block (the flag had no other consumers) and call webserver-
core unconditionally; idempotency for its heavy work is already
handled by `run_once_sys_svc_webserver_core` inside webserver-core
itself. The role legitimately has no role-level once semantics (CDN,
inj-*, body_filter.lua are per-application work) so mark it
`# nocheck: run-once` with the reason inline.

Transitive misses (cln-anon-volumes, rpr-container-soft,
cln-faild-bkps) resolve once their includers (cln-docker /
hlth-container / bkp-docker-2-loc) start running again.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 04:23:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3f6e2dd68d refactor(sys-svc-compose): rename docker_git_repository_* → compose_*
Pure 1:1 rename (60 insertions / 60 deletions across 30 files) to
align the variable names with the role that owns them and to drop the
misleading ``docker_git_`` prefix — these knobs configure the
compose-stack's git clone step, not the docker CLI:

- docker_git_repository_address → compose_repository_url
- docker_git_repository_version → compose_repository_ref
- docker_git_repository_pull    → compose_pull_repository
- docker_repository_path        → compose_repository_path

``ref`` better captures git refs (branches/tags/SHAs are not
"versions"); ``url`` is the idiomatic name for the clone source.

Touches every consumer (web-app-akaunting/bigbluebutton/dashboard/
fediwall/hugo/littlejs/mini-qr/minio/openproject/openwebui/
roulette-wheel/taiga + svc-db-openldap/svc-prx-openresty/sys-svc-rdbms
+ web-svc-collabora/coturn/logout/onlyoffice + sys-front-inj-all), the
role's own defaults/tasks, the bigbluebutton compose_mods filter and
its unit test, the declared-in-services lint, and the hugo
requirements doc.

Breaking change for any external override that still sets the old
names — no alias is provided.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 13:43:43 +02:00
kevinveenbirkenbachandClaude Opus 4.7 951f6d36d2 chore(lint): unify autoformat/galaxy parallelism into one PARALLEL env var
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL,
  declare it at file start with default `true`; still env-overridable.
  Reword the worker comment so the literal `# shellcheck` token no
  longer appears at start-of-line and trips SC1072/SC1073.
* scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE /
  GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure
  is always fatal. NAMESPACE is an internal constant. Add the same
  PARALLEL env var (default `true`, env-overridable); falsy demotes the
  xargs fan-out to one worker.
* docs/contributing/environment/variables.md: collapse the four
  AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both
  consumers.
* docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere;
  add the lint-galaxy parallelism note.

Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 00:59:24 +02:00
kevinveenbirkenbachandClaude Opus 4.7 22841d34b2 fix(lookups): route tls/cert/compose/docker_cards through get_merged_domains
Commit 9b5de4e47 removed the `domains` set_fact in 01_constructor.yml
and migrated only the `domain` lookup to get_merged_domains, leaving
`tls`, `cert`, `compose_file_args`, and the dashboard `docker_cards`
lookup still requiring variables['domains']. That caused
`tls_common: required variable 'domains' is missing` during the OIDC
merge task (first lookup('tls', ...) call at defaults_oidc render).

Also updates sys-svc-compose tasks/handlers so the CA-trust when-clauses
use `lookup('domains') | has_domain(application_id)` instead of the
removed top-level variable. Unit tests for the four lookups now patch
get_merged_domains to read from variables['domains'] so they stay
hermetic without touching the real roles/ tree.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-21 17:46:35 +02:00
kevinveenbirkenbach a55c99ef62 refactor(compose): replace docker lookup with container and unify compose paths
- Rename lookup plugin from `docker` to `container`
- Switch all lookups from `lookup('docker', ...)` to `lookup('container', ...)`
- Normalize compose file keys:
  - docker_compose → compose
  - docker_compose_override → compose_override
  - docker_compose_ca_override → compose_ca_override
- Centralize compose path resolution via docker_paths_utils
- Update compose_file_args to always derive paths from DIR_COMPOSITIONS
- Align all roles, templates, and vars with container/compose naming
- Update and rename unit tests accordingly

This prepares the codebase for engine-agnostic container backends
(e.g. Docker ↔ Podman) and removes legacy docker-specific assumptions.

https://chatgpt.com/share/6987a7cc-d374-800f-99ad-bb4c8f92c59e
2026-02-07 21:59:39 +01:00
kevinveenbirkenbach 3bb8c0ce30 refactor(tls): rename tls_resolve lookup to tls
The TLS lookup has been renamed from `tls_resolve` to `tls` to better
reflect its responsibility as a strict TLS state resolver.

This refactor updates all usages across the codebase, including:
- lookup plugins and internal helpers
- roles, vars, tasks, and templates
- tests, stubs, and error messages
- documentation and changelog references

Behavior, lookup API, and semantics remain unchanged.

https://chatgpt.com/share/6985e635-5a94-800f-b794-937e566ee308
2026-02-06 14:01:32 +01:00
kevinveenbirkenbach 8fe6b92e63 refactor(lookups): migrate nginx_paths, tls_resolve and cert_plan to strict positional want-path API
- Replace legacy want= keyword usage with mandatory positional want-path
- Align all nginx_paths, tls_resolve and cert_plan lookups to strict API
- Keep behavior unchanged while enforcing consistent lookup semantics

https://chatgpt.com/share/6985d642-b40c-800f-9cdf-ed54fc484a92
2026-02-06 13:06:09 +01:00
kevinveenbirkenbach 51239968fd refactor: rename docker-compose roles to sys-svc-compose and update paths
Renamed docker-compose and docker-compose-ca roles to sys-svc-compose and sys-svc-compose-ca, updated template includes, handlers, lookups, tests, and variables, and moved compose instances to /opt/compose.

https://chatgpt.com/share/6985ab2c-8344-800f-9e9a-768f562d138d
2026-02-06 09:49:42 +01:00