Commit Graph
23 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 37b4de82f0 style(user-backup): fit the ACL task name into the one-line ceiling
The name introduced in ebbf9d944 was 85 characters; tests/lint/repository/test_name_emoji_and_length.py caps task names at 80. It still carries the filesystem that decided the outcome, which was the point of templating it in.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 14:27:14 +02:00
kevinveenbirkenbachandClaude Opus 5 0aa9767265 fix(user-backup): probe for POSIX ACL support instead of listing filesystems
The block was gated on a denylist of exactly one filesystem, '^nfs'. A zpool defaults to acltype=off and rejects setfacl with the same 'Operation not supported', so three compose jobs of run 30695207248 died the moment the filesystem matrix drew zfs.

Attempt the ACL and gate on its return code. That also covers the next backing store, and stops skipping a zpool that does carry acltype=posixacl. The findmnt task stays as the record, so a skipped block still names the filesystem that caused it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-01 14:27:13 +02:00
kevinveenbirkenbachandClaude Opus 5 93b47a5bee refactor(packages): install every package through the registry
Replaces the raw package modules at 76 call sites with package_install, moves
each role's distribution mapping into its own meta/packages.yml, and turns the
inventory PACKAGES variable from source-keyed literal names into a list of
declared package ids. The lint that forbids the raw modules is enabled here,
because the migration burns it down from 76 offenders to zero.

Removes sys-aur, sys-aur-install and sys-package. The AUR build user, the
passwordless pacman rule and the inventory-driven install are part of the plan
the action plugin executes, so the roles no longer carry them.

Declaring the mappings surfaced breakage the raw modules had hidden: on Debian
the `which` binary comes from debianutils, jrnl is in no Debian or Ubuntu
suite, chromium-browser is absent from Debian stable, JDK 11 is unobtainable on
RedHat, wireguard had no RedHat branch at all, qbittorrent was built from the
AUR although Arch ships it in extra, and desk-docker installed a Docker variant
that conflicts with the one sys-svc-container installs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 23:25:50 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4039313266 refactor(tasks): enforce and apply a single task-file numbering scheme
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.

Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:31 +02:00
kevinveenbirkenbachandClaude Opus 4.8 671a1894f3 fix(user-backup): gate POSIX ACLs on the runtime filesystem type
The ACL include was gated on declared intent (storage.backend != 'nfs'), but the host-mode CI deploy NFSv4-mounts /var/lib/infinito via the nfs-server controller mount while storage.backend stays at its 'local' default, so setfacl ran against nfs4 and failed with EOPNOTSUPP (the Linux NFSv4 client exposes no POSIX ACL xattrs). Probe the fstype of the backups root with findmnt and skip ACLs on any nfs*; the existing chown backup:backup + 0700 already enforce the same posture there.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 23:22:47 +02:00
kevinveenbirkenbachandClaude Fable 5 3c1a004647 fix(backup): make the remote pull chain deployable
On Debian the system user backup (uid 34) already exists with
/var/backups and nologin; without explicit home/shell the user module
left it untouched, the role wrote authorized_keys to /home/backup/.ssh
while sshd searched /var/backups/.ssh, and every remote pull died with
'Permission denied (publickey)'. Arch hosts never showed it because the
user did not pre-exist there.

svc-bkp-remote-2-local also no longer ExecStarts during the deploy
flush: the pull can only work once the operator has paired the ssh
identity on the providers, so the debug-mode smoke start failed every
swarm deploy carrying a backup host. Same system_service_suppress_flush
idiom as the volume, secrets and nfs sibling roles; the timer stays
enabled.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 10:47:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 fb423359d2 fix(user-backup): skip POSIX ACLs on NFS-backed backup roots
setfacl returns "Operation not supported" on NFS exports, hard-failing the
backup user setup in the swarm-nfs topology. Extract the ACL tasks into
03b_acl.yml and include them only when storage.backend \!= 'nfs'; chown
backup:backup + 0700 already enforce the same posture on NFS.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 22:01:51 +02:00
kevinveenbirkenbachandClaude Opus 4.8 cddb0dc59b feat(lint): require emoji-prefixed task/step/job names + convert existing
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.

- emit emoji names at the source: cli/build/include generator now writes
  "🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
  tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
  task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
  🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 22:45:06 +02:00
kevinveenbirkenbachandClaude Opus 4.7 59c221521c fix(cluster-a): NFS-everywhere for cluster-shared /var/lib/infinito state
Resolves CI swarm rc=69 ("msmtp: needs a password"): controller-side
Ansible lookups (lookup('email').password etc.) couldn't see the
tokens.yml that Mailu writes target-side, because the controller has
its own filesystem in swarm mode.

Mount NFS-shared /var/lib/infinito on every swarm host AND on the
controller during deploy (in-play delegate_to: localhost task at end
of svc-storage-nfs-server role). State subdir exported separately
with no_root_squash + wildcard CIDRs so root-owned writes succeed
and the controller can mount from outside the svc-swarm IP set.

Single-writer guarantee on FILE_TOKENS via delegate_to: STACK_HOST +
run_once on sys-token-store/{01_core,write}.yml. Backup roles
(svc-bkp-container-2-local, sys-ctl-cln-bkps, sys-ctl-cln-faild-bkps,
user-backup/03_permissions_folders, three direct callers of
04_seed-database-to-backup.yml) gated on IS_STACK_HOST to prevent
N-host write races on the shared NFS backup tree.

Fail-fast assertion in svc-storage-nfs-client + 05_controller_mount
when pre-existing local /var/lib/infinito would be hidden by mount-
over. Operator must migrate or wipe before re-deploying.

SPOTs: SWARM_LAB_NETWORK in workflow env block, storage.nfs.state_subdir
in group_vars/all/15_storage.yml, DIR_VAR_LIB unchanged. UPPER_CASE
constants in NFS roles. Cleanup script umounts controller view before
container teardown to prevent stale-mount leak between matrix jobs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-09 19:02:57 +02:00
kevinveenbirkenbachandClaude Opus 4.7 05f6a97bd8 refactor(stack_host_template): generalise IS_STACK_HOST template wrapper across the codebase
Renames the local action plugin from `conf_template` to
`stack_host_template` so the name carries the actual semantics
(IS_STACK_HOST gate, not nginx config), broadens the lint guard to
every bare `template:` task, and migrates every consumer found in the
audit.

Plugin / tests / lint:
  * plugins/action/conf_template.py -> plugins/action/stack_host_template.py
    (drop nginx-specific docstring; trust_as_template gate unchanged)
  * test_conf_template.py            -> test_stack_host_template.py
  * test_conf_template_migration.py  -> test_stack_host_template_migration.py
    Scope expanded from `.conf.j2` to every `template:` task; marker
    renamed to `nocheck: stack-host-template`.

Migrations (~50 tasks now using stack_host_template:):
  * sys-svc-compose/04_files.yml: compose.yml, .env, Dockerfile,
    compose.override (every per-app compose-tree file)
  * web-app-keycloak: sso_proxy upstream-config (fixes the friendica
    CI failure that prompted this refactor) + realm-import
  * web-app-taiga, web-app-matrix, web-app-opentalk, web-app-pgadmin,
    web-app-moodle, web-app-listmonk, web-app-littlejs, web-app-fediwall,
    web-app-wordpress, web-app-discourse, web-app-bigbluebutton,
    web-app-jenkins, web-app-jitsi, web-app-mobilizon, web-app-suitecrm,
    web-app-decidim, web-app-dashboard, web-app-nextcloud, web-app-xwiki,
    web-app-mailu, web-app-magento, web-app-odoo, web-app-prometheus,
    web-app-matrix, web-app-peertube, web-app-mailu (+ existing `conf_template:`
    callers updated to the new name)
  * sys-front-tls, svc-db-openldap, sys-svc-webserver-core,
    sys-svc-rdbms (dedicated DB env + initdb), sys-front-inj-css,
    sys-front-inj-dashboard, sys-front-inj-all, sys-stk-front-proxy,
    web-svc-cdn-css, web-svc-xmpp, web-svc-legal, web-svc-simpleicons,
    web-opt-rdr-domains

Nochecks (per-host or already-structurally-stack-host):
  * sys-svc-mail-smtp/postfix, sys-svc-sshd, sys-svc-container/daemon.json
  * sys-timer, sys-service, sys-svc-mail-msmtp, sys-svc-journalctl,
    sys-daemon, sys-svc-compose/utils/_render_one_template (delegate_to
    stack host with peer slurp dance)
  * dev-locales, dev-nix, desk-*, user, user-backup, user-workstation
  * svc-storage-nfs-server/exports, test-e2e-playwright stage dirs

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-06 15:45:38 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2a023ca6a1 chore(lint): autoformat strict-failure semantics + claude.sh rename
* scripts/lint/autoformat.sh:
  - rename run_sort_claude_settings worker to run_claude (matches the
    renamed scripts/lint/claude.sh).
  - drop the EXIT trap that wiped status_dir; cleanup happens explicitly
    after `wait` returns, so a parallel worker that interrupts the
    parent never races a vanished status dir.
  - each worker now captures its tool's exit code and writes one of
    OK / SKIP / FAIL. Tolerated `expected violations remain`-style exits
    (markdownlint-cli2 rc≤1, ansible-lint rc≤2) still count as OK; only
    real failures count as FAIL.
  - all workers run to completion regardless of individual tool failures
    (so the user always sees the full picture); after the summary, the
    script exits 1 if any worker reported FAIL.
* scripts/lint/sort_claude_settings.sh -> scripts/lint/claude.sh
  (history preserved via git mv); update test_claude_settings_sorted.py
  fix-hint accordingly.

Plus working-tree drift across roles/, group_vars/, inventories/,
requirements/, and .pre-commit-config.yaml accumulated from the
make-test setup phase regenerating generated artefacts during recent
test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 01:30:49 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c52e4269bf chore(tooling): integrate mbake (Makefile linter/formatter) + working-tree drift sync
Makefile lint integration:
* scripts/install/lint.sh: replace the prior checkmake binary install with
  ensure_mbake (mbake is pip-installable, parses our Makefile correctly
  whereas checkmake bailed on .ONESHELL: / .SHELLFLAGS: / ifneq).
* scripts/lint/makefile.sh: new runner — `mbake format --check Makefile`
  reports formatting drift and missing .PHONY declarations.
* scripts/lint/autoformat.sh: add a two-pass `mbake format Makefile`
  (mbake's formatter is not strictly idempotent on the first pass).
* Makefile: split the multi-line `.PHONY:` block into 18 single-line
  `.PHONY:` declarations so mbake's parser can validate it; add the
  previously-missing `install-python-dev` and `build-dependency` PHONY
  targets; add `lint-makefile` aggregate-and-direct target.
* .pre-commit-config.yaml: new make-lint-makefile hook scoped to
  `^Makefile$`.
* .github/workflows/lint-makefile.yml: new CI workflow (pip-install mbake
  + make lint-makefile).
* .github/workflows/ci-orchestrator.yml: wire lint-makefile into
  lint-gate.needs.

Misc:
* .gitignore: add roles/importer_result.json (tooling artefact written by
  galaxy-importer when it runs in roles/).

Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 00:32:44 +02:00
kevinveenbirkenbachandClaude Opus 4.6 b70a783423 refactor(lookups): migrate to runtime applications/users lookups
Replaces inline `applications`/`users` merging with dedicated lookup
plugins and runtime hydration. Introduces `add_csp_hash` filter,
integrates sort into `find_dock_val_by_bkp_entr`, splits the Keycloak
one-shot realm import into `05a_realm_import_run.yml`, and fixes the
Mastodon administrator task YAML. Removes obsolete `csp_hashes` filter,
`sys-token-store` Jinja filter, and empty CLI setup modules.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 23:44:19 +02:00
kevinveenbirkenbach 579017508a user-backup: install acl package before ACL tasks 2026-02-23 08:31:43 +01:00
kevinveenbirkenbach ff74bfc222 Fix docker readiness and make SSH restart handler distro-safe 2026-02-22 00:25:35 +01:00
kevinveenbirkenbach d33418176b fix(user-backup): ensure backup home exists before ssh wrapper deploy 2026-02-21 19:45:17 +01:00
kevinveenbirkenbach e1e1f3eb11 chore(vars): rename PATH_* to DIR_/BIN_/FILE_ conventions
https://chatgpt.com/share/6985cf49-3c74-800f-af97-f68b1969ab01
2026-02-06 12:23:46 +01:00
kevinveenbirkenbach d7f13d30c6 fix(ci): wait for systemd readiness after compose up
Add a systemd readiness gate to the CI compose wrapper to avoid a race
where systemctl is called before DBus/systemd is fully ready.
This prevents sporadic failures like:
"System has not been booted with systemd as init system (PID 1)".

No Dockerfile or Ansible role changes required.

https://chatgpt.com/share/696045b5-bd08-800f-acd9-e3a4e2e23e52
2026-01-09 01:02:54 +01:00
kevinveenbirkenbach 9905ed619a refactor(user): migrate shared user handling to strict user_key SPOT and introduce workstation role
- Refactor shared `user` role to use `user_key` exclusively (remove user_name)
- Resolve system username, home, group, and ownership via users[user_key]
- Update SSH authorized_keys handling to read from users[user_key]
- Adjust documentation to clearly describe user_key-based SPOT model
- Migrate all calling roles (administrator, backup, root) to pass user_key
- Introduce new `user-workstation` role with dedicated README, vars, tasks, and metadata
- Add workstation stage integration to ensure workstation user setup runs explicitly
- Ensure consistent variable naming (lowercase internals, uppercase constants)
- Improve robustness and clarity of user management across all roles

https://chatgpt.com/share/695ea93e-3fa0-800f-b848-ec2ced2a8291
2026-01-07 20:27:28 +01:00
kevinveenbirkenbach fcf46bc978 fix(user-backup,ssh): correct authorized_keys rendering and newline handling
- Prefix backup SSH keys safely without regex backreferences
- Switch authorized_keys content to literal block for deterministic output
- Strip literal "\n" and trailing whitespace from SSH keys
- Remove fragile ternary-based newline logic to avoid malformed files

https://chatgpt.com/share/695e895d-15e8-800f-864f-92a59b10733a
2026-01-07 17:26:57 +01:00
kevinveenbirkenbach 533a1235fe Implement SPOT-based authorized_keys management for users
* Move SSH key handling to users.<name>.authorized_keys (list-only) and write keys via shared user role
* Enforce at least one SSH key for administrator user
* Wrap backup user keys with restricted command wrapper before writing authorized_keys
* Remove legacy inventory file copy and CLI flag/function for administrator authorized_keys
* Extend user setup generator to include authorized_keys field
* Rename administrator scripts path to root scripts path and update dependent variables
* Update inventory sample and docs to reflect new SSH key workflow

https://chatgpt.com/share/695510b8-23f4-800f-b5db-877b07a1469b
2025-12-31 13:01:45 +01:00
kevinveenbirkenbach 2d691ff933 fix(backup): ensure backup user and root directory are initialized before services
- Move run-once flag execution before dependency includes
- Add user-backup as explicit dependency for backup and cleanup roles
- Ensure backup root directory exists before cleanup services run
- Fix dependency ordering to prevent cleanback failures on missing paths

https://chatgpt.com/share/6954df53-b9d8-800f-b9e7-a1508cf00279
2025-12-31 09:31:12 +01:00
kevinveenbirkenbach 8dbe1d66c5 refactor(user-backup): rename sys-bkp-provider-user role and clean up SSH handling
https://chatgpt.com/share/69518670-34b0-800f-84f9-70d001b1e030
2025-12-28 20:35:07 +01:00