Commit Graph
30 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 21eba1e88f fix(backup): carry hard links, ACLs and xattrs on every backup and recover hop
baudolo 7.1.x writes volume generations with `rsync -aHAX`, so the
producing host keeps `user.*` extended attributes (OpenCloud keeps its
storage metadata in `user.oc.*`). Every later copy dropped them again,
because none of the other rsync calls passed `-X` (nor `-A`/`-H`):

- NFS export snapshots (svc-bkp-nfs-2-local `script.sh`),
- the backup host's pull (svc-bkp-remote-2-local `pull_specific_host.py`),
- the device sync (svc-bkp-local-2-device `script.py`),
- every recover into a live target (`utils/recovery/base.py`, used by the
  volume, NFS and device recover scripts),
- the `recover full` and `recover remote` tree pulls.

A disaster recovery through the documented chain therefore restored
OpenCloud without `user.oc.*`, silently: the copies succeeded and the
dirval stamp does not cover attributes.

All of these now run `rsync -aHAX` (`-aHAXP` for the device sync). A
target filesystem without xattr support makes rsync exit non-zero as soon
as a source file carries attributes, so the hop fails loudly instead of
dropping them; there is no per-device opt-out. The single-image pull in
`recover remote` keeps `-a`. The provider's ssh-wrapper needs no change:
its glob accepts any server flags and its character guard allows them.

Two tests write a `user.oc.id` attribute and assert it survives the NFS
snapshot and the recover; with the old `-a` both fail with
`OSError: [Errno 61] No data available`.

Closes #1021

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 18:20:46 +02:00
evangelostsak 33a8689f4c feat(bkp-nfs): let a deployment put this backup role into meshes 2026-10-14 15:03:40 +02:00
kevinveenbirkenbachandClaude Opus 5 4ac07edaea refactor(meta): move the info-media exemption to the file it speaks about
The marker names the reason a role has no upstream homepage or demo video, and
it sat in meta/main.yml because a comment-only info.yml parses to an empty
document, which the schema lint rejected outright. So the exemption lived one
file away from the two fields it explains, and a reader looking for the video
found neither the field nor the reason it was missing.

The schema lint now accepts an empty document exactly when it carries the
marker, and that is what allows the move. The media lint reads the marker from
info.yml rather than main.yml, and its failure message names that same file, so
the fix it suggests lands where the absent field would have gone.

A hundred and two roles follow. Every occurrence was a standalone comment on
the second line of main.yml, directly under the document start, so the move is
a line delete and an insert at the same offset. Seventy-three roles already had
an info.yml and gain the line; twenty-nine get one whose whole content is the
document start and the marker. All hundred and two reasons carry over verbatim,
including the thirty-five bespoke ones.

Nothing downstream sees a change. Of the four readers of meta/info.yml, three
coerce a null parse through `or {}` behind an isinstance guard, and
utils/cache/applications.py skips a falsy parse before assignment, so
applications.<role>.info stays absent for the twenty-nine exactly as it was
while the file did not exist.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 22:26:38 +02:00
kevinveenbirkenbachandClaude Opus 5 4986e80dd4 feat(readme): ask the reader for a domain the deployment can resolve
The Production block never set one. inventory provision writes only
ansible_connection for a local host, so DOMAIN_PRIMARY stayed at the
group_vars default of infinito.localhost and the deployment served a zone its
own resolver knows nothing about. CoreDNS templates answers for
INFINITO_DOMAIN_RE and nothing else, so every lookup of the deployment's own
name came back NXDOMAIN.

Mastodon is where that surfaced. tootctl accounts create validates the address
through EmailMxValidator, which resolves MX and then, for the domain itself,
A and AAAA. All six queries returned NXDOMAIN in run 33574056480, the
validator recorded email :unreachable, and the admin account was never created.
Only the guide rows were affected: the matrix rows write
DOMAIN_PRIMARY: "{{ lookup('env', 'INFINITO_DOMAIN') }}" into their host_vars
and resolve fine.

The template now carries DOMAIN=<your-domain> in both the host and the
container form and threads it into --vars, with the container form passing it
through the docker run boundary as well. The two guide runners substitute it
the way they already substitute the SSH key, so the replay deploys under
INFINITO_DOMAIN like every other CI deploy. compose_deploy.sh moves its env
load ahead of the substitutions to have the value; host_deploy.sh had no env
load at all.

A production reader is asked for a domain now instead of silently inheriting
infinito.localhost, which no real deployment wants either. The shipped default
keeps its value: sys-svc-nss documents that *.localhost resolves through the
myhostname NSS module without any zone, and host-side tasks lean on it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 18:30:25 +02:00
kevinveenbirkenbachandClaude Opus 5 7162b7bf4c Merge fork/feature/svc-net-tor into feature/ai-agent-employees
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.

The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.

Resolutions that were not unions:

Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.

plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.

compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.

tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.

Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.

01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.

Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.

roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.

config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.

READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.

Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 16:32:30 +02:00
kevinveenbirkenbachandClaude Opus 5 575a5a87ac refactor(layout): sort shipped scripts by language and split the unit suite
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).

This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.

Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:

  * 7 `src:` values built from a role variable holding a bare filename
    (svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
    The variable doubles as the destination name inside the container, so the
    language segment belongs at the call site, not in the variable.
  * `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
    through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
    18 roles ship exactly that and have no template fallback: the play would
    have died on each.
  * `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
    interface now reads `files/javascript/javascript.js`.
  * `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
    path through `path_absolute(... 'files', ...)`.
  * 4 `ansible.builtin.script:` invocations naming a bare command
    (pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
  * `dev-python/files/shell/install.sh` pointed at a sibling role's script.

Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 01:31:09 +02:00
kevinveenbirkenbachandClaude Opus 5 c55f508e05 feat(readme): draw mandatory and disabled service relations
The cosmos generator linked a consumer to its provider only through an
'<role>' in group_names reference or an explicit provides: name. A hard
dependency has neither -- pinning the provider on unconditionally is
exactly what removes the Jinja reference the generator was reading -- so
web-svc-asset's bond to web-svc-file vanished from the diagram the moment
it became binding. Match a service key against the provider's own primary
key from the registry, in both directions.

Services the role turns off drew a solid 1:1 edge, claiming a mandatory
relationship for something that carries nothing at deploy time. They now
render as a red 0..0 edge; edge merging gained a real precedence so a
provider consumed through two keys stays live when only one is off.

Named colour, not hex: a '#' opens an entity code in the mermaid lexer and
turns the whole diagram into a parse error on GitHub.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:02:31 +02:00
kevinveenbirkenbachandClaude Opus 5 6c13ea853e fix(svc-bkp-nfs-2-local): drop the variant without an export
Variant 1 pinned nfs-server enabled false, so the matrix deployed the role that backs up the NFS export on a host where no export exists. The unit's own guard then refused the run and the job died: 'ERROR: /srv/nfs missing; this host is expected to serve the NFS export', followed by 'Failed to start svc-bkp-nfs-2-local.12.0.0.infinito.nexus.service'.

The guard is right; the variant was not. The role declares run_after svc-storage-nfs-server and reads its source through lookup('nfs_export_base') -- the export is the subject of the backup, not an optional service. The sibling role shows the legitimate shape: svc-storage-nfs-server toggles its consumer nfs_backup, and the server survives either way.

With no variant left to carry the false polarity, nfs-server takes the exemption the coverage lint documents for exactly this case: flags whose group-derived value is a hard invariant that variants must never override.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 03:38:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 479a55ef5c style(readme): drop the em dash from the role README template
House style bans em and en dashes. Established role READMEs carry none,
but the template had one in its Quick Setup line, so it had been added
after those roles were generated and reappeared in every regenerated
README since. Replacing it with a colon propagates to the 54 roles the
generator refreshed.

The Credits condition switches from an "is defined" test to a truthiness
test because the generator now always sets application_author_url,
passing None for an unknown author. Left as it was, the branch would have
rendered a broken [Name](None) link instead of falling back to the bare
name.

roles/categories.yml loses the same dashes from the svc.virt description.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 20:19:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4039313266 refactor(tasks): enforce and apply a single task-file numbering scheme
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.

Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:31 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ea9ebe0de3 refactor(inventory): rename prod inventory to production
Standardize the private inventory dir on the fully spelled "production" (matching inventories/development). Update the role README template (SPOT) and regenerate 153 role READMEs plus the admin deploy guide; add inventories/production/ to .gitignore, propagated to .dockerignore via make fix-dockerignore.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 19:08:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9ad8fa2971 feat(guide): default self_signed and test the whole README block
The guide deploys to a .localhost target, where letsencrypt cannot issue (Cloudflare DNS-01 needs a real domain). Default TLS_MODE=self_signed in the generated Production block so the guide uses the working self-signed provider. Document every deploy parameter as a top-level shell var (APP, HOST, TLS_MODE, SSH_PUBLIC_KEY) instead of an inline placeholder.

The compose guide now extracts and runs the whole README block, sourcing params from it (no more -e injection); HOST is forced to localhost. Skip checkout/login under ACT so the guide workflow is iterable locally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 21:23:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 972596a758 fix(guide): document the administrator SSH key step and seed it in CI via --vars
The README Production block went provision -> deploy with no admin-key step, so user-administrator failed early (authorized_keys empty) for both the Guide CI jobs and any real user copying the README. Add a visible 'provision --vars' step that sets users.administrator.authorized_keys (merged into host_vars) in both Production variants of the role README template, regenerate all role READMEs, and replace the invisible CI payload splice with a visible substitution of the documented <your-ssh-public-key> placeholder.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 05:45:26 +02:00
kevinveenbirkenbachandClaude Opus 4.8 973bcf1c6f feat(readme): render a host-install Quick Setup for host roles and auto-regenerate it
A host role installs onto the machine itself, so its README Production block now shows the direct local-connection procedure (clone, scripts/install/package.sh, make install, source the env, then provision + deploy against localhost) instead of the container docker-run aimed at a remote server; stack roles keep the docker-run block. The mode is derived from role_has_stack. make docs now runs readme-generate quick_setup=true so the block stays in sync, and every host role README is regenerated accordingly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 02:39:28 +02:00
kevinveenbirkenbachandClaude Fable 5 17cd53609a feat(readme): merge the Production deploy into one docker command
The role README Production section now runs provision and deploy in a
single docker call: APP and HOST are shell variables passed via -e,
INVENTORY is the in-container SPOT for the duplicated paths, and the
redundant --vars-file (optional, default None) and --id (the inventory
is already narrowed by --include) flags are gone. All 153 role READMEs
regenerated from the template; test-run.yml executes exactly this block.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 13:03:25 +02:00
kevinveenbirkenbachandClaude Fable 5 194671d90f feat(readme): uniform description schema and clearer role titles
Rewrite all 254 role descriptions to a uniform noun-phrase schema
(function-first, no leading verb, no trailing period, <=120 chars) and
enforce it via the extended description lint. Rename 12 role README H1
titles for clarity (e.g. Backup Container to Local -> Backup Docker
Volumes, CSS Service -> Corporate Design, NGINX Legal -> Imprint) and
regenerate their Cosmos and Quick Setup sections. Overview table
regenerated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 01:11:48 +02:00
kevinveenbirkenbachandClaude Fable 5 da598a2d67 feat(readme): roles overview table with homepage/video backfill
Add cli.build.readme.overview (make readme-index / make docs) rendering a
roles overview in the root README: Name, Lifecycle, Homepage, Video,
Description, Integrated with, Install. New info-media lint requires
homepage + video in every invokable role's meta/info.yml (opt out via
'# nocheck: info-media' in meta/main.yml); backfilled 147 roles with
researched, adversarially-verified homepages and official videos.

Retire the 'unsupported' lifecycle value: fold it into 'eol' as a valid
live end-of-life status (shipped but untested/unmaintained) across the
stage lists, symbol glossary, allow-list lint and docs. Retag confluence,
jira, minio, phpldapadmin to eol and magento to pre-alpha.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 19:10:01 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3e8f453f9f docs(svc-bkp): regenerate READMEs with current cosmos diagrams
make cosmos regenerated the backup-role READMEs against the current meta state: cardinality labels, the symbol legend and gear-marked Ansible dependencies from the newer generator, without the parked full-backup orchestrator edges.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 14:44:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 90b3845998 feat(readme): role README generator with Cosmos diagrams, Quick Setup, and lint
Add cli/build/readme: derives each role README's managed sections from
templates/roles/README.md.j2.tmpl (the Cosmos mermaid diagram from
meta/services.yml, an invokable-gated Quick Setup, and Credits).

The Cosmos diagram shows dependencies, every declared service, and
dependents (roles that consume this one, matched by role name and by
provides provision-name in both directions). Role nodes carry deploy-mode
emoji markers (compose/swarm/host); disabled and shared services get a
stop mark; dependents are capped at 12.

test_readme enforces the Quick Setup body is byte-exact with the template
render. `make readme-generate` and `make cosmos` regenerate; `make cosmos`
also runs inside `make quality`.

Refactor: utils/symbol_glossary.py is the single source of truth for the
word/emoji mapping (unique project-wide), consumed by the complexity
matrix and the Cosmos builder. utils/roles/deploy.py holds role_has_stack
and role_deploy_modes, shared by both so the logic never drifts.

Add `make lint-mermaid` and its CI workflow, rendering every diagram via
mermaid-cli; the render step provisions chrome-headless-shell and unzip
on demand so it works in the minimal container.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 14:44:33 +02:00
kevinveenbirkenbachandClaude Opus 4.8 672e93246b feat(deploy): add host deploy mode + matrix symbol view
Host lane for invokable non-stack roles (drivers, backup, wireguard, swapfile, ...). modes.host.enabled declared on 56 such roles; new lint test_role_host_mode enforces it. The complexity matrix gains a host column (not stack and modes.host.enabled), --deploy-mode host, and host bundles (one per variant); apps.sh accepts INFINITO_DEPLOY_MODE=host and filters host == true. New test-deploy-host.yml workflow (INFINITO_MAX_JOBS=10) deploys discovered host roles via the mode-agnostic ci/all.sh path, wired into ci-orchestrator (modes default 'swarm compose host').

Also: complexity -s/--symbol compact view (emoji-only headers, true/false as check/cross, lifecycle stage symbols, grouped legend with per-entry explanations, display-width aligned); stack column moved after host. scripts/meta/env/load.sh made zsh-compatible by delegating to bash and importing INFINITO_/PYTHON/VENV/PATH back.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 7a6268fa96 refactor(deploy): replace skip with services modes
Per-role deploy-mode opt-out moves from meta/tests.yml 'skip:' to meta/services.yml <entity>.modes.{compose,swarm}.enabled on the primary entity. get_role_skip now derives from it; get_role_mode_enabled added. A new lint requires every role shipping templates/*compose*.yml.j2 to declare both mode flags. Migrates 107 services.yml and drops 11 skip-only tests.yml. Also removes the variant_status deployability gate so swarm runs one job per variant unconditionally (variant_status.py and deployable_variant_indices deleted, complexity model simplified). Adds the i8matix alias.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 69321d4135 fix(backup): preserve numeric ownership and NFS coherence across the DR chain
Every cross-host hop in the backup/recover chain remapped file owners
by NAME on the receiving host (rsync -a default; GNU tar extract as
root prefers name lookup), so a restored uid-1000-owned 0600 config
(n8n) came back owned by a foreign uid and the post-recovery redeploy
crash-looped on EACCES. Pass --numeric-ids to the capture, remote-pull,
device-sync and restore rsyncs and --numeric-owner to the drill's tar
extracts.

The drill additionally wiped and rewrote the export subtree directly on
the ganesha server's backing filesystem while the server kept running
and the nodes kept their client mounts: ganesha's mdcache and the
client dentry caches still described the pre-wipe tree, so update-pass
tasks were Rejected with readdirent ENOENT for the whole converge
window (moodle, espocrm multi-replica). Step 8b now restarts ganesha,
remounts every node and probes a real write through the manager's
mount, each bounded by timeout so a wedged DinD node fails loudly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 16:15:27 +02:00
kevinveenbirkenbachandClaude Opus 4.8 75d84af636 feat(recover): unified recover CLI + swarm DR drill via CLI
Add cli.administration.recover: one uniform type/source/target command wrapping each svc-bkp-* role files/recover.py (nfs, volume, secrets, device) plus full for whole-tree recovery. Colon-encoded sources are classified by shape; device supports an on-device subpath, a generation, and remote pull over ssh.

Replace the per-host swarm DR-drill scripts (03/06/07/08_recover_*.sh) with the recover CLI in backup/base.sh. Pass the device on-device subpath relative (the CLI reads absolute segments as the restore-root) and load the applications-free .deploy.yml extras twin on the step-10 redeploy so host_vars credentials survive the -e override.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 02:47:39 +02:00
kevinveenbirkenbachandClaude Fable 5 ed23a19a46 feat(svc-bkp): run pre-state backups in the service-loader preload
Replaces the constructor's pre_deploy_snapshot.sh loop with the loader
itself: volume/nfs/secrets instant-start their oneshot in the preload
pass under MODE_BACKUP (force_flush_instant + state started), so the
live pre-update state is snapshotted before any app mutation; nfs keeps
run_after svc-storage-nfs-server, volume pulls sys-svc-container for a
live dockerd and pre-creates /opt/compose. remote pulls at end of play
via the terminator (force_flush_final) once every provider snapshot is
done. Dropping suppress_flush arms the result assert: a failed
pre-state backup aborts the deploy. Role READMEs carry the new trigger
schema as mermaid diagrams.

Known residual: on a host whose pull identity is not yet provisioned
the terminator-forced remote pull fails and turns the deploy red; the
swarm lab installs the trust only in the DR drill after round 1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 01:47:52 +02:00
kevinveenbirkenbachandClaude Fable 5 cc4945aaa5 fix(svc-bkp-nfs-2-local): stop snapshotting the shared backup root
The nfs backup rsyncs the whole export, and the shared backup root
lives inside it (every node's DIR_BACKUPS is the mounted state dir), so
each generation embedded a copy of all previous backups and grew
quadratically; the giant listings also flushed real errors out of
captured journals. The unit now passes the backup root relative to the
export as an anchored rsync exclude, and recover.py protects the live
root from --delete while never replaying embedded stale copies from old
generations (DirectoryRecovery gains rsync_extra_args for that).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 22:44:17 +02:00
kevinveenbirkenbachandClaude Fable 5 329897f335 refactor(tests): move the skip list into meta/tests.yml
meta/tests.yml becomes the SPOT for per-role test metadata: the
test-deploy discovery skip list moves there (top-level skip:) from the
primary service entity in meta/services.yml, next to the existing
cli.timeout. get_role_skip reads the new location; the drv lint,
--skip-mode help and unit fixtures follow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 21:13:40 +02:00
kevinveenbirkenbachandClaude Fable 5 7b6ca22d75 feat(backup): recover.py framework for the svc-bkp-* role family
Add utils/recovery/base.py DirectoryRecovery: a service-backed pre-recover
(start the role's backup unit first) + rsync -a --delete mirror. Each
svc-bkp-* role gets a files/recover.py subclass (volume resolves the docker
mountpoint, nfs mirrors the export, local-2-device opens the newest LUKS
snapshot across machine hashes) or a documented one-way opt-out
(remote-2-local recover.py.nocheck). A lint guard enforces every svc-bkp-*
role ships a recover.py and a ## Recover README section; docs/architecture/
backup.md documents the end-to-end schema. Unit tests cover the base flow,
the per-role class logic and every recover.py main() CLI wiring.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 03:15:52 +02:00
kevinveenbirkenbachandClaude Opus 4.8 1dc58df31e fix(backup): unify the pre-deploy snapshot into one gated loop
The container and NFS pre-deploy snapshots ran two near-identical role
scripts and two constructor tasks. Merge them into one shared script under
scripts/system/backup/ whose content check handles a file (databases.csv)
or a directory (export base), and drive both from a single looped
constructor task keyed by role, unit, and source. Each iteration keeps its
own group_names gate, its own timeout, and reports changed only when it did
not skip.

The shared script also reaches the fresh-host skip loudly: systemctl
list-unit-files exits non-zero when the version glob matches nothing, so the
listing now sits in an if instead of a bare command substitution that
aborted the snapshot under set -euo pipefail (the compose test-development
failure). The volume e2e test starts the unit through an if and dumps the
unit journal on failure so the real start error surfaces.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 03:04:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 1a908a6e5b perf(tests): parallelize suites, hoist fixtures, gate ansible-lint
Run the python suites through pytest-xdist when pytest is importable
(id-level collection parity with unittest discover verified per suite;
serial unittest stays the fallback), with -rP so warnings printed by
passing tests, like the update annotations, still reach the log. run.sh
self-provisions the dev extras through scripts/install/dev-extras.sh,
which stamps inside the venv and skips read-only venvs; make onboard
pre-warms the same path in the running container.

Hoist per-test cache resets and roles scans to setUpClass in the lookup
and resolvable suites (a per-test fingerprint-memo reset closes an
id()-reuse cache-poisoning flake the hoist exposed), inject generation
names into the nfs backup script test instead of sleeping, and narrow
ansible-lint scope to real ansible content. ansible-lint additionally
sits behind a content fingerprint gate that skips the run when nothing
it walks has changed; without a usable git (container bind mounts) the
gate disables itself and the full run remains.

Drop the import-linter layer and ignore for the deleted deps module;
the contract now locks compose -> common -> env. Measured on the host:
unit 101s to 20s, integration 90s to 24s, lint tests 43s to 19s,
repeated ansible-lint 53s to 0.4s.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 21:20:46 +02:00
kevinveenbirkenbachandClaude Opus 4.8 adb8bcf696 feat(svc-bkp-nfs-2-local): scheduled differential NFS export backup
Add a systemd-driven rsync --link-dest backup of the NFS export into the
baudolo-compatible local snapshot tree, plus a pre-deploy snapshot hook in
the constructor. A missing export fails the nightly unit loudly, a failed
rsync removes its incomplete generation so it cannot poison the next
--link-dest, and rsync exit 24 from the live export is tolerated.

Both pre-deploy snapshot scripts (container and nfs) now resolve the
previous deploy's unit through a version glob, so the snapshot still runs
after a project version bump. The user-backup ssh wrapper whitelists the
new backup-nfs-to-local type for downstream pulls. The nfs-server role
declares the backup as a dynamic consumer with true/false pinning
variants; svc-bkp-container-2-local drops its variants.yml because
non-subject closure members resolve foreign round indices against it and
leak its companion pins into other rounds.

Unit tests execute the real script against tempdirs and prove hardlink
deduplication by inode, history isolation, delete semantics, loud failure
on a missing source, and cleanup of incomplete generations.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 18:43:35 +02:00