baudolo 7.1.x writes volume generations with `rsync -aHAX`, so the
producing host keeps `user.*` extended attributes (OpenCloud keeps its
storage metadata in `user.oc.*`). Every later copy dropped them again,
because none of the other rsync calls passed `-X` (nor `-A`/`-H`):
- NFS export snapshots (svc-bkp-nfs-2-local `script.sh`),
- the backup host's pull (svc-bkp-remote-2-local `pull_specific_host.py`),
- the device sync (svc-bkp-local-2-device `script.py`),
- every recover into a live target (`utils/recovery/base.py`, used by the
volume, NFS and device recover scripts),
- the `recover full` and `recover remote` tree pulls.
A disaster recovery through the documented chain therefore restored
OpenCloud without `user.oc.*`, silently: the copies succeeded and the
dirval stamp does not cover attributes.
All of these now run `rsync -aHAX` (`-aHAXP` for the device sync). A
target filesystem without xattr support makes rsync exit non-zero as soon
as a source file carries attributes, so the hop fails loudly instead of
dropping them; there is no per-device opt-out. The single-image pull in
`recover remote` keeps `-a`. The provider's ssh-wrapper needs no change:
its glob accepts any server flags and its character guard allows them.
Two tests write a `user.oc.id` attribute and assert it survives the NFS
snapshot and the recover; with the old `-a` both fail with
`OSError: [Errno 61] No data available`.
Closes#1021
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ruff 0.17.0 adds `datetime` -> `dt` to the default aliases of
flake8-import-conventions, and the project selects ICN, so lint-python
failed with ICN001 in seven files.
The files now import `datetime as dt`. ruff's unsafe fix also renamed
two local names in utils/github/playwright_summary.py into the alias:
the `_to_iso_z` parameter and the parsed value in
`_parse_iso_timestamp`. Both are now called `moment`; with the bare fix,
nine tests of test_playwright_summary.py failed with UnboundLocalError
and AttributeError.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
rsync -b turns a deletion into a rename, so a file the source dropped between
two writes of the same generation survives as name~ beside the real one and is
restored into live data. baudolo 3.4.1 fixes the copy that caused the
elasticsearch restore failure; this is the sweep over the sibling call sites
here.
svc-bkp-remote-2-local is the live one: the rsync command is assembled once and
re-executed up to twelve times against the same destination, which is created
with exist_ok before the loop starts. A retry after a partial transfer is
structurally the same second write as baudolo's hot/cold pair, only triggered by
--timeout instead of a container stop. That path pulls other hosts' backups, and
retries are its normal case rather than an exception.
svc-bkp-local-2-device cannot fire today - it writes one pass into a directory
named after the current second - but mkdir(exist_ok=True) makes a re-entry the
same two-write case, so the flag goes rather than the argument for keeping it.
The integration test mirrors the production command and follows.
The lint is what makes this complete: it scans every tracked .py/.sh/.j2/.yml
for a backup flag near an rsync token, and it found the integration test that
the manual sweep had missed. It excludes exactly one file, its own, because the
rule has to name the token it forbids; the sibling guard for || true avoids the
same problem only by scanning .sh alone.
Verified: make quality green across all four targets, the ssh-wrapper
integration test passes with real rsync, and test-lint goes 307 -> 308 with the
new guard collected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every cross-host hop in the backup/recover chain remapped file owners
by NAME on the receiving host (rsync -a default; GNU tar extract as
root prefers name lookup), so a restored uid-1000-owned 0600 config
(n8n) came back owned by a foreign uid and the post-recovery redeploy
crash-looped on EACCES. Pass --numeric-ids to the capture, remote-pull,
device-sync and restore rsyncs and --numeric-owner to the drill's tar
extracts.
The drill additionally wiped and rewrote the export subtree directly on
the ganesha server's backing filesystem while the server kept running
and the nodes kept their client mounts: ganesha's mdcache and the
client dentry caches still described the pre-wipe tree, so update-pass
tasks were Rejected with readdirent ENOENT for the whole converge
window (moodle, espocrm multi-replica). Step 8b now restarts ganesha,
remounts every node and probes a real write through the manager's
mount, each bounded by timeout so a wedged DinD node fails loudly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
svc-bkp-volume-2-local now declares secrets_backup (enabled+shared true):
host secrets are captured unconditionally wherever container backups run.
svc-bkp-remote-2-local declares local-2-device gated on group membership
(operator flag, no variants) so the pull host syncs to the encrypted
device when the role group is assigned; local-2-device registers as a
shared provider and gains the 01_core run-once schema. local-2-device
recover.py learns --device-target: snapshots live under <mount>/<target>,
so recovering without the configured target subpath found nothing.
secrets-2-local memory shrinks to 128m/256m (a oneshot rsync) to keep
variant stacks under the host memory budget.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add utils/recovery/base.py DirectoryRecovery: a service-backed pre-recover
(start the role's backup unit first) + rsync -a --delete mirror. Each
svc-bkp-* role gets a files/recover.py subclass (volume resolves the docker
mountpoint, nfs mirrors the export, local-2-device opens the newest LUKS
snapshot across machine hashes) or a documented one-way opt-out
(remote-2-local recover.py.nocheck). A lint guard enforces every svc-bkp-*
role ships a recover.py and a ## Recover README section; docs/architecture/
backup.md documents the end-to-end schema. Unit tests cover the base flow,
the per-role class logic and every recover.py main() CLI wiring.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Consolidates the backup-role topology around four directional roles
under the existing svc.bkp category:
sys-ctl-bkp-docker-2-loc → svc-bkp-container-2-local
svc-bkp-loc-2-usb → svc-bkp-local-2-device
svc-bkp-rmt-2-loc → svc-bkp-remote-2-local
sys-bkp-provider → deleted (purely a bundler with no own logic;
the two pieces it pulled in — user-backup
and sys-timer-cln-bkps — are now declared
directly by callers that need them)
Every renamed role moves into svc.bkp (already invokable per
roles/categories.yml), gets an `application_id` in vars/main.yml, and
declares the per-app primary-entity service flags + resource limits the
invokable-role lint requires.
Companion changes:
- svc-bkp-container-2-local: meta/variants.yml declares three matrix
rounds (gitea+keycloak, postgres+mariadb, openldap) that pin the
companion topology via service-key flags the CombinedResolver
translates into provider-role includes.
- sys-svc-compose: container-2-local is no longer in the docker-mgmt
include loop; it joins via a separate `'svc-bkp-container-2-local'
in group_names` gate so the 3 backup roles are deployable
independently (each addresses its own failure mode).
- The three `04_seed-database-to-backup.yml` includes (web-app-matrix,
web-app-bigbluebutton, sys-svc-rdbms) are gated by the same
group_names check so consumer apps only seed when the container
backup is actually in scope.
- web-app-gitea picks up `enabled: true, shared: true, provides:
gitea` on its primary entity so the service registry can pull it in
via a `services.gitea.enabled+shared` flag (needed for the
container-backup variant 0).
- group_vars constants and the pyproject ruff per-file-ignore + the
.ansible-lint-ignore paths follow the role-id rename.
- The remote-2-local script picks up the snake_case rename
(`pull-specific-host.py` → `pull_specific_host.py`) and the
`os.makedirs` → `Path.mkdir` conversion so the role no longer needs
the PTH/N999 ruff escape hatch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
kevinveenbirkenbachandClaude Opus 5.5