Compare commits

...
15 Commits
Author SHA1 Message Date
kevinveenbirkenbach b43177f0e7 Release version 7.1.2 2026-10-18 00:43:44 +02:00
kevinveenbirkenbachandClaude Opus 5.5 0735e5fa7b fix(backup): mount the zfs snapshot itself so a container can read it
The zfs snapshot was read through <subject>/.zfs/snapshot/<name>. The
kernel automounts that directory through a usermode helper that runs in
the host's mount namespace, so a containerised baudolo never sees the
mount and finds no snapshot. The snapshot is now mounted explicitly at
<subject>/.baudolo-<tag>, beside where btrfs carves its own, and removed
in order: umount, rmdir, zfs destroy. A failed mount destroys the
snapshot at once instead of leaking one per run.

The e2e suite could not reach this code at all: zpool create on the /img
file failed with "no such pool or dataset", because the kernel opens
vdevs from a taskq thread that resolves paths in the host's root, where
the container's /img does not exist. The pool now sits on a loop device,
whose /dev/loopN node the kernel creates on the host as well, and it is
destroyed on exit once this script created it, since a pool is
kernel-wide.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-18 00:22:53 +02:00
kevinveenbirkenbach 389b418f42 Release version 7.1.1 2026-10-17 23:31:28 +02:00
kevinveenbirkenbachandClaude Opus 5.5 948b8347f4 fix(test): wait for MariaDB with the root password, not the init window
The entrypoint installs root@localhost without unix_socket auth and sets
its password during init, so the passwordless socket probe only passed
when it landed in the window before SET PASSWORD. Missing that window
timed the e2e MariaDB suites out at random.

The probe now authenticates with the root password it already received
and requires @@skip_networking = 0, so the entrypoint's temporary server
no longer counts as ready. The anonymous-preemption bootstrap logs in
with the root password as well.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-17 23:30:00 +02:00
kevinveenbirkenbach f824fabea1 Release version 7.1.0 2026-10-17 22:35:15 +02:00
kevinveenbirkenbachandClaude Opus 5 07d819cb41 fix(backup): carry extended attributes, ACLs and hardlinks into a generation
rsync -a expands to -rlptgoD, so the backup stored neither extended
attributes nor ACLs nor hardlinks while the restore side asks for
-aHAX. A generation therefore never held what a restore wants back, and
rsync faithfully mirrored a snapshot that has none.

Reproduced end to end against a source carrying user.oc.* attributes and
a hardlink: with the old argv, getfattr -d -R over the generation returns
nothing and the hardlink splits into two inodes; with -aHAX both sides
come back identical to the source. That is the shape in which an
OpenCloud volume survived a disaster-recovery drill with its files intact
and its storage metadata gone, so its posix driver skipped every space it
could no longer read.

restore/files.py had the same gap on this repository's own restore path.

Two consequences worth knowing. The first backup after this re-copies
every attribute-carrying volume once, because rsync declines to hardlink
against a generation whose xattrs do not match. And a backup destination
on a filesystem without xattr support now fails the run instead of
producing a generation that cannot be restored.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-17 22:28:49 +02:00
kevinveenbirkenbach 022b096617 Release version 7.0.2 2026-09-23 15:36:18 +02:00
kevinveenbirkenbachandClaude Opus 5.5 161db213cb fix(backup): let the stopped copy replace a failed live pre-copy
A volume whose containers must stop was copied twice: live first, then
with the containers stopped and --checksum. A live writer that truncates
a file under rsync (read errors mapping ...: No data available (61),
exit 23) aborted the run in the first pass, before the containers were
stopped, although the second pass rewrites every differing file anyway.
Seen in infinito-nexus CI on kix_kix_shared/objecticons.

The live copy now only warns when a stopped copy follows. Without a stop
it stays the only copy and still fails the run, and a failing stopped
copy still fails it.

The e2e test drives a writer that shrinks files mid-read; without the
fix it reproduces the CI failure verbatim.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-23 15:34:19 +02:00
dependabot[bot] 9d4cc24a59 build(deps-dev): bump ruff from 0.16.6 to 0.16.7 (#7)
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.16.6 to 0.16.7.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.6...0.16.7)

---
updated-dependencies:
- dependency-name: ruff
  dependency-version: 0.16.7
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-19 00:59:45 +00:00
dependabot[bot] 5bf6bc2afe build(deps-dev): bump ruff from 0.16.5 to 0.16.6 (#6)
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.16.5 to 0.16.6.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.5...0.16.6)

---
updated-dependencies:
- dependency-name: ruff
  dependency-version: 0.16.6
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-12 00:42:58 +00:00
dependabot[bot] 4b47ceab30 build(deps-dev): bump ruff from 0.16.4 to 0.16.5 (#5)
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.16.4 to 0.16.5.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.4...0.16.5)

---
updated-dependencies:
- dependency-name: ruff
  dependency-version: 0.16.5
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-05 00:48:55 +00:00
dependabot[bot] a139f0faa1 build(deps-dev): bump ruff from 0.16.3 to 0.16.4 (#4)
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.16.3 to 0.16.4.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.3...0.16.4)

---
updated-dependencies:
- dependency-name: ruff
  dependency-version: 0.16.4
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-29 00:43:48 +00:00
dependabot[bot] b6ed45770b build(deps-dev): bump ruff from 0.16.1 to 0.16.3 (#3)
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.16.1 to 0.16.3.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.1...0.16.3)

---
updated-dependencies:
- dependency-name: ruff
  dependency-version: 0.16.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-22 01:03:10 +00:00
kevinveenbirkenbach 44b1f16f7c Release version 7.0.1 2026-08-18 10:12:10 +02:00
kevinveenbirkenbachandClaude Opus 5 7f51748486 fix(restore): leave an extension's own objects to the extension
The --empty pre-clean picked its candidates by owner, on the stated
assumption that extension members are superuser-owned and would never be
selected. That holds only when a superuser installed the extension. A role
that installs one itself owns its functions, so they were listed for a
one-by-one DROP that postgres refuses:

    cannot drop function vector_in(cstring,oid,integer)
    because extension vector requires it

Under ON_ERROR_STOP that ends the whole restore, which is how a discourse
generation - it declares the vector extension - became unreplayable.

Membership now comes from pg_depend rather than from ownership. Each branch
carries its oid and classid so one NOT EXISTS covers all seven instead of
seven separate predicates, and the schema branch gets the same guard because
an extension can own a schema too. Skipping the members is enough: the dump's
CREATE EXTENSION IF NOT EXISTS finds the surviving extension either way.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 10:09:04 +02:00
16 changed files with 412 additions and 46 deletions

No files matched your search

+63
View File
@@ -1,5 +1,68 @@
# Changelog
## [7.1.2] - 2026-10-18
Fixed:
- Backup: *--snapshot zfs* found no snapshot inside a container; now mounts it
- Tests: e2e zfs pool now sits on a loop device; a file vdev never worked
Upgrade notes:
- Backup: zfs snapshots mount at *.baudolo-TAG* in the subject, not *.zfs*
- Image: first ghcr image since 3.1.2; the 4.0.0 to 7.1.1 notes apply
## [7.1.1] - 2026-10-17
Fixed:
- Tests: e2e MariaDB probe ran root passwordless and timed out at random
- Tests: probe now logs in with the root password, skips the temp server
- Tests: anonymous-preemption bootstrap now uses the root password too
Upgrade notes:
- 7.1.0 never reached stable: its CI failed, no image was published
- 7.1.1 is the first published -aHAX build; 7.1.0 upgrade notes apply
## [7.1.0] - 2026-10-17
Fixed:
- Backup: rsync -a stored no xattrs, ACLs or hardlinks; now -aHAX
- Restore: restore_volume_files had the same gap; now -aHAX too
- An OpenCloud volume lost its storage metadata through a DR drill
Upgrade notes:
- First backup re-copies every attribute-carrying volume once in full
- A destination without xattr support now fails the backup run outright
- exFAT, xattr-less NFS and object-storage FUSE mounts must move first
## [7.0.2] - 2026-09-23
* Backup: failed live pre-copy no longer aborts when a stopped copy follows
* Backup: stopped --checksum copy replaces pre-copy hit by live writer (rc 23)
* Backup: failing sole live copy or failing stopped copy still fails the run
* Tests: e2e live writer shrinking files mid-read reproduces the rsync failure
## [7.0.1] - 2026-08-18
- Restore: *--empty* no longer aborts on a database that carries an extension.
The pre-clean picked its candidates by owner, on the stated assumption that
extension members are superuser-owned and would therefore never be selected.
That holds only when a superuser installed the extension: a role that
installs one itself owns its functions, so they were listed for a one-by-one
*DROP* that postgres refuses — *cannot drop function
vector_in(cstring,oid,integer) because extension vector requires it*. Under
*ON_ERROR_STOP* that ends the whole restore, which is how a generation of an
application declaring the *vector* extension became unreplayable. Membership
now comes from *pg_depend* rather than from ownership; each branch carries
its oid and classid so one *NOT EXISTS* covers all seven instead of seven
separate predicates, and the schema branch is guarded too because an
extension can own a schema. Skipping the members suffices — the dump's
*CREATE EXTENSION IF NOT EXISTS* finds the surviving extension either way.
## [7.0.0] - 2026-08-18
Breaking:
+2 -2
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "backup-docker-to-local"
version = "7.0.0"
version = "7.1.2"
description = "Backup Docker volumes to local with rsync and optional DB dumps."
readme = "README.md"
requires-python = ">=3.9"
@@ -19,7 +19,7 @@ dependencies = [
[project.optional-dependencies]
# Pinned: a ruff minor bump changes which rules fire, and `make test` gates on
# a clean run, so an unpinned lint would fail the suite on an unrelated day.
lint = ["ruff==0.16.1"]
lint = ["ruff==0.16.7"]
[project.scripts]
baudolo = "baudolo.backup.__main__:main"
+18 -7
View File
@@ -23,6 +23,7 @@ from .layout import (
write_manifest,
)
from .policy import requires_stop, volume_is_fully_ignored
from .shell import BackupError
from .snapshot import snapshot_source, volume_snapshot
from .volume import backup_volume, inspect_backing
@@ -125,13 +126,23 @@ def main() -> int:
copy(authoritative=False)
continue
copy(authoritative=False)
if requires_stop(containers, args.images_no_stop_required):
stoppable = filter_stoppable(containers)
change_containers_status(stoppable, "stop")
copy(authoritative=True)
if not args.shutdown:
change_containers_status(stoppable, "start")
if not requires_stop(containers, args.images_no_stop_required):
copy(authoritative=False)
continue
try:
copy(authoritative=False)
except BackupError as error:
print(
f"WARNING: live pre-copy of volume '{volume_name}' failed; "
f"the copy with its containers stopped replaces it.\n{error}",
flush=True,
)
stoppable = filter_stoppable(containers)
change_containers_status(stoppable, "stop")
copy(authoritative=True)
if not args.shutdown:
change_containers_status(stoppable, "start")
write_manifest(version_dir, outcomes)
stamp_directory(version_dir)
+30 -11
View File
@@ -58,27 +58,50 @@ def _resolver(subject: str, root: str) -> Callable[[str], str]:
return resolve
def _remove(
root: str, commands: list[list[str]], run: Callable[[list[str]], list[str]]
) -> None:
try:
for command in commands:
run(command)
except BackupError as error:
# Raising here would also mask whatever the body raised.
print(f"WARNING: {root} could not be removed: {error}", flush=True)
def _btrfs(
subject: str, name: str, run: Callable[[list[str]], list[str]]
) -> tuple[str, list[str]]:
) -> tuple[str, list[list[str]]]:
# The snapshot goes inside the subject, never beside it: the kernel rejects
# a snapshot whose destination is on another filesystem, which is exactly
# what the parent directory is when the subject is a mountpoint of its own.
target = str(Path(os.path.abspath(subject)) / f".{name}") # noqa: PTH100 - see _resolver
run(["btrfs", "subvolume", "snapshot", "-r", subject, target])
return target, ["btrfs", "subvolume", "delete", target]
return target, [["btrfs", "subvolume", "delete", target]]
def _zfs(
subject: str, name: str, run: Callable[[list[str]], list[str]]
) -> tuple[str, list[str]]:
) -> tuple[str, list[list[str]]]:
output = run(["zfs", "list", "-H", "-o", "name", subject])
dataset = (output[0] if output else "").strip()
if not dataset:
raise SnapshotError(f"no zfs dataset is mounted at {subject}")
run(["zfs", "snapshot", f"{dataset}@{name}"])
root = str(Path(subject) / ".zfs" / "snapshot" / name)
return root, ["zfs", "destroy", f"{dataset}@{name}"]
snapshot = f"{dataset}@{name}"
# Exception: mounted here rather than read through .zfs/snapshot, which the
# kernel automounts via a usermode helper in the host's mount namespace,
# so a containerised run never sees it.
root = str(Path(os.path.abspath(subject)) / f".{name}") # noqa: PTH100 - see _resolver
run(["zfs", "snapshot", snapshot])
remove = [["zfs", "destroy", snapshot]]
try:
run(["mkdir", root])
remove.insert(0, ["rmdir", root])
run(["mount", "-t", "zfs", snapshot, root])
except BaseException:
_remove(root, remove, run)
raise
return root, [["umount", root], *remove]
_CREATE = {"btrfs": _btrfs, "zfs": _zfs}
@@ -169,8 +192,4 @@ def volume_snapshot(
try:
yield _resolver(subject, root)
finally:
try:
run(remove)
except BackupError as error:
# Raising here would also mask whatever the body raised.
print(f"WARNING: {root} could not be removed: {error}", flush=True)
_remove(root, remove, run)
+1 -1
View File
@@ -75,7 +75,7 @@ def backup_volume(
pathlib.Path(dest).mkdir(parents=True, exist_ok=True)
last = get_last_backup_dir(versions_dir, volume_name, dest)
cmd = ["rsync", "-aP", "--no-D", "--delete", "--delete-excluded"]
cmd = ["rsync", "-aHAX", "-P", "--no-D", "--delete", "--delete-excluded"]
if authoritative:
cmd.append("--checksum")
if last:
+28 -9
View File
@@ -3,8 +3,12 @@
-- would run every DROP in one transaction and exhaust max_locks_per_transaction on
-- large schemas (e.g. gitlab). Also drops user-owned non-public schemas so a dump
-- that CREATE SCHEMAs (e.g. discourse's discourse_functions) does not fail on an
-- already-existing schema. Extension members (pg_trgm's set_limit) are
-- superuser-owned; IF EXISTS absorbs the CASCADE fallout.
-- already-existing schema. Objects belonging to an extension are skipped: postgres
-- refuses to drop them one by one ("cannot drop function vector_in(...) because
-- extension vector requires it"), and the dump's CREATE EXTENSION IF NOT EXISTS
-- finds the surviving extension either way. Owning them is not enough to make them
-- droppable - an extension a role installed itself is owned by that role, so the
-- owner filter alone lets pgvector's members through.
SELECT format('DROP %s IF EXISTS public.%s CASCADE', obj.type, obj.name)
FROM (
SELECT format('%I', c.relname) AS name,
@@ -13,7 +17,8 @@ SELECT format('DROP %s IF EXISTS public.%s CASCADE', obj.type, obj.name)
WHEN 'm' THEN 'MATERIALIZED VIEW'
WHEN 'f' THEN 'FOREIGN TABLE'
ELSE 'TABLE'
END AS type
END AS type,
c.oid AS objid, 'pg_class'::regclass AS classid
FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE n.nspname = 'public' AND c.relkind IN ('r', 'p', 'v', 'm', 'f')
AND pg_get_userbyid(c.relowner) = current_user
@@ -21,17 +26,20 @@ SELECT format('DROP %s IF EXISTS public.%s CASCADE', obj.type, obj.name)
-- Overloaded functions share a proname; DROP needs the identity
-- signature or psql aborts with "function name is not unique".
SELECT format('%I(%s)', p.proname, pg_get_function_identity_arguments(p.oid)) AS name,
CASE p.prokind WHEN 'p' THEN 'PROCEDURE' ELSE 'FUNCTION' END AS type
CASE p.prokind WHEN 'p' THEN 'PROCEDURE' ELSE 'FUNCTION' END AS type,
p.oid AS objid, 'pg_proc'::regclass AS classid
FROM pg_proc p JOIN pg_namespace n ON n.oid = p.pronamespace
WHERE n.nspname = 'public' AND p.prokind IN ('f', 'p', 'w')
AND pg_get_userbyid(p.proowner) = current_user
UNION ALL
SELECT format('%I', c.relname) AS name, 'SEQUENCE' AS type
SELECT format('%I', c.relname) AS name, 'SEQUENCE' AS type,
c.oid AS objid, 'pg_class'::regclass AS classid
FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE n.nspname = 'public' AND c.relkind = 'S'
AND pg_get_userbyid(c.relowner) = current_user
UNION ALL
SELECT format('%I', t.typname) AS name, 'TYPE' AS type
SELECT format('%I', t.typname) AS name, 'TYPE' AS type,
t.oid AS objid, 'pg_type'::regclass AS classid
FROM pg_type t JOIN pg_namespace n ON n.oid = t.typnamespace
WHERE n.nspname = 'public'
AND pg_get_userbyid(t.typowner) = current_user
@@ -40,25 +48,36 @@ SELECT format('DROP %s IF EXISTS public.%s CASCADE', obj.type, obj.name)
SELECT 1 FROM pg_class c2
WHERE c2.oid = t.typrelid AND c2.relkind = 'c')))
UNION ALL
SELECT format('%I', col.collname) AS name, 'COLLATION' AS type
SELECT format('%I', col.collname) AS name, 'COLLATION' AS type,
col.oid AS objid, 'pg_collation'::regclass AS classid
FROM pg_collation col JOIN pg_namespace n ON n.oid = col.collnamespace
WHERE n.nspname = 'public'
AND pg_get_userbyid(col.collowner) = current_user
UNION ALL
SELECT format('%I', ts.cfgname) AS name, 'TEXT SEARCH CONFIGURATION' AS type
SELECT format('%I', ts.cfgname) AS name, 'TEXT SEARCH CONFIGURATION' AS type,
ts.oid AS objid, 'pg_ts_config'::regclass AS classid
FROM pg_ts_config ts JOIN pg_namespace n ON n.oid = ts.cfgnamespace
WHERE n.nspname = 'public'
AND pg_get_userbyid(ts.cfgowner) = current_user
UNION ALL
SELECT format('%I', d.dictname) AS name, 'TEXT SEARCH DICTIONARY' AS type
SELECT format('%I', d.dictname) AS name, 'TEXT SEARCH DICTIONARY' AS type,
d.oid AS objid, 'pg_ts_dict'::regclass AS classid
FROM pg_ts_dict d JOIN pg_namespace n ON n.oid = d.dictnamespace
WHERE n.nspname = 'public'
AND pg_get_userbyid(d.dictowner) = current_user
) obj
WHERE NOT EXISTS (
SELECT 1 FROM pg_depend dep
WHERE dep.classid = obj.classid AND dep.objid = obj.objid
AND dep.deptype = 'e')
UNION ALL
SELECT format('DROP SCHEMA IF EXISTS %I CASCADE', n.nspname)
FROM pg_namespace n
WHERE NOT starts_with(n.nspname, 'pg_')
AND n.nspname NOT IN ('public', 'information_schema')
AND pg_get_userbyid(n.nspowner) = current_user
AND NOT EXISTS (
SELECT 1 FROM pg_depend dep
WHERE dep.classid = 'pg_namespace'::regclass AND dep.objid = n.oid
AND dep.deptype = 'e')
\gexec
+1 -1
View File
@@ -59,6 +59,6 @@ def restore_volume_files(volume_name: str, backup_files_dir: str) -> int:
# rsync reads "dir/" as its contents and "dir" as the directory itself.
src = f"{Path(backup_files_dir)}{os.sep}"
dest = f"{Path(mountpoint)}{os.sep}"
run(["rsync", "-avv", "--delete", src, dest])
run(["rsync", "-aHAX", "-vv", "--delete", src, dest])
print("File restore complete.")
return 0
+2 -2
View File
@@ -25,7 +25,7 @@ def backup_run(
images_no_stop_required: list[str],
images_no_backup_required: list[str] | None = None,
only_sql: bool = False,
) -> None:
) -> subprocess.CompletedProcess:
cmd = [
"baudolo",
"--compose-dir",
@@ -49,7 +49,7 @@ def backup_run(
cmd += ["--only-sql"]
try:
run(cmd, capture=True, check=True)
return run(cmd, capture=True, check=True)
except subprocess.CalledProcessError as e:
print(">>> baudolo failed (exit code:", e.returncode, ")")
if e.stdout:
+11 -7
View File
@@ -96,12 +96,13 @@ def wait_for_mariadb(
container: str, *, root_password: str, timeout_s: int = 90
) -> None:
"""
Liveness probe for MariaDB.
Readiness probe for MariaDB, as root over the socket.
IMPORTANT (MariaDB 11):
Root TCP auth is often restricted (unix_socket auth), so a TCP ping like
`mariadb-admin -uroot -p... -h localhost ping` can fail even though the server is up.
We therefore check readiness via a socket-based query.
The entrypoint drops root@127.0.0.1, so root is socket-only, and it
installs root@localhost without unix_socket auth: a probe without the
password only passes in the init window before the password is set.
`@@skip_networking` must be 0 so the entrypoint's temporary server,
which shuts down right after init, does not count as ready.
"""
deadline = time.time() + timeout_s
while time.time() < deadline:
@@ -112,12 +113,15 @@ def wait_for_mariadb(
container,
"sh",
"-lc",
'mariadb -uroot --protocol=socket -e "SELECT 1;"',
(
f"mariadb -uroot -p{root_password} --protocol=socket "
'-N -e "SELECT @@skip_networking;"'
),
],
capture=True,
check=False,
)
if p.returncode == 0:
if p.returncode == 0 and p.stdout.strip() == "0":
return
time.sleep(1)
raise TimeoutError(
@@ -0,0 +1,110 @@
"""A writer that shrinks files mid-read fails the live pre-copy, not the run."""
import unittest
from .helpers import (
backup_path,
backup_run,
cleanup_docker,
create_minimal_compose_dir,
ensure_empty_dir,
latest_version_dir,
require_docker,
run,
unique,
wait_for_log,
write_databases_csv,
)
CHURN_FILES = 8
WRITER = f"""
trap 'exit 0' TERM
echo hello > /data/hello.txt
for i in $(seq 1 {CHURN_FILES}); do
(while :; do truncate -s 0 /data/churn$i; truncate -s 16M /data/churn$i; done) &
done
echo ready
wait
"""
class TestE2EFilesLiveWriterPreCopy(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
require_docker()
cls.prefix = unique("baudolo-e2e-live-writer")
cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
ensure_empty_dir(cls.backups_dir)
cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
cls.repo_name = cls.prefix
cls.volume = f"{cls.prefix}-vol"
cls.writer = f"{cls.prefix}-writer"
cls.containers = [cls.writer]
cls.volumes = [cls.volume]
run(["docker", "volume", "create", cls.volume])
run(
[
"docker",
"run",
"-d",
"--name",
cls.writer,
"-v",
f"{cls.volume}:/data",
"alpine:3.20",
"sh",
"-c",
WRITER,
]
)
wait_for_log(cls.writer, "ready")
cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
write_databases_csv(cls.databases_csv, [])
cls.result = backup_run(
backups_dir=cls.backups_dir,
repo_name=cls.repo_name,
compose_dir=cls.compose_dir,
databases_csv=cls.databases_csv,
database_containers=["dummy-db"],
images_no_stop_required=["dummy-image"],
)
cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
@classmethod
def tearDownClass(cls) -> None:
cleanup_docker(containers=cls.containers, volumes=cls.volumes)
def test_the_live_pre_copy_hit_the_writer(self) -> None:
self.assertIn(
f"WARNING: live pre-copy of volume '{self.volume}' failed",
self.result.stdout,
"the writer never shrank a file under rsync, so this run proves nothing",
)
def test_the_stopped_copy_captured_the_volume(self) -> None:
files = (
backup_path(self.backups_dir, self.repo_name, self.version, self.volume)
/ "files"
)
self.assertEqual((files / "hello.txt").read_text().strip(), "hello")
self.assertEqual(
sorted(p.name for p in files.glob("churn*")),
sorted(f"churn{i}" for i in range(1, CHURN_FILES + 1)),
)
def test_the_writer_runs_again_after_the_backup(self) -> None:
state = run(
["docker", "inspect", "-f", "{{.State.Running}}", self.writer]
).stdout.strip()
self.assertEqual(state, "true")
if __name__ == "__main__":
unittest.main()
@@ -100,7 +100,10 @@ class TestE2EMariaDBAnonymousPreemption(unittest.TestCase):
cls.db_container,
"sh",
"-lc",
f'mariadb -uroot --protocol=socket -e "{bootstrap_sql}"',
(
f"mariadb -uroot -p{cls.root_password} --protocol=socket "
f'-e "{bootstrap_sql}"'
),
]
)
@@ -24,6 +24,7 @@ from .helpers import (
# is not unique") and english_stem_nostop reproduces taiga's text search
# dictionary abort (duplicate pg_ts_dict_dictname_index).
SCENARIO_SQL = (
"CREATE EXTENSION pg_trgm;"
"CREATE SCHEMA discourse_functions;"
"CREATE TABLE discourse_functions.helper (id int);"
"INSERT INTO discourse_functions.helper VALUES (1);"
@@ -166,6 +167,13 @@ class TestE2EPostgresEmptyDropHard(unittest.TestCase):
)
self.assertEqual(self._scalar("SELECT public.f(41) + public.f();"), "42")
def test_the_extension_survived_the_preclean(self) -> None:
self.assertEqual(
self._scalar("SELECT count(*) FROM pg_extension WHERE extname='pg_trgm';"),
"1",
)
self.assertEqual(self._scalar("SELECT similarity('abc','abc')::int;"), "1")
def test_text_search_dictionary_restored_once(self) -> None:
self.assertEqual(
self._scalar(
+7 -2
View File
@@ -36,9 +36,14 @@ ZFS_READY = "{ [ -c /dev/zfs ] || modprobe zfs 2>/dev/null; }; [ -c /dev/zfs ]"
def mount_script(fstype: str) -> str:
"""Build a filesystem on a loop device and carve out the snapshot subject."""
if fstype == "zfs":
# Exception: a loop device, not the /img file, because the kernel opens
# vdevs from a taskq thread that resolves paths in the host's root.
# The pool is kernel-wide, so it is destroyed on exit, but only once
# this script created it.
return (
f"{PACKAGES} && {ZFS_READY} && truncate -s 400M /img "
"&& zpool create -m none baudolo /img "
f"{PACKAGES} && {ZFS_READY} && truncate -s 400M /img && {ATTACH} "
'&& zpool create -m none baudolo "$LOOP" '
"&& trap 'zpool destroy -f baudolo; losetup -d \"$LOOP\"' EXIT "
"&& zfs create -o mountpoint=/subject/docker baudolo/docker"
)
return (
@@ -0,0 +1,70 @@
"""Contract of app.main's live copy when no snapshot is taken."""
from __future__ import annotations
import unittest
from unittest import mock
from baudolo.backup import app
from baudolo.backup.shell import BackupError
from baudolo.backup.volume import Backing
from . import BASE_ARGV
def drive(*, stop: bool, fail_live: bool) -> list[str]:
events: list[str] = []
def record(versions_dir, volume_name, volume_dir, *, authoritative, source):
events.append("authoritative" if authoritative else "live")
if fail_live and not authoritative:
raise BackupError("rsync exit code 23")
with (
mock.patch("sys.argv", BASE_ARGV),
mock.patch.object(app, "get_machine_id", return_value="machine"),
mock.patch.object(app, "create_version_directory", return_value="/gen"),
mock.patch.object(app, "create_volume_directory", return_value="/gen/vol"),
mock.patch.object(app, "load_databases_df", return_value=None),
mock.patch.object(app, "docker_volume_names", return_value=["vol"]),
mock.patch.object(app, "containers_using_volume", return_value=["c"]),
mock.patch.object(app, "volume_is_fully_ignored", return_value=False),
mock.patch.object(app, "backup_dumps_for_volume", return_value=(False, False)),
mock.patch.object(
app,
"inspect_backing",
return_value=Backing("/var/lib/docker/volumes/vol/_data"),
),
mock.patch.object(app, "requires_stop", return_value=stop),
mock.patch.object(app, "filter_stoppable", return_value=["c"]),
mock.patch.object(
app,
"change_containers_status",
side_effect=lambda containers, status: events.append(status),
),
mock.patch.object(app, "write_manifest"),
mock.patch.object(app, "stamp_directory"),
mock.patch.object(app, "handle_docker_compose_services"),
mock.patch.object(app, "backup_volume", side_effect=record),
):
app.main()
return events
class TestLivePreCopy(unittest.TestCase):
def test_a_failed_pre_copy_is_replaced_by_the_stopped_copy(self) -> None:
self.assertEqual(
drive(stop=True, fail_live=True),
["live", "stop", "authoritative", "start"],
)
def test_a_failed_live_copy_without_a_stop_aborts_the_run(self) -> None:
with self.assertRaises(BackupError):
drive(stop=False, fail_live=True)
def test_a_volume_without_a_stop_is_copied_live_once(self) -> None:
self.assertEqual(drive(stop=False, fail_live=False), ["live"])
if __name__ == "__main__":
unittest.main()
+53 -2
View File
@@ -21,6 +21,14 @@ class Runner:
return []
class Unmountable(Runner):
def __call__(self, command: list[str]) -> list[str]:
result = super().__call__(command)
if command[0] == "mount":
raise BackupError("permission denied")
return result
class TestBtrfs(unittest.TestCase):
def test_it_creates_a_read_only_snapshot_inside_the_subject(self) -> None:
run = Runner()
@@ -95,14 +103,57 @@ class TestZfs(unittest.TestCase):
run.calls[-1], ["zfs", "destroy", "tank/docker@baudolo-20260731"]
)
def test_it_maps_a_volume_path_through_the_dot_zfs_directory(self) -> None:
def test_it_mounts_the_snapshot_inside_the_subject(self) -> None:
run = self._run()
with volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run):
pass
self.assertIn(
[
"mount",
"-t",
"zfs",
"tank/docker@baudolo-20260731",
"/var/lib/docker/.baudolo-20260731",
],
run.calls,
)
def test_it_maps_a_volume_path_into_the_mounted_snapshot(self) -> None:
run = self._run()
with volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run) as resolve:
self.assertEqual(
resolve("/var/lib/docker/volumes/postgres_data/_data"),
"/var/lib/docker/.zfs/snapshot/baudolo-20260731/volumes/postgres_data/_data",
"/var/lib/docker/.baudolo-20260731/volumes/postgres_data/_data",
)
def test_it_unmounts_before_it_destroys(self) -> None:
run = self._run()
with volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run):
pass
self.assertEqual(
run.calls[-3:],
[
["umount", "/var/lib/docker/.baudolo-20260731"],
["rmdir", "/var/lib/docker/.baudolo-20260731"],
["zfs", "destroy", "tank/docker@baudolo-20260731"],
],
)
def test_a_failed_mount_leaves_no_snapshot_behind(self) -> None:
run = Unmountable({"zfs list": ["tank/docker"]})
with (
self.assertRaises(BackupError),
volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run),
):
pass
self.assertEqual(
run.calls[-2:],
[
["rmdir", "/var/lib/docker/.baudolo-20260731"],
["zfs", "destroy", "tank/docker@baudolo-20260731"],
],
)
def test_an_unmounted_dataset_is_an_error(self) -> None:
run = Runner({"zfs list": [""]})
with (
+4 -1
View File
@@ -46,9 +46,12 @@ class TestBackupVolume(unittest.TestCase):
def test_it_carries_no_kernel_objects_into_a_generation(self) -> None:
self.assertIn("--no-D", self.copy())
def test_it_stores_the_metadata_a_restore_has_to_give_back(self) -> None:
self.assertIn("-aHAX", self.copy())
def test_it_keeps_no_twin_of_what_the_second_pass_replaces(self) -> None:
command = self.copy(authoritative=True)
self.assertEqual(command[:2], ["rsync", "-aP"])
self.assertEqual(command[:3], ["rsync", "-aHAX", "-P"])
self.assertNotIn("--backup", command)
def test_it_creates_the_destination(self) -> None: