mirror of
https://github.com/kevinveenbirkenbach/docker-volume-backup.git
synced 2026-08-01 12:34:50 +00:00
Compare commits
6 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 4e2b3641f9 | |||
| b10d50efbe | |||
| 7e815bfcf7 | |||
| d4317827bd | |||
| ec3d1a5046 | |||
| 37b735cf7b |
9
.github/workflows/ci.yml
vendored
9
.github/workflows/ci.yml
vendored
@@ -29,7 +29,16 @@ jobs:
|
|||||||
docker version
|
docker version
|
||||||
docker info
|
docker info
|
||||||
|
|
||||||
|
- name: Provide zfs so the snapshot suite covers every filesystem
|
||||||
|
run: |
|
||||||
|
sudo apt-get update
|
||||||
|
sudo apt-get install -y --no-install-recommends zfsutils-linux
|
||||||
|
sudo modprobe zfs
|
||||||
|
zpool version
|
||||||
|
|
||||||
- name: Run all tests via Makefile
|
- name: Run all tests via Makefile
|
||||||
|
env:
|
||||||
|
E2E_REQUIRE_FILESYSTEMS: "btrfs ext4 zfs"
|
||||||
run: |
|
run: |
|
||||||
make test
|
make test
|
||||||
|
|
||||||
|
|||||||
84
CHANGELOG.md
84
CHANGELOG.md
@@ -1,5 +1,89 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## [3.2.1] - 2026-07-31
|
||||||
|
|
||||||
|
- Backup: the snapshot resolver keeps the trailing separator *get_storage_path*
|
||||||
|
puts on a volume path — *os.path.abspath* stripped it. rsync reads *dir* as
|
||||||
|
"copy the directory" where *dir/* means "copy its contents", so every snapshot
|
||||||
|
generation landed at *<volume>/files/_data/...* while the live path lands at
|
||||||
|
*<volume>/files/...*. Restores read the live layout, and *--link-dest* had
|
||||||
|
nothing to match against the previous generation.
|
||||||
|
- Backup: snapshot teardown no longer fails a completed run. A busy
|
||||||
|
*btrfs subvolume delete* raised out of the *finally*, skipping the generation
|
||||||
|
stamp and the compose handling on a run whose data was already copied, and
|
||||||
|
masking whatever the body had raised. The leftover is reported instead.
|
||||||
|
- Backup: a volume created after the snapshot was taken is copied live with a
|
||||||
|
warning instead of aborting the run. Nothing is stopped in snapshot mode, so
|
||||||
|
the host keeps creating volumes for the whole copy.
|
||||||
|
- Backup: the snapshot pass compares by content (*--checksum*) again. 3.2.0
|
||||||
|
dropped it because a snapshot source cannot move, which is true, but the
|
||||||
|
comparison that matters is against *--link-dest*: a file that changed while
|
||||||
|
keeping its size and whole-second mtime was hard-linked stale out of the
|
||||||
|
previous generation, and the single pass had no authoritative pass to repair
|
||||||
|
it. Still one pass where the live path takes two.
|
||||||
|
- Backup: *--hard-restart-projects* is refused alongside *--snapshot*, like
|
||||||
|
*--shutdown* already is. It exists for stacks whose database cannot be backed
|
||||||
|
up hot, which is what a snapshot removes.
|
||||||
|
- Tests: the trailing separator, both teardown behaviours, the new refusal, and
|
||||||
|
*app.main* driving the snapshot branch — the caller that runs in production,
|
||||||
|
which no test had exercised, which is why the layout defect shipped.
|
||||||
|
|
||||||
|
## [3.2.0] - 2026-07-31
|
||||||
|
|
||||||
|
- Backup: *--snapshot {btrfs,zfs}* with *--snapshot-subject* captures every
|
||||||
|
volume from one atomic filesystem snapshot. The subject — the btrfs subvolume
|
||||||
|
or zfs dataset holding the docker volumes, e.g. */var/lib/docker* — is frozen
|
||||||
|
once per run, so a generation shares a single point in time and no container
|
||||||
|
is stopped. Restoring such a copy is an ordinary crash recovery, which every
|
||||||
|
supported engine performs at startup. This is the mode 3.1.4 pointed to for
|
||||||
|
volumes where two rsync passes over a live tree stop being affordable.
|
||||||
|
- Backup: snapshot passes copy once and drop *--checksum*. Verification exists
|
||||||
|
because a hot pass writes its destination from a moving source; a snapshot
|
||||||
|
source cannot move, so size and mtime cannot race and the second full read is
|
||||||
|
pure cost.
|
||||||
|
- Backup: an unsupported filesystem or unknown kind fails with *SnapshotError*.
|
||||||
|
The kind is stated, not probed — an inconclusive probe would fall back to a
|
||||||
|
live copy and hand out the torn backup the mode prevents. *--shutdown* is
|
||||||
|
rejected alongside *--snapshot* rather than ignored, since nothing is stopped.
|
||||||
|
- Refactor: *backup/app.py* splits into *layout.py*, *policy.py* and *dumps.py*
|
||||||
|
along the lines it already had; 276 lines down to 124.
|
||||||
|
- Tests: unit coverage for snapshot, layout, policy, volume and CLI validation.
|
||||||
|
E2E cases drive real btrfs, zfs and ext4 on loop devices, one proving the
|
||||||
|
loud refusal; another writes a MariaDB across the snapshot and requires the
|
||||||
|
restored server to recover on its own with every committed row and none of
|
||||||
|
the later ones. CI installs zfs and sets *E2E_REQUIRE_FILESYSTEMS*, so a
|
||||||
|
missing kernel module fails the build instead of skipping a filesystem.
|
||||||
|
|
||||||
|
## [3.1.4] - 2026-07-31
|
||||||
|
|
||||||
|
- Backup: the post-stop volume pass now compares by content (*--checksum*), so
|
||||||
|
it can no longer skip a file the hot pass had copied from a live source. Each
|
||||||
|
volume is rsynced twice into the same destination — once with the container
|
||||||
|
running, once after it is stopped — and the second pass used rsync's quick
|
||||||
|
check (size plus whole-second mtime). A pre-allocated 16 MiB WAL segment never
|
||||||
|
changes size, so when its last pre-stop write and postgres' shutdown
|
||||||
|
checkpoint fell in the same whole second, the cold pass skipped it while still
|
||||||
|
replacing *global/pg_control*, whose previous write was seconds earlier. The
|
||||||
|
generation then paired a post-shutdown *pg_control* with a WAL segment still
|
||||||
|
zeroed at the recorded checkpoint LSN, and restoring it crash-looped postgres
|
||||||
|
with "invalid record length … expected at least 24, got 0" followed by "PANIC:
|
||||||
|
could not locate a valid checkpoint record". *backup_volume* takes
|
||||||
|
*authoritative* as a required keyword rather than an optional flag, so each of
|
||||||
|
the four call sites states which pass it is; the hot passes keep the quick
|
||||||
|
check. *--ignore-times* was rejected because it destroys the *--link-dest*
|
||||||
|
hardlink dedup (measured 20/20 to 0/20, generation size doubled), and
|
||||||
|
*--modify-window=-1* because it makes correctness depend on the destination
|
||||||
|
filesystem preserving sub-second mtimes, which degrades silently on NFS or
|
||||||
|
ext3.
|
||||||
|
- Cost: the quick check scales with file count, *--checksum* with bytes read on
|
||||||
|
both sides, and it runs while the container is stopped. The extra stop time
|
||||||
|
stays under a minute up to roughly 4 GB on spinning disk, 15 GB on a SATA SSD
|
||||||
|
and 60 GB on NVMe; at 1 TB it is 17 minutes on NVMe and over three hours on
|
||||||
|
spinning disk. No size threshold is built in, since a guessed one would drop
|
||||||
|
the guarantee exactly where an unrestorable backup costs most — volumes at
|
||||||
|
that scale want filesystem snapshots or *pg_basebackup* rather than two rsync
|
||||||
|
passes over a live tree.
|
||||||
|
|
||||||
## [3.1.3] - 2026-07-20
|
## [3.1.3] - 2026-07-20
|
||||||
|
|
||||||
- Restore: the postgres dump replay now runs under *--single-transaction*,
|
- Restore: the postgres dump replay now runs under *--single-transaction*,
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
|||||||
|
|
||||||
[project]
|
[project]
|
||||||
name = "backup-docker-to-local"
|
name = "backup-docker-to-local"
|
||||||
version = "3.1.3"
|
version = "3.2.1"
|
||||||
description = "Backup Docker volumes to local with rsync and optional DB dumps."
|
description = "Backup Docker volumes to local with rsync and optional DB dumps."
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
requires-python = ">=3.9"
|
requires-python = ">=3.9"
|
||||||
|
|||||||
@@ -198,6 +198,7 @@ else
|
|||||||
--network "${NET}" \
|
--network "${NET}" \
|
||||||
-e DOCKER_HOST="${DIND_HOST_IN_NET}" \
|
-e DOCKER_HOST="${DIND_HOST_IN_NET}" \
|
||||||
-e E2E_TEST_PATTERN="${TEST_PATTERN}" \
|
-e E2E_TEST_PATTERN="${TEST_PATTERN}" \
|
||||||
|
-e E2E_REQUIRE_FILESYSTEMS="${E2E_REQUIRE_FILESYSTEMS:-}" \
|
||||||
-v "${DIND_VOL}:/var/lib/docker" \
|
-v "${DIND_VOL}:/var/lib/docker" \
|
||||||
-v "${E2E_TMP_VOL}:/tmp" \
|
-v "${E2E_TMP_VOL}:/tmp" \
|
||||||
"${IMG}" \
|
"${IMG}" \
|
||||||
|
|||||||
@@ -1,171 +1,29 @@
|
|||||||
|
"""Back up every Docker volume of a host into a timestamped generation."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import os
|
import os
|
||||||
import pathlib
|
from contextlib import ExitStack
|
||||||
import sys
|
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
import pandas
|
|
||||||
from dirval import create_stamp_file
|
|
||||||
from pandas.errors import EmptyDataError
|
|
||||||
|
|
||||||
from .cli import parse_args
|
from .cli import parse_args
|
||||||
from .compose import handle_docker_compose_services
|
from .compose import handle_docker_compose_services
|
||||||
from .db import backup_database
|
|
||||||
from .docker import (
|
from .docker import (
|
||||||
change_containers_status,
|
change_containers_status,
|
||||||
containers_using_volume,
|
containers_using_volume,
|
||||||
docker_volume_names,
|
docker_volume_names,
|
||||||
filter_stoppable,
|
filter_stoppable,
|
||||||
get_image_info,
|
|
||||||
has_image,
|
|
||||||
is_swarm_task,
|
|
||||||
)
|
)
|
||||||
from .shell import execute_shell_command
|
from .dumps import backup_dumps_for_volume, load_databases_df
|
||||||
from .volume import backup_volume
|
from .layout import (
|
||||||
|
create_version_directory,
|
||||||
|
create_volume_directory,
|
||||||
def get_machine_id() -> str:
|
get_machine_id,
|
||||||
return execute_shell_command("sha256sum /etc/machine-id")[0][0:64]
|
stamp_directory,
|
||||||
|
)
|
||||||
|
from .policy import requires_stop, volume_is_fully_ignored
|
||||||
def stamp_directory(version_dir: str) -> None:
|
from .snapshot import volume_snapshot
|
||||||
"""
|
from .volume import backup_volume, get_storage_path
|
||||||
Use dirval as a Python library to stamp the directory (no CLI dependency).
|
|
||||||
"""
|
|
||||||
create_stamp_file(version_dir)
|
|
||||||
|
|
||||||
|
|
||||||
def create_version_directory(versions_dir: str, backup_time: str) -> str:
|
|
||||||
version_dir = os.path.join(versions_dir, backup_time)
|
|
||||||
pathlib.Path(version_dir).mkdir(parents=True, exist_ok=True)
|
|
||||||
return version_dir
|
|
||||||
|
|
||||||
|
|
||||||
def create_volume_directory(version_dir: str, volume_name: str) -> str:
|
|
||||||
path = os.path.join(version_dir, volume_name)
|
|
||||||
pathlib.Path(path).mkdir(parents=True, exist_ok=True)
|
|
||||||
return path
|
|
||||||
|
|
||||||
|
|
||||||
def is_image_ignored(container: str, images_no_backup_required: list[str]) -> bool:
|
|
||||||
if not images_no_backup_required:
|
|
||||||
return False
|
|
||||||
img = get_image_info(container)
|
|
||||||
return img in images_no_backup_required
|
|
||||||
|
|
||||||
|
|
||||||
def volume_is_fully_ignored(
|
|
||||||
containers: list[str], images_no_backup_required: list[str]
|
|
||||||
) -> bool:
|
|
||||||
"""
|
|
||||||
Skip file backup only if all containers linked to the volume are ignored.
|
|
||||||
"""
|
|
||||||
if not containers:
|
|
||||||
return False
|
|
||||||
return all(is_image_ignored(c, images_no_backup_required) for c in containers)
|
|
||||||
|
|
||||||
|
|
||||||
def requires_stop(containers: list[str], images_no_stop_required: list[str]) -> bool:
|
|
||||||
"""
|
|
||||||
Stop is required if ANY stoppable container image is NOT in the exact
|
|
||||||
image whitelist. Swarm task containers never count: baudolo must
|
|
||||||
not cycle them (see docker.is_swarm_task).
|
|
||||||
"""
|
|
||||||
for c in containers:
|
|
||||||
if is_swarm_task(c):
|
|
||||||
continue
|
|
||||||
img = get_image_info(c)
|
|
||||||
if img not in images_no_stop_required:
|
|
||||||
return True
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def backup_mariadb_or_postgres(
|
|
||||||
*,
|
|
||||||
container: str,
|
|
||||||
volume_dir: str,
|
|
||||||
databases_df: "pandas.DataFrame",
|
|
||||||
database_containers: list[str],
|
|
||||||
) -> tuple[bool, bool]:
|
|
||||||
"""
|
|
||||||
Returns (is_db_container, dumped_any)
|
|
||||||
"""
|
|
||||||
for img in ["mariadb", "postgres"]:
|
|
||||||
if has_image(container, img):
|
|
||||||
dumped = backup_database(
|
|
||||||
container=container,
|
|
||||||
volume_dir=volume_dir,
|
|
||||||
db_type=img,
|
|
||||||
databases_df=databases_df,
|
|
||||||
database_containers=database_containers,
|
|
||||||
)
|
|
||||||
return True, dumped
|
|
||||||
return False, False
|
|
||||||
|
|
||||||
|
|
||||||
def _empty_databases_df() -> "pandas.DataFrame":
|
|
||||||
"""
|
|
||||||
Create an empty DataFrame with the expected schema for databases.csv.
|
|
||||||
|
|
||||||
This allows the backup to continue without DB dumps when the CSV is missing
|
|
||||||
or empty (pandas EmptyDataError).
|
|
||||||
"""
|
|
||||||
return pandas.DataFrame(columns=["instance", "database", "username", "password"])
|
|
||||||
|
|
||||||
|
|
||||||
def _load_databases_df(csv_path: str) -> "pandas.DataFrame":
|
|
||||||
"""
|
|
||||||
Load databases.csv robustly.
|
|
||||||
|
|
||||||
- Missing file -> warn, continue with empty df
|
|
||||||
- Empty file -> warn, continue with empty df
|
|
||||||
- Valid CSV -> return dataframe
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
return pandas.read_csv(csv_path, sep=";", keep_default_na=False, dtype=str)
|
|
||||||
except FileNotFoundError:
|
|
||||||
print(
|
|
||||||
f"WARNING: databases.csv not found: {csv_path}. Continuing without database dumps.",
|
|
||||||
file=sys.stderr,
|
|
||||||
flush=True,
|
|
||||||
)
|
|
||||||
return _empty_databases_df()
|
|
||||||
except EmptyDataError:
|
|
||||||
print(
|
|
||||||
f"WARNING: databases.csv exists but is empty: {csv_path}. Continuing without database dumps.",
|
|
||||||
file=sys.stderr,
|
|
||||||
flush=True,
|
|
||||||
)
|
|
||||||
return _empty_databases_df()
|
|
||||||
|
|
||||||
|
|
||||||
def _backup_dumps_for_volume(
|
|
||||||
*,
|
|
||||||
containers: list[str],
|
|
||||||
vol_dir: str,
|
|
||||||
databases_df: "pandas.DataFrame",
|
|
||||||
database_containers: list[str],
|
|
||||||
) -> tuple[bool, bool]:
|
|
||||||
"""
|
|
||||||
Returns (found_db_container, dumped_any)
|
|
||||||
"""
|
|
||||||
found_db = False
|
|
||||||
dumped_any = False
|
|
||||||
|
|
||||||
for c in containers:
|
|
||||||
is_db, dumped = backup_mariadb_or_postgres(
|
|
||||||
container=c,
|
|
||||||
volume_dir=vol_dir,
|
|
||||||
databases_df=databases_df,
|
|
||||||
database_containers=database_containers,
|
|
||||||
)
|
|
||||||
if is_db:
|
|
||||||
found_db = True
|
|
||||||
if dumped:
|
|
||||||
dumped_any = True
|
|
||||||
|
|
||||||
return found_db, dumped_any
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
def main() -> int:
|
||||||
@@ -183,66 +41,89 @@ def main() -> int:
|
|||||||
#
|
#
|
||||||
# Robust behavior:
|
# Robust behavior:
|
||||||
# - if the file is missing or empty, we continue without DB dumps.
|
# - if the file is missing or empty, we continue without DB dumps.
|
||||||
databases_df = _load_databases_df(args.databases_csv)
|
databases_df = load_databases_df(args.databases_csv)
|
||||||
|
|
||||||
print("💾 Start volume backups...", flush=True)
|
print("💾 Start volume backups...", flush=True)
|
||||||
|
|
||||||
for volume_name in docker_volume_names():
|
with ExitStack() as stack:
|
||||||
print(f"Start backup routine for volume: {volume_name}", flush=True)
|
resolve_source = None
|
||||||
containers = containers_using_volume(volume_name)
|
if args.snapshot:
|
||||||
|
resolve_source = stack.enter_context(
|
||||||
# EARLY SKIP: if all linked containers are ignored, do not create any dirs
|
volume_snapshot(args.snapshot, args.snapshot_subject, backup_time)
|
||||||
if volume_is_fully_ignored(containers, args.images_no_backup_required):
|
|
||||||
print(
|
|
||||||
f"Skipping volume '{volume_name}' entirely (all linked containers are ignored).",
|
|
||||||
flush=True,
|
|
||||||
)
|
)
|
||||||
continue
|
|
||||||
|
|
||||||
vol_dir = create_volume_directory(version_dir, volume_name)
|
for volume_name in docker_volume_names():
|
||||||
|
print(f"Start backup routine for volume: {volume_name}", flush=True)
|
||||||
|
containers = containers_using_volume(volume_name)
|
||||||
|
|
||||||
found_db, dumped_any = _backup_dumps_for_volume(
|
if volume_is_fully_ignored(containers, args.images_no_backup_required):
|
||||||
containers=containers,
|
print(
|
||||||
vol_dir=vol_dir,
|
f"Skipping volume '{volume_name}' entirely (all linked containers are ignored).",
|
||||||
databases_df=databases_df,
|
flush=True,
|
||||||
database_containers=args.database_containers,
|
)
|
||||||
)
|
continue
|
||||||
|
|
||||||
# dump-only-sql logic:
|
vol_dir = create_volume_directory(version_dir, volume_name)
|
||||||
if args.dump_only_sql:
|
|
||||||
if found_db:
|
found_db, dumped_any = backup_dumps_for_volume(
|
||||||
if not dumped_any:
|
containers=containers,
|
||||||
|
vol_dir=vol_dir,
|
||||||
|
databases_df=databases_df,
|
||||||
|
database_containers=args.database_containers,
|
||||||
|
)
|
||||||
|
|
||||||
|
if args.dump_only_sql:
|
||||||
|
if found_db:
|
||||||
|
if not dumped_any:
|
||||||
|
print(
|
||||||
|
f"WARNING: dump-only-sql requested but no DB dump was produced for DB volume '{volume_name}'. "
|
||||||
|
"Falling back to file backup.",
|
||||||
|
flush=True,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
continue
|
||||||
|
|
||||||
|
live_source = get_storage_path(volume_name)
|
||||||
|
|
||||||
|
def copy(*, authoritative: bool, source: str = live_source) -> None:
|
||||||
|
backup_volume(
|
||||||
|
versions_dir,
|
||||||
|
volume_name,
|
||||||
|
vol_dir,
|
||||||
|
authoritative=authoritative,
|
||||||
|
source=source,
|
||||||
|
)
|
||||||
|
|
||||||
|
if resolve_source is not None:
|
||||||
|
snapshot_source = resolve_source(live_source)
|
||||||
|
if os.path.isdir(snapshot_source):
|
||||||
|
copy(authoritative=True, source=snapshot_source)
|
||||||
|
else:
|
||||||
print(
|
print(
|
||||||
f"WARNING: dump-only-sql requested but no DB dump was produced for DB volume '{volume_name}'. "
|
f"WARNING: volume '{volume_name}' is not in the snapshot "
|
||||||
"Falling back to file backup.",
|
"(created after it was taken); copying it live instead.",
|
||||||
flush=True,
|
flush=True,
|
||||||
)
|
)
|
||||||
# fall through to file backup below
|
copy(authoritative=False)
|
||||||
else:
|
continue
|
||||||
# DB volume successfully dumped -> skip file backup
|
|
||||||
continue
|
|
||||||
# Non-DB volume -> always do file backup (fall through)
|
|
||||||
|
|
||||||
if args.everything:
|
if args.everything:
|
||||||
# "everything": always do pre-rsync, then stop + rsync again
|
stoppable = filter_stoppable(containers)
|
||||||
stoppable = filter_stoppable(containers)
|
copy(authoritative=False)
|
||||||
backup_volume(versions_dir, volume_name, vol_dir)
|
change_containers_status(stoppable, "stop")
|
||||||
change_containers_status(stoppable, "stop")
|
copy(authoritative=True)
|
||||||
backup_volume(versions_dir, volume_name, vol_dir)
|
if not args.shutdown:
|
||||||
if not args.shutdown:
|
change_containers_status(stoppable, "start")
|
||||||
change_containers_status(stoppable, "start")
|
continue
|
||||||
continue
|
|
||||||
|
|
||||||
# default: rsync, and if needed stop + rsync
|
copy(authoritative=False)
|
||||||
backup_volume(versions_dir, volume_name, vol_dir)
|
if requires_stop(containers, args.images_no_stop_required):
|
||||||
if requires_stop(containers, args.images_no_stop_required):
|
stoppable = filter_stoppable(containers)
|
||||||
stoppable = filter_stoppable(containers)
|
change_containers_status(stoppable, "stop")
|
||||||
change_containers_status(stoppable, "stop")
|
copy(authoritative=True)
|
||||||
backup_volume(versions_dir, volume_name, vol_dir)
|
if not args.shutdown:
|
||||||
if not args.shutdown:
|
change_containers_status(stoppable, "start")
|
||||||
change_containers_status(stoppable, "start")
|
|
||||||
|
|
||||||
# Stamp the backup version directory using dirval (python lib)
|
|
||||||
stamp_directory(version_dir)
|
stamp_directory(version_dir)
|
||||||
print("Finished volume backups.", flush=True)
|
print("Finished volume backups.", flush=True)
|
||||||
|
|
||||||
|
|||||||
@@ -39,6 +39,16 @@ def parse_args() -> argparse.Namespace:
|
|||||||
help="Backup root directory (e.g. /var/lib/backup/)",
|
help="Backup root directory (e.g. /var/lib/backup/)",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
p.add_argument(
|
||||||
|
"--snapshot",
|
||||||
|
choices=["btrfs", "zfs"],
|
||||||
|
help="Capture every volume from one atomic filesystem snapshot instead of copying the live tree. Containers are not stopped, and the copy is a single pass. Requires --snapshot-subject. Omit to keep the live two-pass copy.",
|
||||||
|
)
|
||||||
|
p.add_argument(
|
||||||
|
"--snapshot-subject",
|
||||||
|
help="Btrfs subvolume or zfs dataset mountpoint holding the docker volumes, e.g. /var/lib/docker. Required with --snapshot.",
|
||||||
|
)
|
||||||
|
|
||||||
p.add_argument(
|
p.add_argument(
|
||||||
"--database-containers",
|
"--database-containers",
|
||||||
nargs="+",
|
nargs="+",
|
||||||
@@ -79,4 +89,14 @@ def parse_args() -> argparse.Namespace:
|
|||||||
"If a DB dump cannot be produced, baudolo falls back to a file backup."
|
"If a DB dump cannot be produced, baudolo falls back to a file backup."
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
return p.parse_args()
|
args = p.parse_args()
|
||||||
|
if bool(args.snapshot) != bool(args.snapshot_subject):
|
||||||
|
p.error("--snapshot and --snapshot-subject must be given together")
|
||||||
|
if args.snapshot and args.shutdown:
|
||||||
|
p.error("--shutdown is meaningless with --snapshot: containers are never stopped")
|
||||||
|
if args.snapshot and args.hard_restart_projects:
|
||||||
|
p.error(
|
||||||
|
"--hard-restart-projects is meaningless with --snapshot: the flag exists "
|
||||||
|
"for stacks whose database cannot be backed up hot, which a snapshot solves"
|
||||||
|
)
|
||||||
|
return args
|
||||||
|
|||||||
98
src/baudolo/backup/dumps.py
Normal file
98
src/baudolo/backup/dumps.py
Normal file
@@ -0,0 +1,98 @@
|
|||||||
|
"""Database dumps taken before a volume's files are copied."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import sys
|
||||||
|
|
||||||
|
import pandas
|
||||||
|
from pandas.errors import EmptyDataError
|
||||||
|
|
||||||
|
from .db import backup_database
|
||||||
|
from .docker import has_image
|
||||||
|
|
||||||
|
|
||||||
|
def backup_mariadb_or_postgres(
|
||||||
|
*,
|
||||||
|
container: str,
|
||||||
|
volume_dir: str,
|
||||||
|
databases_df: "pandas.DataFrame",
|
||||||
|
database_containers: list[str],
|
||||||
|
) -> tuple[bool, bool]:
|
||||||
|
"""
|
||||||
|
Returns (is_db_container, dumped_any)
|
||||||
|
"""
|
||||||
|
for img in ["mariadb", "postgres"]:
|
||||||
|
if has_image(container, img):
|
||||||
|
dumped = backup_database(
|
||||||
|
container=container,
|
||||||
|
volume_dir=volume_dir,
|
||||||
|
db_type=img,
|
||||||
|
databases_df=databases_df,
|
||||||
|
database_containers=database_containers,
|
||||||
|
)
|
||||||
|
return True, dumped
|
||||||
|
return False, False
|
||||||
|
|
||||||
|
|
||||||
|
def _empty_databases_df() -> "pandas.DataFrame":
|
||||||
|
"""
|
||||||
|
Create an empty DataFrame with the expected schema for databases.csv.
|
||||||
|
|
||||||
|
This allows the backup to continue without DB dumps when the CSV is missing
|
||||||
|
or empty (pandas EmptyDataError).
|
||||||
|
"""
|
||||||
|
return pandas.DataFrame(columns=["instance", "database", "username", "password"])
|
||||||
|
|
||||||
|
|
||||||
|
def load_databases_df(csv_path: str) -> "pandas.DataFrame":
|
||||||
|
"""
|
||||||
|
Load databases.csv robustly.
|
||||||
|
|
||||||
|
- Missing file -> warn, continue with empty df
|
||||||
|
- Empty file -> warn, continue with empty df
|
||||||
|
- Valid CSV -> return dataframe
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
return pandas.read_csv(csv_path, sep=";", keep_default_na=False, dtype=str)
|
||||||
|
except FileNotFoundError:
|
||||||
|
print(
|
||||||
|
f"WARNING: databases.csv not found: {csv_path}. Continuing without database dumps.",
|
||||||
|
file=sys.stderr,
|
||||||
|
flush=True,
|
||||||
|
)
|
||||||
|
return _empty_databases_df()
|
||||||
|
except EmptyDataError:
|
||||||
|
print(
|
||||||
|
f"WARNING: databases.csv exists but is empty: {csv_path}. Continuing without database dumps.",
|
||||||
|
file=sys.stderr,
|
||||||
|
flush=True,
|
||||||
|
)
|
||||||
|
return _empty_databases_df()
|
||||||
|
|
||||||
|
|
||||||
|
def backup_dumps_for_volume(
|
||||||
|
*,
|
||||||
|
containers: list[str],
|
||||||
|
vol_dir: str,
|
||||||
|
databases_df: "pandas.DataFrame",
|
||||||
|
database_containers: list[str],
|
||||||
|
) -> tuple[bool, bool]:
|
||||||
|
"""
|
||||||
|
Returns (found_db_container, dumped_any)
|
||||||
|
"""
|
||||||
|
found_db = False
|
||||||
|
dumped_any = False
|
||||||
|
|
||||||
|
for c in containers:
|
||||||
|
is_db, dumped = backup_mariadb_or_postgres(
|
||||||
|
container=c,
|
||||||
|
volume_dir=vol_dir,
|
||||||
|
databases_df=databases_df,
|
||||||
|
database_containers=database_containers,
|
||||||
|
)
|
||||||
|
if is_db:
|
||||||
|
found_db = True
|
||||||
|
if dumped:
|
||||||
|
dumped_any = True
|
||||||
|
|
||||||
|
return found_db, dumped_any
|
||||||
33
src/baudolo/backup/layout.py
Normal file
33
src/baudolo/backup/layout.py
Normal file
@@ -0,0 +1,33 @@
|
|||||||
|
"""Where a backup run puts its files, and how a finished run is stamped."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import pathlib
|
||||||
|
|
||||||
|
from dirval import create_stamp_file
|
||||||
|
|
||||||
|
from .shell import execute_shell_command
|
||||||
|
|
||||||
|
|
||||||
|
def get_machine_id() -> str:
|
||||||
|
return execute_shell_command("sha256sum /etc/machine-id")[0][0:64]
|
||||||
|
|
||||||
|
|
||||||
|
def stamp_directory(version_dir: str) -> None:
|
||||||
|
"""
|
||||||
|
Use dirval as a Python library to stamp the directory (no CLI dependency).
|
||||||
|
"""
|
||||||
|
create_stamp_file(version_dir)
|
||||||
|
|
||||||
|
|
||||||
|
def create_version_directory(versions_dir: str, backup_time: str) -> str:
|
||||||
|
version_dir = os.path.join(versions_dir, backup_time)
|
||||||
|
pathlib.Path(version_dir).mkdir(parents=True, exist_ok=True)
|
||||||
|
return version_dir
|
||||||
|
|
||||||
|
|
||||||
|
def create_volume_directory(version_dir: str, volume_name: str) -> str:
|
||||||
|
path = os.path.join(version_dir, volume_name)
|
||||||
|
pathlib.Path(path).mkdir(parents=True, exist_ok=True)
|
||||||
|
return path
|
||||||
38
src/baudolo/backup/policy.py
Normal file
38
src/baudolo/backup/policy.py
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
"""Which volumes are backed up, and which containers must stop for it."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from .docker import get_image_info, is_swarm_task
|
||||||
|
|
||||||
|
|
||||||
|
def is_image_ignored(container: str, images_no_backup_required: list[str]) -> bool:
|
||||||
|
if not images_no_backup_required:
|
||||||
|
return False
|
||||||
|
img = get_image_info(container)
|
||||||
|
return img in images_no_backup_required
|
||||||
|
|
||||||
|
|
||||||
|
def volume_is_fully_ignored(
|
||||||
|
containers: list[str], images_no_backup_required: list[str]
|
||||||
|
) -> bool:
|
||||||
|
"""
|
||||||
|
Skip file backup only if all containers linked to the volume are ignored.
|
||||||
|
"""
|
||||||
|
if not containers:
|
||||||
|
return False
|
||||||
|
return all(is_image_ignored(c, images_no_backup_required) for c in containers)
|
||||||
|
|
||||||
|
|
||||||
|
def requires_stop(containers: list[str], images_no_stop_required: list[str]) -> bool:
|
||||||
|
"""
|
||||||
|
Stop is required if ANY stoppable container image is NOT in the exact
|
||||||
|
image whitelist. Swarm task containers never count: baudolo must
|
||||||
|
not cycle them (see docker.is_swarm_task).
|
||||||
|
"""
|
||||||
|
for c in containers:
|
||||||
|
if is_swarm_task(c):
|
||||||
|
continue
|
||||||
|
img = get_image_info(c)
|
||||||
|
if img not in images_no_stop_required:
|
||||||
|
return True
|
||||||
|
return False
|
||||||
95
src/baudolo/backup/snapshot.py
Normal file
95
src/baudolo/backup/snapshot.py
Normal file
@@ -0,0 +1,95 @@
|
|||||||
|
"""Capture every volume from one atomic filesystem snapshot.
|
||||||
|
|
||||||
|
Copying a live tree file by file cannot produce a point in time: a database can
|
||||||
|
write between two files and leave a control file and its write-ahead log
|
||||||
|
disagreeing, which no recovery can repair. A snapshot freezes the whole subject
|
||||||
|
at once, so a database reads it as a crash and replays its log - a case it is
|
||||||
|
built for. That also removes the reason to stop containers at all.
|
||||||
|
|
||||||
|
The snapshot kind is stated by the caller rather than probed, because falling
|
||||||
|
back to a live copy when a probe is inconclusive would hand out backups that
|
||||||
|
look consistent and are not.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
from collections.abc import Callable, Iterator
|
||||||
|
from contextlib import contextmanager
|
||||||
|
|
||||||
|
from .shell import BackupException, execute_shell_command
|
||||||
|
|
||||||
|
KINDS = ("btrfs", "zfs")
|
||||||
|
|
||||||
|
|
||||||
|
class SnapshotError(RuntimeError):
|
||||||
|
"""A snapshot could not be created, resolved or removed."""
|
||||||
|
|
||||||
|
|
||||||
|
def _resolver(subject: str, root: str) -> Callable[[str], str]:
|
||||||
|
def resolve(path: str) -> str:
|
||||||
|
relative = os.path.relpath(os.path.abspath(path), os.path.abspath(subject))
|
||||||
|
if relative.startswith(".."):
|
||||||
|
raise SnapshotError(f"{path} lies outside the snapshot subject {subject}")
|
||||||
|
resolved = root if relative == "." else os.path.join(root, relative)
|
||||||
|
|
||||||
|
# abspath drops a trailing separator, and rsync reads "dir/" as its
|
||||||
|
# contents where "dir" means the directory itself.
|
||||||
|
return resolved + os.sep if path.endswith(os.sep) else resolved
|
||||||
|
|
||||||
|
return resolve
|
||||||
|
|
||||||
|
|
||||||
|
def _btrfs(subject: str, name: str, run: Callable[[str], list[str]]) -> tuple[str, str]:
|
||||||
|
target = os.path.join(os.path.dirname(os.path.abspath(subject)), f".{name}")
|
||||||
|
run(f"btrfs subvolume snapshot -r {subject} {target}")
|
||||||
|
return target, f"btrfs subvolume delete {target}"
|
||||||
|
|
||||||
|
|
||||||
|
def _zfs(subject: str, name: str, run: Callable[[str], list[str]]) -> tuple[str, str]:
|
||||||
|
output = run(f"zfs list -H -o name {subject}")
|
||||||
|
dataset = (output[0] if output else "").strip()
|
||||||
|
if not dataset:
|
||||||
|
raise SnapshotError(f"no zfs dataset is mounted at {subject}")
|
||||||
|
run(f"zfs snapshot {dataset}@{name}")
|
||||||
|
root = os.path.join(subject, ".zfs", "snapshot", name)
|
||||||
|
return root, f"zfs destroy {dataset}@{name}"
|
||||||
|
|
||||||
|
|
||||||
|
_CREATE = {"btrfs": _btrfs, "zfs": _zfs}
|
||||||
|
|
||||||
|
|
||||||
|
@contextmanager
|
||||||
|
def volume_snapshot(
|
||||||
|
kind: str,
|
||||||
|
subject: str,
|
||||||
|
tag: str,
|
||||||
|
run: Callable[[str], list[str]] = execute_shell_command,
|
||||||
|
) -> Iterator[Callable[[str], str]]:
|
||||||
|
"""Yield a resolver mapping a path under ``subject`` into a snapshot of it.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
kind: ``btrfs`` or ``zfs``; the caller states it, nothing is probed.
|
||||||
|
subject: the btrfs subvolume or zfs dataset mountpoint holding the
|
||||||
|
volumes, e.g. ``/var/lib/docker``.
|
||||||
|
tag: unique suffix for the snapshot name, e.g. the backup timestamp.
|
||||||
|
run: shell runner, injected so the mechanics are testable.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
SnapshotError: the kind is unknown, or the snapshot cannot be created.
|
||||||
|
Removal failure is reported, not raised: a leftover snapshot is a
|
||||||
|
cleanup problem and must not discard a generation that is complete.
|
||||||
|
"""
|
||||||
|
create = _CREATE.get(kind)
|
||||||
|
if create is None:
|
||||||
|
raise SnapshotError(f"unknown snapshot kind {kind!r}; expected one of {KINDS}")
|
||||||
|
|
||||||
|
root, remove = create(subject, f"baudolo-{tag}", run)
|
||||||
|
try:
|
||||||
|
yield _resolver(subject, root)
|
||||||
|
finally:
|
||||||
|
try:
|
||||||
|
run(remove)
|
||||||
|
except BackupException as error:
|
||||||
|
# Raising here would also mask whatever the body raised.
|
||||||
|
print(f"WARNING: {root} could not be removed: {error}", flush=True)
|
||||||
@@ -24,16 +24,32 @@ def get_last_backup_dir(
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
def backup_volume(versions_dir: str, volume_name: str, volume_dir: str) -> None:
|
def backup_volume(
|
||||||
"""Perform incremental file backup of a Docker volume."""
|
versions_dir: str,
|
||||||
|
volume_name: str,
|
||||||
|
volume_dir: str,
|
||||||
|
*,
|
||||||
|
authoritative: bool,
|
||||||
|
source: str,
|
||||||
|
) -> None:
|
||||||
|
"""Perform incremental file backup of a Docker volume.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
authoritative: compare source and destination by content instead of by
|
||||||
|
size and whole-second mtime. Required on a pass whose destination was
|
||||||
|
already written from a live source, where a file can differ while
|
||||||
|
both attributes still agree.
|
||||||
|
source: directory to read from - the volume's mountpoint, or its path
|
||||||
|
inside a snapshot.
|
||||||
|
"""
|
||||||
dest = os.path.join(volume_dir, "files") + "/"
|
dest = os.path.join(volume_dir, "files") + "/"
|
||||||
pathlib.Path(dest).mkdir(parents=True, exist_ok=True)
|
pathlib.Path(dest).mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
last = get_last_backup_dir(versions_dir, volume_name, dest)
|
last = get_last_backup_dir(versions_dir, volume_name, dest)
|
||||||
link_dest = f"--link-dest='{last}'" if last else ""
|
link_dest = f"--link-dest='{last}'" if last else ""
|
||||||
source = get_storage_path(volume_name)
|
verify = "--checksum " if authoritative else ""
|
||||||
|
|
||||||
cmd = f"rsync -abP --delete --delete-excluded {link_dest} {source} {dest}"
|
cmd = f"rsync -abP --delete --delete-excluded {verify}{link_dest} {source} {dest}"
|
||||||
|
|
||||||
try:
|
try:
|
||||||
execute_shell_command(cmd)
|
execute_shell_command(cmd)
|
||||||
|
|||||||
4
tests/e2e/helpers/__init__.py
Normal file
4
tests/e2e/helpers/__init__.py
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
"""Shared e2e helpers, re-exported so tests import one name."""
|
||||||
|
|
||||||
|
from .fixtures import * # noqa: F401,F403
|
||||||
|
from .process import * # noqa: F401,F403
|
||||||
114
tests/e2e/helpers/fixtures.py
Normal file
114
tests/e2e/helpers/fixtures.py
Normal file
@@ -0,0 +1,114 @@
|
|||||||
|
"""Fixtures and paths the e2e suite builds its scenarios from."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from .process import machine_hash, run
|
||||||
|
|
||||||
|
# postgres 18+ mounts at /var/lib/postgresql, not /var/lib/postgresql/data.
|
||||||
|
POSTGRES_IMAGE = "postgres:alpine"
|
||||||
|
POSTGRES_DATA_DIR = "/var/lib/postgresql"
|
||||||
|
MARIADB_IMAGE = "mariadb:latest"
|
||||||
|
MARIADB_DATA_DIR = "/var/lib/mysql"
|
||||||
|
|
||||||
|
|
||||||
|
def backup_run(
|
||||||
|
*,
|
||||||
|
backups_dir: str,
|
||||||
|
repo_name: str,
|
||||||
|
compose_dir: str,
|
||||||
|
databases_csv: str,
|
||||||
|
database_containers: list[str],
|
||||||
|
images_no_stop_required: list[str],
|
||||||
|
images_no_backup_required: list[str] | None = None,
|
||||||
|
dump_only_sql: bool = False,
|
||||||
|
) -> None:
|
||||||
|
cmd = [
|
||||||
|
"baudolo",
|
||||||
|
"--compose-dir",
|
||||||
|
compose_dir,
|
||||||
|
"--hard-restart-projects",
|
||||||
|
"mailu",
|
||||||
|
"--repo-name",
|
||||||
|
repo_name,
|
||||||
|
"--databases-csv",
|
||||||
|
databases_csv,
|
||||||
|
"--backups-dir",
|
||||||
|
backups_dir,
|
||||||
|
"--database-containers",
|
||||||
|
*database_containers,
|
||||||
|
"--images-no-stop-required",
|
||||||
|
*images_no_stop_required,
|
||||||
|
]
|
||||||
|
if images_no_backup_required:
|
||||||
|
cmd += ["--images-no-backup-required", *images_no_backup_required]
|
||||||
|
if dump_only_sql:
|
||||||
|
cmd += ["--dump-only-sql"]
|
||||||
|
|
||||||
|
try:
|
||||||
|
run(cmd, capture=True, check=True)
|
||||||
|
except subprocess.CalledProcessError as e:
|
||||||
|
print(">>> baudolo failed (exit code:", e.returncode, ")")
|
||||||
|
if e.stdout:
|
||||||
|
print(">>> baudolo STDOUT:\n" + e.stdout)
|
||||||
|
if e.stderr:
|
||||||
|
print(">>> baudolo STDERR:\n" + e.stderr)
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
def latest_version_dir(backups_dir: str, repo_name: str) -> tuple[str, str]:
|
||||||
|
"""
|
||||||
|
Returns (hash, version) for the latest backup.
|
||||||
|
"""
|
||||||
|
h = machine_hash()
|
||||||
|
root = Path(backups_dir) / h / repo_name
|
||||||
|
if not root.is_dir():
|
||||||
|
raise FileNotFoundError(str(root))
|
||||||
|
|
||||||
|
versions = sorted([p.name for p in root.iterdir() if p.is_dir()])
|
||||||
|
if not versions:
|
||||||
|
raise RuntimeError(f"No versions found under {root}")
|
||||||
|
return h, versions[-1]
|
||||||
|
|
||||||
|
|
||||||
|
def backup_path(backups_dir: str, repo_name: str, version: str, volume: str) -> Path:
|
||||||
|
h = machine_hash()
|
||||||
|
return Path(backups_dir) / h / repo_name / version / volume
|
||||||
|
|
||||||
|
|
||||||
|
def create_minimal_compose_dir(base: str) -> str:
|
||||||
|
"""
|
||||||
|
baudolo requires --compose-dir. Create an empty dir with one non-compose subdir.
|
||||||
|
"""
|
||||||
|
p = Path(base) / "compose-root"
|
||||||
|
p.mkdir(parents=True, exist_ok=True)
|
||||||
|
(p / "noop").mkdir(parents=True, exist_ok=True)
|
||||||
|
return str(p)
|
||||||
|
|
||||||
|
|
||||||
|
def write_databases_csv(path: str, rows: list[tuple[str, str, str, str]]) -> None:
|
||||||
|
"""
|
||||||
|
rows: (instance, database, username, password)
|
||||||
|
database may be '' (empty) to trigger pg_dumpall behavior if you want, but here we use db name.
|
||||||
|
"""
|
||||||
|
Path(path).parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with open(path, "w", encoding="utf-8") as f:
|
||||||
|
f.write("instance;database;username;password\n")
|
||||||
|
for inst, db, user, pw in rows:
|
||||||
|
f.write(f"{inst};{db};{user};{pw}\n")
|
||||||
|
|
||||||
|
|
||||||
|
def cleanup_docker(*, containers: list[str], volumes: list[str]) -> None:
|
||||||
|
for c in containers:
|
||||||
|
run(["docker", "rm", "-f", c], capture=True, check=False)
|
||||||
|
for v in volumes:
|
||||||
|
run(["docker", "volume", "rm", "-f", v], capture=True, check=False)
|
||||||
|
|
||||||
|
|
||||||
|
def ensure_empty_dir(path: str) -> None:
|
||||||
|
p = Path(path)
|
||||||
|
if p.exists():
|
||||||
|
shutil.rmtree(p)
|
||||||
|
p.mkdir(parents=True, exist_ok=True)
|
||||||
@@ -1,19 +1,9 @@
|
|||||||
# tests/e2e/helpers.py
|
"""Process, docker and readiness helpers for the e2e suite."""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import shutil
|
|
||||||
import subprocess
|
import subprocess
|
||||||
import time
|
import time
|
||||||
import uuid
|
import uuid
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
# SPOT for the database images and their in-container data dirs the e2e
|
|
||||||
# suite runs against. postgres:alpine tracks latest (18+), which mounts at
|
|
||||||
# /var/lib/postgresql (not /var/lib/postgresql/data); bump here only.
|
|
||||||
POSTGRES_IMAGE = "postgres:alpine"
|
|
||||||
POSTGRES_DATA_DIR = "/var/lib/postgresql"
|
|
||||||
MARIADB_IMAGE = "mariadb:latest"
|
|
||||||
MARIADB_DATA_DIR = "/var/lib/mysql"
|
|
||||||
|
|
||||||
|
|
||||||
def run(
|
def run(
|
||||||
@@ -163,103 +153,3 @@ def wait_for_mariadb_sql(
|
|||||||
raise TimeoutError(
|
raise TimeoutError(
|
||||||
f"Timed out waiting for MariaDB SQL login readiness in container {container}"
|
f"Timed out waiting for MariaDB SQL login readiness in container {container}"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def backup_run(
|
|
||||||
*,
|
|
||||||
backups_dir: str,
|
|
||||||
repo_name: str,
|
|
||||||
compose_dir: str,
|
|
||||||
databases_csv: str,
|
|
||||||
database_containers: list[str],
|
|
||||||
images_no_stop_required: list[str],
|
|
||||||
images_no_backup_required: list[str] | None = None,
|
|
||||||
dump_only_sql: bool = False,
|
|
||||||
) -> None:
|
|
||||||
cmd = [
|
|
||||||
"baudolo",
|
|
||||||
"--compose-dir",
|
|
||||||
compose_dir,
|
|
||||||
"--hard-restart-projects",
|
|
||||||
"mailu",
|
|
||||||
"--repo-name",
|
|
||||||
repo_name,
|
|
||||||
"--databases-csv",
|
|
||||||
databases_csv,
|
|
||||||
"--backups-dir",
|
|
||||||
backups_dir,
|
|
||||||
"--database-containers",
|
|
||||||
*database_containers,
|
|
||||||
"--images-no-stop-required",
|
|
||||||
*images_no_stop_required,
|
|
||||||
]
|
|
||||||
if images_no_backup_required:
|
|
||||||
cmd += ["--images-no-backup-required", *images_no_backup_required]
|
|
||||||
if dump_only_sql:
|
|
||||||
cmd += ["--dump-only-sql"]
|
|
||||||
|
|
||||||
try:
|
|
||||||
run(cmd, capture=True, check=True)
|
|
||||||
except subprocess.CalledProcessError as e:
|
|
||||||
print(">>> baudolo failed (exit code:", e.returncode, ")")
|
|
||||||
if e.stdout:
|
|
||||||
print(">>> baudolo STDOUT:\n" + e.stdout)
|
|
||||||
if e.stderr:
|
|
||||||
print(">>> baudolo STDERR:\n" + e.stderr)
|
|
||||||
raise
|
|
||||||
|
|
||||||
|
|
||||||
def latest_version_dir(backups_dir: str, repo_name: str) -> tuple[str, str]:
|
|
||||||
"""
|
|
||||||
Returns (hash, version) for the latest backup.
|
|
||||||
"""
|
|
||||||
h = machine_hash()
|
|
||||||
root = Path(backups_dir) / h / repo_name
|
|
||||||
if not root.is_dir():
|
|
||||||
raise FileNotFoundError(str(root))
|
|
||||||
|
|
||||||
versions = sorted([p.name for p in root.iterdir() if p.is_dir()])
|
|
||||||
if not versions:
|
|
||||||
raise RuntimeError(f"No versions found under {root}")
|
|
||||||
return h, versions[-1]
|
|
||||||
|
|
||||||
|
|
||||||
def backup_path(backups_dir: str, repo_name: str, version: str, volume: str) -> Path:
|
|
||||||
h = machine_hash()
|
|
||||||
return Path(backups_dir) / h / repo_name / version / volume
|
|
||||||
|
|
||||||
|
|
||||||
def create_minimal_compose_dir(base: str) -> str:
|
|
||||||
"""
|
|
||||||
baudolo requires --compose-dir. Create an empty dir with one non-compose subdir.
|
|
||||||
"""
|
|
||||||
p = Path(base) / "compose-root"
|
|
||||||
p.mkdir(parents=True, exist_ok=True)
|
|
||||||
(p / "noop").mkdir(parents=True, exist_ok=True)
|
|
||||||
return str(p)
|
|
||||||
|
|
||||||
|
|
||||||
def write_databases_csv(path: str, rows: list[tuple[str, str, str, str]]) -> None:
|
|
||||||
"""
|
|
||||||
rows: (instance, database, username, password)
|
|
||||||
database may be '' (empty) to trigger pg_dumpall behavior if you want, but here we use db name.
|
|
||||||
"""
|
|
||||||
Path(path).parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
with open(path, "w", encoding="utf-8") as f:
|
|
||||||
f.write("instance;database;username;password\n")
|
|
||||||
for inst, db, user, pw in rows:
|
|
||||||
f.write(f"{inst};{db};{user};{pw}\n")
|
|
||||||
|
|
||||||
|
|
||||||
def cleanup_docker(*, containers: list[str], volumes: list[str]) -> None:
|
|
||||||
for c in containers:
|
|
||||||
run(["docker", "rm", "-f", c], capture=True, check=False)
|
|
||||||
for v in volumes:
|
|
||||||
run(["docker", "volume", "rm", "-f", v], capture=True, check=False)
|
|
||||||
|
|
||||||
|
|
||||||
def ensure_empty_dir(path: str) -> None:
|
|
||||||
p = Path(path)
|
|
||||||
if p.exists():
|
|
||||||
shutil.rmtree(p)
|
|
||||||
p.mkdir(parents=True, exist_ok=True)
|
|
||||||
41
tests/e2e/snapshot_db_driver.py
Normal file
41
tests/e2e/snapshot_db_driver.py
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
"""Copy a live database's volume out of a snapshot, using the real backup path.
|
||||||
|
|
||||||
|
Runs inside the privileged container built by test_e2e_snapshot_db.py, where a
|
||||||
|
database is mid-write on a btrfs subvolume. Exercises volume_snapshot and
|
||||||
|
backup_volume exactly as a backup run would.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
|
||||||
|
sys.path.insert(0, "/src")
|
||||||
|
|
||||||
|
from baudolo.backup.snapshot import SnapshotError, volume_snapshot # noqa: E402
|
||||||
|
from baudolo.backup.volume import backup_volume # noqa: E402
|
||||||
|
|
||||||
|
SUBJECT = "/subject/docker"
|
||||||
|
VOLUME = "mariadb_data"
|
||||||
|
DATADIR = f"{SUBJECT}/volumes/{VOLUME}/_data"
|
||||||
|
VERSIONS = "/backups"
|
||||||
|
GENERATION = f"{VERSIONS}/20260731"
|
||||||
|
|
||||||
|
|
||||||
|
def shell(command: str) -> list[str]:
|
||||||
|
proc = subprocess.run(command, shell=True, capture_output=True, text=True)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
raise SnapshotError(f"{command} exited {proc.returncode}: {proc.stderr.strip()}")
|
||||||
|
return proc.stdout.splitlines()
|
||||||
|
|
||||||
|
|
||||||
|
with volume_snapshot("btrfs", SUBJECT, "dbtest", run=shell) as resolve:
|
||||||
|
backup_volume(
|
||||||
|
VERSIONS,
|
||||||
|
VOLUME,
|
||||||
|
f"{GENERATION}/{VOLUME}",
|
||||||
|
authoritative=True,
|
||||||
|
source=resolve(f"{DATADIR}/"),
|
||||||
|
)
|
||||||
|
|
||||||
|
print("SNAPSHOT COPY DONE", flush=True)
|
||||||
59
tests/e2e/snapshot_driver.py
Normal file
59
tests/e2e/snapshot_driver.py
Normal file
@@ -0,0 +1,59 @@
|
|||||||
|
"""Exercise volume_snapshot against a real filesystem, from inside a container.
|
||||||
|
|
||||||
|
Runs where loop devices exist. Prints one PASS/FAIL line per assertion and exits
|
||||||
|
non-zero on the first failure, so the calling test can surface the reason.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
sys.path.insert(0, "/src")
|
||||||
|
|
||||||
|
from baudolo.backup.snapshot import SnapshotError, volume_snapshot # noqa: E402
|
||||||
|
|
||||||
|
KIND = sys.argv[1]
|
||||||
|
SUBJECT = sys.argv[2]
|
||||||
|
EXPECT = sys.argv[3]
|
||||||
|
|
||||||
|
|
||||||
|
def shell(command: str) -> list[str]:
|
||||||
|
proc = subprocess.run(command, shell=True, capture_output=True, text=True)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
raise SnapshotError(f"{command} exited {proc.returncode}: {proc.stderr.strip()}")
|
||||||
|
return proc.stdout.splitlines()
|
||||||
|
|
||||||
|
|
||||||
|
def check(label: str, condition: bool) -> None:
|
||||||
|
print(f"{'PASS' if condition else 'FAIL'} {label}", flush=True)
|
||||||
|
if not condition:
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
|
||||||
|
volume = Path(SUBJECT) / "volumes" / "demo" / "_data"
|
||||||
|
volume.mkdir(parents=True, exist_ok=True)
|
||||||
|
(volume / "state").write_text("before\n")
|
||||||
|
|
||||||
|
if EXPECT == "unsupported":
|
||||||
|
try:
|
||||||
|
with volume_snapshot(KIND, SUBJECT, "e2e", run=shell):
|
||||||
|
check("snapshot on an unsupported filesystem must not succeed", False)
|
||||||
|
except SnapshotError as exc:
|
||||||
|
check(f"refused loudly: {str(exc)[:60]}", True)
|
||||||
|
sys.exit(0)
|
||||||
|
|
||||||
|
with volume_snapshot(KIND, SUBJECT, "e2e", run=shell) as resolve:
|
||||||
|
frozen = Path(resolve(str(volume))) / "state"
|
||||||
|
check("the snapshot exposes the volume", frozen.is_file())
|
||||||
|
check("the snapshot carries the content", frozen.read_text() == "before\n")
|
||||||
|
|
||||||
|
(volume / "state").write_text("after\n")
|
||||||
|
check("a later write does not reach the snapshot", frozen.read_text() == "before\n")
|
||||||
|
check("the live tree did change", (volume / "state").read_text() == "after\n")
|
||||||
|
|
||||||
|
root = Path(resolve(SUBJECT))
|
||||||
|
|
||||||
|
check("the snapshot is removed afterwards", not root.exists() or not (root / "volumes").exists())
|
||||||
|
print("ALL OK", flush=True)
|
||||||
141
tests/e2e/test_e2e_snapshot.py
Normal file
141
tests/e2e/test_e2e_snapshot.py
Normal file
@@ -0,0 +1,141 @@
|
|||||||
|
"""Snapshot capture against real filesystems.
|
||||||
|
|
||||||
|
Loop devices are only available to a privileged container, so each case builds
|
||||||
|
its filesystem inside one and drives snapshot_driver.py there. A filesystem
|
||||||
|
without snapshot support must fail loudly rather than degrade to a live copy,
|
||||||
|
which is the property that makes the mode safe to offer at all.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from .helpers import require_docker, run, unique
|
||||||
|
|
||||||
|
REPO_SRC = Path(__file__).resolve().parents[2] / "src"
|
||||||
|
DRIVER = Path(__file__).resolve().parent / "snapshot_driver.py"
|
||||||
|
IMAGE = "alpine:3.20"
|
||||||
|
PACKAGES = "apk add -q btrfs-progs e2fsprogs zfs python3 util-linux"
|
||||||
|
ATTACH = (
|
||||||
|
"LOOP=$(losetup -f | awk '{print $1}') "
|
||||||
|
'&& { [ -b "$LOOP" ] || mknod "$LOOP" b 7 "${LOOP#/dev/loop}"; }; '
|
||||||
|
'losetup "$LOOP" /img'
|
||||||
|
)
|
||||||
|
LOOP_FS = {
|
||||||
|
"btrfs": "btrfs subvolume create /subject/docker >/dev/null",
|
||||||
|
"ext4": "mkdir -p /subject/docker",
|
||||||
|
}
|
||||||
|
# A container carries no /lib/modules, so modprobe fails even on a loaded module.
|
||||||
|
ZFS_READY = '{ [ -c /dev/zfs ] || modprobe zfs 2>/dev/null; }; [ -c /dev/zfs ]'
|
||||||
|
|
||||||
|
|
||||||
|
def mount_script(fstype: str) -> str:
|
||||||
|
"""Build a filesystem on a loop device and carve out the snapshot subject."""
|
||||||
|
if fstype == "zfs":
|
||||||
|
return (
|
||||||
|
f"{PACKAGES} && {ZFS_READY} && truncate -s 400M /img "
|
||||||
|
"&& zpool create -m none baudolo /img "
|
||||||
|
"&& zfs create -o mountpoint=/subject/docker baudolo/docker"
|
||||||
|
)
|
||||||
|
return (
|
||||||
|
f"{PACKAGES} && truncate -s 400M /img && mkfs.{fstype} -q /img "
|
||||||
|
f'&& mkdir -p /subject && {ATTACH} && mount -t {fstype} "$LOOP" /subject '
|
||||||
|
f"&& {LOOP_FS[fstype]}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def zfs_usable() -> bool:
|
||||||
|
"""Whether this host's kernel can serve zfs to a privileged container."""
|
||||||
|
proc = run(
|
||||||
|
[
|
||||||
|
"docker", "run", "--rm", "--privileged", IMAGE, "sh", "-lc",
|
||||||
|
f"apk add -q zfs >/dev/null 2>&1 && {ZFS_READY}",
|
||||||
|
],
|
||||||
|
capture=True,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
return proc.returncode == 0
|
||||||
|
|
||||||
|
|
||||||
|
def required(fstype: str) -> bool:
|
||||||
|
"""Whether this run must cover ``fstype`` instead of skipping it.
|
||||||
|
|
||||||
|
CI sets E2E_REQUIRE_FILESYSTEMS so a missing kernel module fails the build
|
||||||
|
rather than passing it with a filesystem silently untested.
|
||||||
|
"""
|
||||||
|
demanded = os.environ.get("E2E_REQUIRE_FILESYSTEMS", "")
|
||||||
|
return fstype in demanded.replace(",", " ").split()
|
||||||
|
|
||||||
|
|
||||||
|
def stage() -> Path:
|
||||||
|
"""Copy source and driver under /tmp, the only path the DinD daemon shares."""
|
||||||
|
staged = Path("/tmp") / unique("baudolo-e2e-snapshot")
|
||||||
|
shutil.copytree(REPO_SRC, staged / "src")
|
||||||
|
shutil.copy(DRIVER, staged / "driver.py")
|
||||||
|
return staged
|
||||||
|
|
||||||
|
|
||||||
|
def drive(fstype: str, kind: str, expect: str) -> str:
|
||||||
|
staged = stage()
|
||||||
|
script = (
|
||||||
|
f"set -e; {mount_script(fstype)}; "
|
||||||
|
f"python3 /driver.py {kind} /subject/docker {expect}"
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
proc = run(
|
||||||
|
[
|
||||||
|
"docker", "run", "--rm", "--privileged",
|
||||||
|
"--name", staged.name,
|
||||||
|
"-v", f"{staged / 'src'}:/src:ro",
|
||||||
|
"-v", f"{staged / 'driver.py'}:/driver.py:ro",
|
||||||
|
IMAGE, "sh", "-lc", script,
|
||||||
|
],
|
||||||
|
capture=True,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
shutil.rmtree(staged, ignore_errors=True)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
raise AssertionError(f"{fstype}/{kind} driver failed:\n{proc.stdout}\n{proc.stderr}")
|
||||||
|
return proc.stdout
|
||||||
|
|
||||||
|
|
||||||
|
class TestE2ESnapshot(unittest.TestCase):
|
||||||
|
@classmethod
|
||||||
|
def setUpClass(cls) -> None:
|
||||||
|
require_docker()
|
||||||
|
|
||||||
|
def assert_freezes(self, fstype: str) -> None:
|
||||||
|
output = drive(fstype, fstype, "supported")
|
||||||
|
self.assertIn("PASS the snapshot exposes the volume", output)
|
||||||
|
self.assertIn("PASS a later write does not reach the snapshot", output)
|
||||||
|
self.assertIn("PASS the snapshot is removed afterwards", output)
|
||||||
|
self.assertIn("ALL OK", output)
|
||||||
|
|
||||||
|
def test_btrfs_snapshot_freezes_the_volume(self) -> None:
|
||||||
|
self.assert_freezes("btrfs")
|
||||||
|
|
||||||
|
def test_zfs_snapshot_freezes_the_volume(self) -> None:
|
||||||
|
if not zfs_usable():
|
||||||
|
if required("zfs"):
|
||||||
|
self.fail(
|
||||||
|
"E2E_REQUIRE_FILESYSTEMS demands zfs, but this kernel provides no "
|
||||||
|
"zfs module; load it before running the suite"
|
||||||
|
)
|
||||||
|
self.skipTest("this kernel provides no zfs module, so no pool can be created")
|
||||||
|
self.assert_freezes("zfs")
|
||||||
|
|
||||||
|
def test_ext4_has_no_snapshot_and_says_so(self) -> None:
|
||||||
|
output = drive("ext4", "btrfs", "unsupported")
|
||||||
|
self.assertIn("PASS refused loudly", output)
|
||||||
|
|
||||||
|
def test_an_unknown_kind_is_refused_before_touching_the_filesystem(self) -> None:
|
||||||
|
output = drive("ext4", "lvm", "unsupported")
|
||||||
|
self.assertIn("PASS refused loudly", output)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
103
tests/e2e/test_e2e_snapshot_db.py
Normal file
103
tests/e2e/test_e2e_snapshot_db.py
Normal file
@@ -0,0 +1,103 @@
|
|||||||
|
"""A live database survives being captured from a snapshot.
|
||||||
|
|
||||||
|
The database is written to while the snapshot is taken and keeps writing
|
||||||
|
afterwards, so the copy can only be a point in time - never a clean shutdown.
|
||||||
|
A second server is then started on that copy: it must recover on its own and
|
||||||
|
still hold every row committed before the snapshot.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import shutil
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from .helpers import require_docker, run, unique
|
||||||
|
|
||||||
|
REPO_SRC = Path(__file__).resolve().parents[2] / "src"
|
||||||
|
DRIVER = Path(__file__).resolve().parent / "snapshot_db_driver.py"
|
||||||
|
IMAGE = "alpine:3.20"
|
||||||
|
SOCKET = "/tmp/live.sock"
|
||||||
|
RESTORED_SOCKET = "/tmp/restored.sock"
|
||||||
|
DATADIR = "/subject/docker/volumes/mariadb_data/_data"
|
||||||
|
RESTORED = "/restored"
|
||||||
|
|
||||||
|
SCRIPT = f"""set -e
|
||||||
|
apk add -q btrfs-progs util-linux python3 mariadb mariadb-client rsync
|
||||||
|
truncate -s 900M /img
|
||||||
|
mkfs.btrfs -q /img
|
||||||
|
mkdir -p /subject
|
||||||
|
LOOP=$(losetup -f | awk '{{print $1}}')
|
||||||
|
{{ [ -b "$LOOP" ] || mknod "$LOOP" b 7 "${{LOOP#/dev/loop}}"; }}
|
||||||
|
losetup "$LOOP" /img
|
||||||
|
mount -t btrfs "$LOOP" /subject
|
||||||
|
btrfs subvolume create /subject/docker >/dev/null
|
||||||
|
mkdir -p {DATADIR}
|
||||||
|
|
||||||
|
mariadb-install-db --user=root --datadir={DATADIR} >/dev/null 2>&1
|
||||||
|
mariadbd --user=root --datadir={DATADIR} --socket={SOCKET} --skip-networking &
|
||||||
|
for i in $(seq 1 60); do mariadb-admin --socket={SOCKET} ping >/dev/null 2>&1 && break; sleep 1; done
|
||||||
|
|
||||||
|
mariadb --socket={SOCKET} -e "CREATE DATABASE demo;
|
||||||
|
CREATE TABLE demo.t (id INT PRIMARY KEY, v VARCHAR(32)) ENGINE=InnoDB;
|
||||||
|
INSERT INTO demo.t VALUES (1,'committed'),(2,'committed');"
|
||||||
|
|
||||||
|
mariadb --socket={SOCKET} -e "INSERT INTO demo.t VALUES (3,'committed');"
|
||||||
|
python3 /driver.py
|
||||||
|
mariadb --socket={SOCKET} -e "INSERT INTO demo.t VALUES (4,'after-snapshot');"
|
||||||
|
|
||||||
|
mkdir -p {RESTORED}
|
||||||
|
rsync -a /backups/20260731/mariadb_data/files/ {RESTORED}/
|
||||||
|
mariadbd --user=root --datadir={RESTORED} --socket={RESTORED_SOCKET} --skip-networking &
|
||||||
|
for i in $(seq 1 60); do mariadb-admin --socket={RESTORED_SOCKET} ping >/dev/null 2>&1 && break; sleep 1; done
|
||||||
|
|
||||||
|
echo "RESTORED_ROWS=$(mariadb --socket={RESTORED_SOCKET} -N -B -e 'SELECT COUNT(*) FROM demo.t;')"
|
||||||
|
echo "RESTORED_AFTER=$(mariadb --socket={RESTORED_SOCKET} -N -B -e \\
|
||||||
|
"SELECT COUNT(*) FROM demo.t WHERE v='after-snapshot';")"
|
||||||
|
echo DB_OK
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
class TestE2ESnapshotDatabase(unittest.TestCase):
|
||||||
|
@classmethod
|
||||||
|
def setUpClass(cls) -> None:
|
||||||
|
require_docker()
|
||||||
|
staged = Path("/tmp") / unique("baudolo-e2e-snapshot-db")
|
||||||
|
shutil.copytree(REPO_SRC, staged / "src")
|
||||||
|
shutil.copy(DRIVER, staged / "driver.py")
|
||||||
|
try:
|
||||||
|
proc = run(
|
||||||
|
[
|
||||||
|
"docker", "run", "--rm", "--privileged",
|
||||||
|
"--name", staged.name,
|
||||||
|
"-v", f"{staged / 'src'}:/src:ro",
|
||||||
|
"-v", f"{staged / 'driver.py'}:/driver.py:ro",
|
||||||
|
IMAGE, "sh", "-lc", SCRIPT,
|
||||||
|
],
|
||||||
|
capture=True,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
shutil.rmtree(staged, ignore_errors=True)
|
||||||
|
cls.output = proc.stdout + proc.stderr
|
||||||
|
cls.returncode = proc.returncode
|
||||||
|
|
||||||
|
def test_the_run_completed(self) -> None:
|
||||||
|
self.assertEqual(self.returncode, 0, self.output)
|
||||||
|
self.assertIn("DB_OK", self.output)
|
||||||
|
|
||||||
|
def test_the_backup_came_from_the_snapshot(self) -> None:
|
||||||
|
self.assertIn("SNAPSHOT COPY DONE", self.output)
|
||||||
|
|
||||||
|
def test_the_restored_server_recovered_on_its_own(self) -> None:
|
||||||
|
self.assertIn("RESTORED_ROWS=3", self.output)
|
||||||
|
|
||||||
|
def test_writes_after_the_snapshot_are_absent(self) -> None:
|
||||||
|
self.assertIn("RESTORED_AFTER=0", self.output)
|
||||||
|
|
||||||
|
def test_the_copy_was_an_unclean_state_the_engine_had_to_repair(self) -> None:
|
||||||
|
self.assertRegex(self.output, r"(?i)crash recovery|rolling back|log sequence")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
44
tests/unit/backup/compose_fixture.py
Normal file
44
tests/unit/backup/compose_fixture.py
Normal file
@@ -0,0 +1,44 @@
|
|||||||
|
"""Builds the on-disk compose directories the compose tests discover."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import shutil
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
def touch(p: Path) -> None:
|
||||||
|
p.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
# ".env/env" leaves ".env" behind as a directory, which blocks a later ".env" file.
|
||||||
|
if p.exists() and p.is_dir():
|
||||||
|
shutil.rmtree(p)
|
||||||
|
|
||||||
|
p.write_text("x", encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def setup_compose_dir(
|
||||||
|
tmp_path: Path,
|
||||||
|
name: str = "mailu",
|
||||||
|
*,
|
||||||
|
compose_name: str = "docker-compose.yml",
|
||||||
|
with_override: bool = False,
|
||||||
|
with_ca_override: bool = False,
|
||||||
|
env_layout: str | None = None, # None | ".env" | ".env/env"
|
||||||
|
) -> Path:
|
||||||
|
d = tmp_path / name
|
||||||
|
d.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
touch(d / compose_name)
|
||||||
|
|
||||||
|
if with_override:
|
||||||
|
touch(d / "docker-compose.override.yml")
|
||||||
|
|
||||||
|
if with_ca_override:
|
||||||
|
touch(d / "docker-compose.ca.override.yml")
|
||||||
|
|
||||||
|
if env_layout == ".env":
|
||||||
|
touch(d / ".env")
|
||||||
|
elif env_layout == ".env/env":
|
||||||
|
touch(d / ".env" / "env")
|
||||||
|
|
||||||
|
return d
|
||||||
@@ -7,7 +7,7 @@ from contextlib import redirect_stderr
|
|||||||
import pandas as pd
|
import pandas as pd
|
||||||
|
|
||||||
# Adjust if your package name/import path differs.
|
# Adjust if your package name/import path differs.
|
||||||
from baudolo.backup.app import _load_databases_df
|
from baudolo.backup.dumps import load_databases_df
|
||||||
|
|
||||||
|
|
||||||
EXPECTED_COLUMNS = ["instance", "database", "username", "password"]
|
EXPECTED_COLUMNS = ["instance", "database", "username", "password"]
|
||||||
@@ -20,7 +20,7 @@ class TestLoadDatabasesDf(unittest.TestCase):
|
|||||||
|
|
||||||
buf = io.StringIO()
|
buf = io.StringIO()
|
||||||
with redirect_stderr(buf):
|
with redirect_stderr(buf):
|
||||||
df = _load_databases_df(missing_path)
|
df = load_databases_df(missing_path)
|
||||||
|
|
||||||
stderr = buf.getvalue()
|
stderr = buf.getvalue()
|
||||||
self.assertIn("WARNING:", stderr)
|
self.assertIn("WARNING:", stderr)
|
||||||
@@ -39,7 +39,7 @@ class TestLoadDatabasesDf(unittest.TestCase):
|
|||||||
|
|
||||||
buf = io.StringIO()
|
buf = io.StringIO()
|
||||||
with redirect_stderr(buf):
|
with redirect_stderr(buf):
|
||||||
df = _load_databases_df(empty_path)
|
df = load_databases_df(empty_path)
|
||||||
|
|
||||||
stderr = buf.getvalue()
|
stderr = buf.getvalue()
|
||||||
self.assertIn("WARNING:", stderr)
|
self.assertIn("WARNING:", stderr)
|
||||||
@@ -59,7 +59,7 @@ class TestLoadDatabasesDf(unittest.TestCase):
|
|||||||
|
|
||||||
buf = io.StringIO()
|
buf = io.StringIO()
|
||||||
with redirect_stderr(buf):
|
with redirect_stderr(buf):
|
||||||
df = _load_databases_df(csv_path)
|
df = load_databases_df(csv_path)
|
||||||
|
|
||||||
stderr = buf.getvalue()
|
stderr = buf.getvalue()
|
||||||
self.assertEqual(stderr, "") # no warning expected
|
self.assertEqual(stderr, "") # no warning expected
|
||||||
|
|||||||
79
tests/unit/backup/test_app_snapshot.py
Normal file
79
tests/unit/backup/test_app_snapshot.py
Normal file
@@ -0,0 +1,79 @@
|
|||||||
|
"""Contract of app.main's snapshot branch - the caller that runs in production."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import unittest
|
||||||
|
from unittest import mock
|
||||||
|
|
||||||
|
from baudolo.backup import app
|
||||||
|
from baudolo.backup.snapshot import volume_snapshot
|
||||||
|
|
||||||
|
|
||||||
|
def stubbed_snapshot(kind: str, subject: str, tag: str):
|
||||||
|
return volume_snapshot(kind, subject, tag, run=lambda command: [])
|
||||||
|
|
||||||
|
ARGV = [
|
||||||
|
"baudolo",
|
||||||
|
"--compose-dir",
|
||||||
|
"/compose",
|
||||||
|
"--backups-dir",
|
||||||
|
"/backups",
|
||||||
|
"--snapshot",
|
||||||
|
"btrfs",
|
||||||
|
"--snapshot-subject",
|
||||||
|
"/var/lib/docker",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def drive(*, present: bool) -> list[dict]:
|
||||||
|
calls: list[dict] = []
|
||||||
|
|
||||||
|
def record(versions_dir, volume_name, volume_dir, *, authoritative, source):
|
||||||
|
calls.append(
|
||||||
|
{"volume": volume_name, "authoritative": authoritative, "source": source}
|
||||||
|
)
|
||||||
|
|
||||||
|
with (
|
||||||
|
mock.patch("sys.argv", ARGV),
|
||||||
|
mock.patch.object(app, "get_machine_id", return_value="machine"),
|
||||||
|
mock.patch.object(app, "create_version_directory", return_value="/gen"),
|
||||||
|
mock.patch.object(app, "create_volume_directory", return_value="/gen/vol"),
|
||||||
|
mock.patch.object(app, "load_databases_df", return_value=None),
|
||||||
|
mock.patch.object(app, "docker_volume_names", return_value=["vol"]),
|
||||||
|
mock.patch.object(app, "containers_using_volume", return_value=[]),
|
||||||
|
mock.patch.object(app, "volume_is_fully_ignored", return_value=False),
|
||||||
|
mock.patch.object(app, "backup_dumps_for_volume", return_value=(False, False)),
|
||||||
|
mock.patch.object(
|
||||||
|
app, "get_storage_path", return_value="/var/lib/docker/volumes/vol/_data/"
|
||||||
|
),
|
||||||
|
mock.patch.object(app, "stamp_directory"),
|
||||||
|
mock.patch.object(app, "handle_docker_compose_services"),
|
||||||
|
mock.patch.object(app.os.path, "isdir", return_value=present),
|
||||||
|
mock.patch.object(app, "backup_volume", side_effect=record),
|
||||||
|
mock.patch.object(app, "volume_snapshot", stubbed_snapshot),
|
||||||
|
):
|
||||||
|
app.main()
|
||||||
|
return calls
|
||||||
|
|
||||||
|
|
||||||
|
class TestSnapshotBranch(unittest.TestCase):
|
||||||
|
def test_it_passes_a_path_ending_in_a_separator(self) -> None:
|
||||||
|
source = drive(present=True)[0]["source"]
|
||||||
|
self.assertTrue(source.endswith("/volumes/vol/_data/"), source)
|
||||||
|
self.assertNotIn("/var/lib/docker/volumes", source)
|
||||||
|
|
||||||
|
def test_it_reads_from_the_snapshot_and_not_from_the_live_tree(self) -> None:
|
||||||
|
source = drive(present=True)[0]["source"]
|
||||||
|
self.assertTrue(source.startswith("/var/lib/.baudolo-"), source)
|
||||||
|
|
||||||
|
def test_it_compares_by_content_against_the_previous_generation(self) -> None:
|
||||||
|
self.assertTrue(drive(present=True)[0]["authoritative"])
|
||||||
|
|
||||||
|
def test_a_volume_missing_from_the_snapshot_is_copied_live(self) -> None:
|
||||||
|
call = drive(present=False)[0]
|
||||||
|
self.assertEqual(call["source"], "/var/lib/docker/volumes/vol/_data/")
|
||||||
|
self.assertFalse(call["authoritative"])
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
81
tests/unit/backup/test_cli.py
Normal file
81
tests/unit/backup/test_cli.py
Normal file
@@ -0,0 +1,81 @@
|
|||||||
|
"""Contract of the backup CLI, in particular the snapshot flag pairing."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import unittest
|
||||||
|
from unittest import mock
|
||||||
|
|
||||||
|
from baudolo.backup.cli import parse_args
|
||||||
|
|
||||||
|
REQUIRED = ["--compose-dir", "/compose", "--backups-dir", "/backups"]
|
||||||
|
|
||||||
|
|
||||||
|
def parse(*extra: str):
|
||||||
|
with mock.patch("sys.argv", ["baudolo", *REQUIRED, *extra]):
|
||||||
|
return parse_args()
|
||||||
|
|
||||||
|
|
||||||
|
class TestSnapshotFlags(unittest.TestCase):
|
||||||
|
def test_no_snapshot_by_default(self) -> None:
|
||||||
|
args = parse()
|
||||||
|
self.assertIsNone(args.snapshot)
|
||||||
|
self.assertIsNone(args.snapshot_subject)
|
||||||
|
|
||||||
|
def test_both_flags_together_are_accepted(self) -> None:
|
||||||
|
args = parse("--snapshot", "btrfs", "--snapshot-subject", "/var/lib/docker")
|
||||||
|
self.assertEqual(args.snapshot, "btrfs")
|
||||||
|
self.assertEqual(args.snapshot_subject, "/var/lib/docker")
|
||||||
|
|
||||||
|
def test_the_kind_alone_is_rejected(self) -> None:
|
||||||
|
with self.assertRaises(SystemExit):
|
||||||
|
parse("--snapshot", "btrfs")
|
||||||
|
|
||||||
|
def test_the_subject_alone_is_rejected(self) -> None:
|
||||||
|
with self.assertRaises(SystemExit):
|
||||||
|
parse("--snapshot-subject", "/var/lib/docker")
|
||||||
|
|
||||||
|
def test_an_unsupported_kind_is_rejected(self) -> None:
|
||||||
|
with self.assertRaises(SystemExit):
|
||||||
|
parse("--snapshot", "ext4", "--snapshot-subject", "/var/lib/docker")
|
||||||
|
|
||||||
|
def test_zfs_is_accepted(self) -> None:
|
||||||
|
self.assertEqual(parse("--snapshot", "zfs", "--snapshot-subject", "/d").snapshot, "zfs")
|
||||||
|
|
||||||
|
def test_shutdown_is_rejected_because_nothing_is_stopped(self) -> None:
|
||||||
|
with self.assertRaises(SystemExit):
|
||||||
|
parse("--snapshot", "btrfs", "--snapshot-subject", "/d", "--shutdown")
|
||||||
|
|
||||||
|
def test_shutdown_stays_available_without_a_snapshot(self) -> None:
|
||||||
|
self.assertTrue(parse("--shutdown").shutdown)
|
||||||
|
|
||||||
|
def test_hard_restart_is_rejected_because_nothing_is_stopped(self) -> None:
|
||||||
|
with self.assertRaises(SystemExit):
|
||||||
|
parse(
|
||||||
|
"--snapshot",
|
||||||
|
"btrfs",
|
||||||
|
"--snapshot-subject",
|
||||||
|
"/d",
|
||||||
|
"--hard-restart-projects",
|
||||||
|
"mailu",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_hard_restart_stays_available_without_a_snapshot(self) -> None:
|
||||||
|
self.assertEqual(
|
||||||
|
parse("--hard-restart-projects", "mailu").hard_restart_projects, ["mailu"]
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class TestRequiredFlags(unittest.TestCase):
|
||||||
|
def test_backups_dir_is_required(self) -> None:
|
||||||
|
with mock.patch("sys.argv", ["baudolo", "--compose-dir", "/compose"]):
|
||||||
|
with self.assertRaises(SystemExit):
|
||||||
|
parse_args()
|
||||||
|
|
||||||
|
def test_compose_dir_is_required(self) -> None:
|
||||||
|
with mock.patch("sys.argv", ["baudolo", "--backups-dir", "/backups"]):
|
||||||
|
with self.assertRaises(SystemExit):
|
||||||
|
parse_args()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -1,50 +1,12 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import shutil
|
|
||||||
import tempfile
|
import tempfile
|
||||||
import unittest
|
import unittest
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import List
|
from typing import List
|
||||||
from unittest.mock import patch
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
from .compose_fixture import setup_compose_dir as _setup_compose_dir
|
||||||
def _touch(p: Path) -> None:
|
|
||||||
p.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
# If the path already exists as a directory (e.g. ".env" created by ".env/env"),
|
|
||||||
# remove it so we can create a file with the same name.
|
|
||||||
if p.exists() and p.is_dir():
|
|
||||||
shutil.rmtree(p)
|
|
||||||
|
|
||||||
p.write_text("x", encoding="utf-8")
|
|
||||||
|
|
||||||
|
|
||||||
def _setup_compose_dir(
|
|
||||||
tmp_path: Path,
|
|
||||||
name: str = "mailu",
|
|
||||||
*,
|
|
||||||
compose_name: str = "docker-compose.yml",
|
|
||||||
with_override: bool = False,
|
|
||||||
with_ca_override: bool = False,
|
|
||||||
env_layout: str | None = None, # None | ".env" | ".env/env"
|
|
||||||
) -> Path:
|
|
||||||
d = tmp_path / name
|
|
||||||
d.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
_touch(d / compose_name)
|
|
||||||
|
|
||||||
if with_override:
|
|
||||||
_touch(d / "docker-compose.override.yml")
|
|
||||||
|
|
||||||
if with_ca_override:
|
|
||||||
_touch(d / "docker-compose.ca.override.yml")
|
|
||||||
|
|
||||||
if env_layout == ".env":
|
|
||||||
_touch(d / ".env")
|
|
||||||
elif env_layout == ".env/env":
|
|
||||||
_touch(d / ".env" / "env")
|
|
||||||
|
|
||||||
return d
|
|
||||||
|
|
||||||
|
|
||||||
class TestCompose(unittest.TestCase):
|
class TestCompose(unittest.TestCase):
|
||||||
@@ -249,63 +211,3 @@ class TestCompose(unittest.TestCase):
|
|||||||
),
|
),
|
||||||
],
|
],
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
class HardRestartArgTests(unittest.TestCase):
|
|
||||||
"""The hard-restart list defaults to empty (no compose down/up); callers
|
|
||||||
opt in per dir, e.g. compose hosts pass 'mailu' while swarm hosts, where
|
|
||||||
the dir is a stack whose overlay network collides with compose up, pass
|
|
||||||
nothing."""
|
|
||||||
|
|
||||||
def _parse(self, extra: List[str]):
|
|
||||||
import sys
|
|
||||||
|
|
||||||
from baudolo.backup import cli
|
|
||||||
|
|
||||||
argv = [
|
|
||||||
"baudolo",
|
|
||||||
"--compose-dir",
|
|
||||||
"/tmp",
|
|
||||||
"--backups-dir",
|
|
||||||
"/tmp/backup",
|
|
||||||
"--database-containers",
|
|
||||||
"postgres",
|
|
||||||
"--images-no-stop-required",
|
|
||||||
"redis",
|
|
||||||
*extra,
|
|
||||||
]
|
|
||||||
with patch.object(sys, "argv", argv):
|
|
||||||
return cli.parse_args()
|
|
||||||
|
|
||||||
def test_default_is_empty(self) -> None:
|
|
||||||
args = self._parse([])
|
|
||||||
self.assertEqual(args.hard_restart_projects, [])
|
|
||||||
|
|
||||||
def test_empty_flag_stays_empty(self) -> None:
|
|
||||||
args = self._parse(["--hard-restart-projects"])
|
|
||||||
self.assertEqual(args.hard_restart_projects, [])
|
|
||||||
|
|
||||||
def test_explicit_names_preserved(self) -> None:
|
|
||||||
args = self._parse(["--hard-restart-projects", "mailu", "foo"])
|
|
||||||
self.assertEqual(args.hard_restart_projects, ["mailu", "foo"])
|
|
||||||
|
|
||||||
def test_backups_dir_is_required(self) -> None:
|
|
||||||
import sys
|
|
||||||
|
|
||||||
from baudolo.backup import cli
|
|
||||||
|
|
||||||
argv = [
|
|
||||||
"baudolo",
|
|
||||||
"--compose-dir",
|
|
||||||
"/tmp",
|
|
||||||
"--database-containers",
|
|
||||||
"postgres",
|
|
||||||
"--images-no-stop-required",
|
|
||||||
"redis",
|
|
||||||
]
|
|
||||||
with patch.object(sys, "argv", argv), self.assertRaises(SystemExit):
|
|
||||||
cli.parse_args()
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
unittest.main(verbosity=2)
|
|
||||||
|
|||||||
65
tests/unit/backup/test_compose_hard_restart.py
Normal file
65
tests/unit/backup/test_compose_hard_restart.py
Normal file
@@ -0,0 +1,65 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import unittest
|
||||||
|
from typing import List
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
|
||||||
|
class HardRestartArgTests(unittest.TestCase):
|
||||||
|
"""The hard-restart list defaults to empty (no compose down/up); callers
|
||||||
|
opt in per dir, e.g. compose hosts pass 'mailu' while swarm hosts, where
|
||||||
|
the dir is a stack whose overlay network collides with compose up, pass
|
||||||
|
nothing."""
|
||||||
|
|
||||||
|
def _parse(self, extra: List[str]):
|
||||||
|
import sys
|
||||||
|
|
||||||
|
from baudolo.backup import cli
|
||||||
|
|
||||||
|
argv = [
|
||||||
|
"baudolo",
|
||||||
|
"--compose-dir",
|
||||||
|
"/tmp",
|
||||||
|
"--backups-dir",
|
||||||
|
"/tmp/backup",
|
||||||
|
"--database-containers",
|
||||||
|
"postgres",
|
||||||
|
"--images-no-stop-required",
|
||||||
|
"redis",
|
||||||
|
*extra,
|
||||||
|
]
|
||||||
|
with patch.object(sys, "argv", argv):
|
||||||
|
return cli.parse_args()
|
||||||
|
|
||||||
|
def test_default_is_empty(self) -> None:
|
||||||
|
args = self._parse([])
|
||||||
|
self.assertEqual(args.hard_restart_projects, [])
|
||||||
|
|
||||||
|
def test_empty_flag_stays_empty(self) -> None:
|
||||||
|
args = self._parse(["--hard-restart-projects"])
|
||||||
|
self.assertEqual(args.hard_restart_projects, [])
|
||||||
|
|
||||||
|
def test_explicit_names_preserved(self) -> None:
|
||||||
|
args = self._parse(["--hard-restart-projects", "mailu", "foo"])
|
||||||
|
self.assertEqual(args.hard_restart_projects, ["mailu", "foo"])
|
||||||
|
|
||||||
|
def test_backups_dir_is_required(self) -> None:
|
||||||
|
import sys
|
||||||
|
|
||||||
|
from baudolo.backup import cli
|
||||||
|
|
||||||
|
argv = [
|
||||||
|
"baudolo",
|
||||||
|
"--compose-dir",
|
||||||
|
"/tmp",
|
||||||
|
"--database-containers",
|
||||||
|
"postgres",
|
||||||
|
"--images-no-stop-required",
|
||||||
|
"redis",
|
||||||
|
]
|
||||||
|
with patch.object(sys, "argv", argv), self.assertRaises(SystemExit):
|
||||||
|
cli.parse_args()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main(verbosity=2)
|
||||||
51
tests/unit/backup/test_layout.py
Normal file
51
tests/unit/backup/test_layout.py
Normal file
@@ -0,0 +1,51 @@
|
|||||||
|
"""Contract of where a backup run puts its directories."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest import mock
|
||||||
|
|
||||||
|
from baudolo.backup import layout as mod
|
||||||
|
|
||||||
|
|
||||||
|
class TestVersionDirectory(unittest.TestCase):
|
||||||
|
def test_it_creates_the_generation_directory(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
created = mod.create_version_directory(tmp, "20260731020304")
|
||||||
|
self.assertTrue(Path(created).is_dir())
|
||||||
|
self.assertEqual(Path(created).name, "20260731020304")
|
||||||
|
|
||||||
|
def test_it_is_idempotent(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
first = mod.create_version_directory(tmp, "20260731")
|
||||||
|
second = mod.create_version_directory(tmp, "20260731")
|
||||||
|
self.assertEqual(first, second)
|
||||||
|
|
||||||
|
def test_it_creates_missing_parents(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
nested = str(Path(tmp) / "machine" / "repo")
|
||||||
|
created = mod.create_version_directory(nested, "20260731")
|
||||||
|
self.assertTrue(Path(created).is_dir())
|
||||||
|
|
||||||
|
|
||||||
|
class TestVolumeDirectory(unittest.TestCase):
|
||||||
|
def test_it_nests_the_volume_under_the_generation(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
created = mod.create_volume_directory(tmp, "postgres_data")
|
||||||
|
self.assertEqual(Path(created).parent, Path(tmp))
|
||||||
|
self.assertTrue(Path(created).is_dir())
|
||||||
|
|
||||||
|
|
||||||
|
class TestMachineId(unittest.TestCase):
|
||||||
|
def test_it_takes_the_hash_without_the_filename(self) -> None:
|
||||||
|
digest = "a" * 64
|
||||||
|
with mock.patch.object(
|
||||||
|
mod, "execute_shell_command", return_value=[f"{digest} /etc/machine-id"]
|
||||||
|
):
|
||||||
|
self.assertEqual(mod.get_machine_id(), digest)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
68
tests/unit/backup/test_policy.py
Normal file
68
tests/unit/backup/test_policy.py
Normal file
@@ -0,0 +1,68 @@
|
|||||||
|
"""Contract of the rules deciding what is backed up and what must stop."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import unittest
|
||||||
|
from unittest import mock
|
||||||
|
|
||||||
|
from baudolo.backup import policy as mod
|
||||||
|
|
||||||
|
|
||||||
|
class TestIsImageIgnored(unittest.TestCase):
|
||||||
|
def test_an_empty_whitelist_ignores_nothing(self) -> None:
|
||||||
|
self.assertFalse(mod.is_image_ignored("c1", []))
|
||||||
|
|
||||||
|
def test_a_listed_image_is_ignored(self) -> None:
|
||||||
|
with mock.patch.object(mod, "get_image_info", return_value="alpine:3.20"):
|
||||||
|
self.assertTrue(mod.is_image_ignored("c1", ["alpine:3.20"]))
|
||||||
|
|
||||||
|
def test_matching_is_exact(self) -> None:
|
||||||
|
with mock.patch.object(mod, "get_image_info", return_value="alpine:3.21"):
|
||||||
|
self.assertFalse(mod.is_image_ignored("c1", ["alpine:3.20"]))
|
||||||
|
|
||||||
|
|
||||||
|
class TestVolumeIsFullyIgnored(unittest.TestCase):
|
||||||
|
def test_a_volume_without_containers_is_kept(self) -> None:
|
||||||
|
self.assertFalse(mod.volume_is_fully_ignored([], ["alpine:3.20"]))
|
||||||
|
|
||||||
|
def test_it_needs_every_container_to_be_ignored(self) -> None:
|
||||||
|
with mock.patch.object(mod, "get_image_info", side_effect=["a", "b"]):
|
||||||
|
self.assertFalse(mod.volume_is_fully_ignored(["c1", "c2"], ["a"]))
|
||||||
|
|
||||||
|
def test_all_ignored_skips_the_volume(self) -> None:
|
||||||
|
with mock.patch.object(mod, "get_image_info", side_effect=["a", "a"]):
|
||||||
|
self.assertTrue(mod.volume_is_fully_ignored(["c1", "c2"], ["a"]))
|
||||||
|
|
||||||
|
|
||||||
|
class TestRequiresStop(unittest.TestCase):
|
||||||
|
def test_no_containers_means_no_stop(self) -> None:
|
||||||
|
self.assertFalse(mod.requires_stop([], []))
|
||||||
|
|
||||||
|
def test_a_swarm_task_never_forces_a_stop(self) -> None:
|
||||||
|
with mock.patch.object(mod, "is_swarm_task", return_value=True):
|
||||||
|
self.assertFalse(mod.requires_stop(["c1"], []))
|
||||||
|
|
||||||
|
def test_a_whitelisted_image_does_not_force_a_stop(self) -> None:
|
||||||
|
with (
|
||||||
|
mock.patch.object(mod, "is_swarm_task", return_value=False),
|
||||||
|
mock.patch.object(mod, "get_image_info", return_value="alpine:3.20"),
|
||||||
|
):
|
||||||
|
self.assertFalse(mod.requires_stop(["c1"], ["alpine:3.20"]))
|
||||||
|
|
||||||
|
def test_an_unlisted_image_forces_a_stop(self) -> None:
|
||||||
|
with (
|
||||||
|
mock.patch.object(mod, "is_swarm_task", return_value=False),
|
||||||
|
mock.patch.object(mod, "get_image_info", return_value="postgres:17"),
|
||||||
|
):
|
||||||
|
self.assertTrue(mod.requires_stop(["c1"], ["alpine:3.20"]))
|
||||||
|
|
||||||
|
def test_one_unlisted_container_is_enough(self) -> None:
|
||||||
|
with (
|
||||||
|
mock.patch.object(mod, "is_swarm_task", return_value=False),
|
||||||
|
mock.patch.object(mod, "get_image_info", side_effect=["alpine:3.20", "x"]),
|
||||||
|
):
|
||||||
|
self.assertTrue(mod.requires_stop(["c1", "c2"], ["alpine:3.20"]))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
134
tests/unit/backup/test_snapshot.py
Normal file
134
tests/unit/backup/test_snapshot.py
Normal file
@@ -0,0 +1,134 @@
|
|||||||
|
"""Contract of the filesystem snapshot used to capture volumes atomically."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
from baudolo.backup.shell import BackupException
|
||||||
|
from baudolo.backup.snapshot import SnapshotError, volume_snapshot
|
||||||
|
|
||||||
|
|
||||||
|
class Runner:
|
||||||
|
def __init__(self, replies: dict[str, list[str]] | None = None) -> None:
|
||||||
|
self.calls: list[str] = []
|
||||||
|
self.replies = replies or {}
|
||||||
|
|
||||||
|
def __call__(self, command: str) -> list[str]:
|
||||||
|
self.calls.append(command)
|
||||||
|
for prefix, reply in self.replies.items():
|
||||||
|
if command.startswith(prefix):
|
||||||
|
return reply
|
||||||
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
class TestBtrfs(unittest.TestCase):
|
||||||
|
def test_it_creates_a_read_only_snapshot_beside_the_subject(self) -> None:
|
||||||
|
run = Runner()
|
||||||
|
with volume_snapshot("btrfs", "/var/lib/docker", "20260731", run=run):
|
||||||
|
pass
|
||||||
|
self.assertEqual(
|
||||||
|
run.calls[0],
|
||||||
|
"btrfs subvolume snapshot -r /var/lib/docker /var/lib/.baudolo-20260731",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_it_removes_the_snapshot_afterwards(self) -> None:
|
||||||
|
run = Runner()
|
||||||
|
with volume_snapshot("btrfs", "/var/lib/docker", "20260731", run=run):
|
||||||
|
pass
|
||||||
|
self.assertEqual(run.calls[-1], "btrfs subvolume delete /var/lib/.baudolo-20260731")
|
||||||
|
|
||||||
|
def test_it_maps_a_volume_path_into_the_snapshot(self) -> None:
|
||||||
|
run = Runner()
|
||||||
|
with volume_snapshot("btrfs", "/var/lib/docker", "20260731", run=run) as resolve:
|
||||||
|
self.assertEqual(
|
||||||
|
resolve("/var/lib/docker/volumes/postgres_data/_data"),
|
||||||
|
"/var/lib/.baudolo-20260731/volumes/postgres_data/_data",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_it_keeps_the_trailing_slash_rsync_reads_as_contents(self) -> None:
|
||||||
|
run = Runner()
|
||||||
|
with volume_snapshot("btrfs", "/var/lib/docker", "20260731", run=run) as resolve:
|
||||||
|
self.assertEqual(
|
||||||
|
resolve("/var/lib/docker/volumes/postgres_data/_data/"),
|
||||||
|
"/var/lib/.baudolo-20260731/volumes/postgres_data/_data/",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_it_removes_the_snapshot_even_when_the_body_raises(self) -> None:
|
||||||
|
run = Runner()
|
||||||
|
with self.assertRaises(ZeroDivisionError):
|
||||||
|
with volume_snapshot("btrfs", "/var/lib/docker", "20260731", run=run):
|
||||||
|
raise ZeroDivisionError
|
||||||
|
self.assertTrue(run.calls[-1].startswith("btrfs subvolume delete"))
|
||||||
|
|
||||||
|
|
||||||
|
class TestZfs(unittest.TestCase):
|
||||||
|
def _run(self) -> Runner:
|
||||||
|
return Runner({"zfs list": ["tank/docker"]})
|
||||||
|
|
||||||
|
def test_it_snapshots_the_dataset_mounted_at_the_subject(self) -> None:
|
||||||
|
run = self._run()
|
||||||
|
with volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run):
|
||||||
|
pass
|
||||||
|
self.assertIn("zfs snapshot tank/docker@baudolo-20260731", run.calls)
|
||||||
|
|
||||||
|
def test_it_destroys_the_snapshot_afterwards(self) -> None:
|
||||||
|
run = self._run()
|
||||||
|
with volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run):
|
||||||
|
pass
|
||||||
|
self.assertEqual(run.calls[-1], "zfs destroy tank/docker@baudolo-20260731")
|
||||||
|
|
||||||
|
def test_it_maps_a_volume_path_through_the_dot_zfs_directory(self) -> None:
|
||||||
|
run = self._run()
|
||||||
|
with volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run) as resolve:
|
||||||
|
self.assertEqual(
|
||||||
|
resolve("/var/lib/docker/volumes/postgres_data/_data"),
|
||||||
|
"/var/lib/docker/.zfs/snapshot/baudolo-20260731/volumes/postgres_data/_data",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_an_unmounted_dataset_is_an_error(self) -> None:
|
||||||
|
run = Runner({"zfs list": [""]})
|
||||||
|
with self.assertRaises(SnapshotError):
|
||||||
|
with volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class TestRejections(unittest.TestCase):
|
||||||
|
def test_an_unknown_kind_is_rejected(self) -> None:
|
||||||
|
run = Runner()
|
||||||
|
with self.assertRaises(SnapshotError):
|
||||||
|
with volume_snapshot("ext4", "/var/lib/docker", "20260731", run=run):
|
||||||
|
pass
|
||||||
|
self.assertEqual(run.calls, [])
|
||||||
|
|
||||||
|
def test_a_path_outside_the_subject_is_rejected(self) -> None:
|
||||||
|
run = Runner()
|
||||||
|
with volume_snapshot("btrfs", "/var/lib/docker", "20260731", run=run) as resolve:
|
||||||
|
with self.assertRaises(SnapshotError):
|
||||||
|
resolve("/etc/passwd")
|
||||||
|
|
||||||
|
def test_the_subject_itself_resolves_to_the_snapshot_root(self) -> None:
|
||||||
|
run = Runner()
|
||||||
|
with volume_snapshot("btrfs", "/var/lib/docker", "20260731", run=run) as resolve:
|
||||||
|
self.assertEqual(resolve("/var/lib/docker"), "/var/lib/.baudolo-20260731")
|
||||||
|
|
||||||
|
|
||||||
|
class Busy(Runner):
|
||||||
|
def __call__(self, command: str) -> list[str]:
|
||||||
|
if command.startswith("btrfs subvolume delete"):
|
||||||
|
raise BackupException("target is busy")
|
||||||
|
return super().__call__(command)
|
||||||
|
|
||||||
|
|
||||||
|
class TestRemovalFailure(unittest.TestCase):
|
||||||
|
def test_a_failed_removal_does_not_fail_a_completed_run(self) -> None:
|
||||||
|
with volume_snapshot("btrfs", "/var/lib/docker", "20260731", run=Busy()):
|
||||||
|
pass
|
||||||
|
|
||||||
|
def test_a_failed_removal_does_not_mask_the_body(self) -> None:
|
||||||
|
with self.assertRaises(ZeroDivisionError):
|
||||||
|
with volume_snapshot("btrfs", "/var/lib/docker", "20260731", run=Busy()):
|
||||||
|
raise ZeroDivisionError
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
87
tests/unit/backup/test_volume.py
Normal file
87
tests/unit/backup/test_volume.py
Normal file
@@ -0,0 +1,87 @@
|
|||||||
|
"""Contract of the rsync invocation that copies a volume."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest import mock
|
||||||
|
|
||||||
|
from baudolo.backup import volume as mod
|
||||||
|
|
||||||
|
|
||||||
|
class TestBackupVolume(unittest.TestCase):
|
||||||
|
def copy(self, **kwargs) -> str:
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
defaults = {
|
||||||
|
"versions_dir": tmp,
|
||||||
|
"volume_name": "demo",
|
||||||
|
"volume_dir": str(Path(tmp) / "gen" / "demo"),
|
||||||
|
"authoritative": False,
|
||||||
|
"source": "/var/lib/docker/volumes/demo/_data/",
|
||||||
|
}
|
||||||
|
defaults.update(kwargs)
|
||||||
|
with mock.patch.object(mod, "execute_shell_command") as run:
|
||||||
|
mod.backup_volume(
|
||||||
|
defaults.pop("versions_dir"),
|
||||||
|
defaults.pop("volume_name"),
|
||||||
|
defaults.pop("volume_dir"),
|
||||||
|
**defaults,
|
||||||
|
)
|
||||||
|
return run.call_args[0][0]
|
||||||
|
|
||||||
|
def test_the_quick_check_pass_carries_no_checksum(self) -> None:
|
||||||
|
self.assertNotIn("--checksum", self.copy(authoritative=False))
|
||||||
|
|
||||||
|
def test_the_authoritative_pass_compares_by_content(self) -> None:
|
||||||
|
self.assertIn("--checksum", self.copy(authoritative=True))
|
||||||
|
|
||||||
|
def test_it_reads_from_the_given_source(self) -> None:
|
||||||
|
command = self.copy(source="/snapshot/volumes/demo/_data/")
|
||||||
|
self.assertIn("/snapshot/volumes/demo/_data/", command)
|
||||||
|
|
||||||
|
def test_it_always_deletes_what_the_source_no_longer_has(self) -> None:
|
||||||
|
self.assertIn("--delete", self.copy())
|
||||||
|
|
||||||
|
def test_it_creates_the_destination(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
dest = Path(tmp) / "gen" / "demo"
|
||||||
|
with mock.patch.object(mod, "execute_shell_command"):
|
||||||
|
mod.backup_volume(
|
||||||
|
tmp, "demo", str(dest), authoritative=False, source="/src/"
|
||||||
|
)
|
||||||
|
self.assertTrue((dest / "files").is_dir())
|
||||||
|
|
||||||
|
def test_source_is_required(self) -> None:
|
||||||
|
with self.assertRaises(TypeError):
|
||||||
|
mod.backup_volume("/v", "demo", "/d", authoritative=False)
|
||||||
|
|
||||||
|
def test_authoritative_is_required(self) -> None:
|
||||||
|
with self.assertRaises(TypeError):
|
||||||
|
mod.backup_volume("/v", "demo", "/d", source="/src/")
|
||||||
|
|
||||||
|
|
||||||
|
class TestLastBackupDir(unittest.TestCase):
|
||||||
|
def test_it_ignores_the_generation_being_written(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
current = Path(tmp) / "20260101" / "demo" / "files"
|
||||||
|
current.mkdir(parents=True)
|
||||||
|
found = mod.get_last_backup_dir(tmp, "demo", str(current) + "/")
|
||||||
|
self.assertIsNone(found)
|
||||||
|
|
||||||
|
def test_it_finds_the_previous_generation(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
older = Path(tmp) / "20260101" / "demo" / "files"
|
||||||
|
older.mkdir(parents=True)
|
||||||
|
current = Path(tmp) / "20260102" / "demo" / "files"
|
||||||
|
current.mkdir(parents=True)
|
||||||
|
found = mod.get_last_backup_dir(tmp, "demo", str(current) + "/")
|
||||||
|
self.assertEqual(found, str(older) + "/")
|
||||||
|
|
||||||
|
def test_a_first_run_has_no_predecessor(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
self.assertIsNone(mod.get_last_backup_dir(tmp, "demo", f"{tmp}/x/"))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -1,12 +1,12 @@
|
|||||||
import unittest
|
import unittest
|
||||||
from unittest.mock import patch
|
from unittest.mock import patch
|
||||||
|
|
||||||
from baudolo.backup.app import requires_stop
|
from baudolo.backup.policy import requires_stop
|
||||||
|
|
||||||
|
|
||||||
@patch("baudolo.backup.app.is_swarm_task", return_value=False)
|
@patch("baudolo.backup.policy.is_swarm_task", return_value=False)
|
||||||
class TestRequiresStop(unittest.TestCase):
|
class TestRequiresStop(unittest.TestCase):
|
||||||
@patch("baudolo.backup.app.get_image_info")
|
@patch("baudolo.backup.policy.get_image_info")
|
||||||
def test_requires_stop_false_when_all_images_are_whitelisted(
|
def test_requires_stop_false_when_all_images_are_whitelisted(
|
||||||
self, mock_get_image_info, _mock_is_swarm_task
|
self, mock_get_image_info, _mock_is_swarm_task
|
||||||
):
|
):
|
||||||
@@ -18,7 +18,7 @@ class TestRequiresStop(unittest.TestCase):
|
|||||||
whitelist = ["repo/mastodon:v4", "repo/wordpress:latest"]
|
whitelist = ["repo/mastodon:v4", "repo/wordpress:latest"]
|
||||||
self.assertFalse(requires_stop(containers, whitelist))
|
self.assertFalse(requires_stop(containers, whitelist))
|
||||||
|
|
||||||
@patch("baudolo.backup.app.get_image_info")
|
@patch("baudolo.backup.policy.get_image_info")
|
||||||
def test_requires_stop_true_when_any_image_is_not_whitelisted(
|
def test_requires_stop_true_when_any_image_is_not_whitelisted(
|
||||||
self, mock_get_image_info, _mock_is_swarm_task
|
self, mock_get_image_info, _mock_is_swarm_task
|
||||||
):
|
):
|
||||||
@@ -30,7 +30,7 @@ class TestRequiresStop(unittest.TestCase):
|
|||||||
whitelist = ["repo/mastodon:v4", "repo/wordpress:latest"]
|
whitelist = ["repo/mastodon:v4", "repo/wordpress:latest"]
|
||||||
self.assertTrue(requires_stop(containers, whitelist))
|
self.assertTrue(requires_stop(containers, whitelist))
|
||||||
|
|
||||||
@patch("baudolo.backup.app.get_image_info")
|
@patch("baudolo.backup.policy.get_image_info")
|
||||||
def test_requires_stop_true_on_substring_only_match(
|
def test_requires_stop_true_on_substring_only_match(
|
||||||
self, mock_get_image_info, _mock_is_swarm_task
|
self, mock_get_image_info, _mock_is_swarm_task
|
||||||
):
|
):
|
||||||
@@ -38,7 +38,7 @@ class TestRequiresStop(unittest.TestCase):
|
|||||||
self.assertTrue(requires_stop(["c1"], ["mastodon"]))
|
self.assertTrue(requires_stop(["c1"], ["mastodon"]))
|
||||||
self.assertTrue(requires_stop(["c1"], ["repo/mastodon:v4"]))
|
self.assertTrue(requires_stop(["c1"], ["repo/mastodon:v4"]))
|
||||||
|
|
||||||
@patch("baudolo.backup.app.get_image_info")
|
@patch("baudolo.backup.policy.get_image_info")
|
||||||
def test_requires_stop_true_when_whitelist_empty(
|
def test_requires_stop_true_when_whitelist_empty(
|
||||||
self, mock_get_image_info, _mock_is_swarm_task
|
self, mock_get_image_info, _mock_is_swarm_task
|
||||||
):
|
):
|
||||||
@@ -47,8 +47,8 @@ class TestRequiresStop(unittest.TestCase):
|
|||||||
|
|
||||||
|
|
||||||
class TestRequiresStopSwarm(unittest.TestCase):
|
class TestRequiresStopSwarm(unittest.TestCase):
|
||||||
@patch("baudolo.backup.app.get_image_info")
|
@patch("baudolo.backup.policy.get_image_info")
|
||||||
@patch("baudolo.backup.app.is_swarm_task", return_value=True)
|
@patch("baudolo.backup.policy.is_swarm_task", return_value=True)
|
||||||
def test_swarm_tasks_never_require_stop(
|
def test_swarm_tasks_never_require_stop(
|
||||||
self, _mock_is_swarm_task, mock_get_image_info
|
self, _mock_is_swarm_task, mock_get_image_info
|
||||||
):
|
):
|
||||||
|
|||||||
Reference in New Issue
Block a user