Release version 7.0.0

Two shapes fell through the inline regex, which knew `database`, `db` and
`postgres` only. A container named exactly after its engine - what a compose
file writes as `container_name: postgres` - carries no separator before the
token, so it resolved to nothing and 6.0.0 stopped dumping it without saying
so. And a swarm task of a central MariaDB reads `mariadb_mariadb.1.<id>`,
where `_mariadb` was no token at all, so that database has never been dumped
under swarm at all.

ENGINE_NAMES states the set once and serves both readings: carried as a
suffix it makes the rest the instance, being one outright makes the container
its own instance.

backup_mariadb_or_postgres stops calling an application container a database.
container_engine recognises an engine by its client tools, which an
application image often ships, so refusing the dump alone would have recorded
the volume as `database: true, dumped: false` - the exact shape a restore
drill reads as a database that was missed. Without an instance there is no
database to record.

BREAKING CHANGE: `mariadb` and `mysql` join the suffix tokens, so a container
named `<app>-mariadb` resolves to the instance `<app>` rather than to its own
name. A databases.csv keyed on the full container name has to move to the
application name, or name the container in --database-containers.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

.claude/.gitignore

@@ -0,0 +1,3 @@
+*
+!.gitignore
+!settings.json

.claude/settings.json

@@ -0,0 +1,11 @@
+{
+  "permissions": {
+    "ask": [
+      "Bash(git commit*)",
+      "Edit(CHANGELOG.md)",
+      "Write(CHANGELOG.md)",
+      "Edit(pyproject.toml)",
+      "Write(pyproject.toml)"
+    ]
+  }
+}

.dockerignore

@@ -0,0 +1,13 @@
+.git
+.github
+__pycache__
+**/__pycache__
+*.egg-info
+**/*.egg-info
+artifacts/
+dist/
+build/
+.venv
+.ruff_cache
+.pytest_cache
+.mcp.json

.github/dependabot.yml

@@ -0,0 +1,21 @@
+---
+version: 2
+updates:
+  - package-ecosystem: github-actions
+    directory: /
+    schedule:
+      interval: weekly
+    groups:
+      actions:
+        patterns:
+          - "*"
+
+  - package-ecosystem: docker
+    directory: /
+    schedule:
+      interval: weekly
+
+  - package-ecosystem: pip
+    directory: /
+    schedule:
+      interval: weekly

.github/workflows/ci.yml

@@ -22,20 +22,29 @@ jobs:
 
     steps:
       - name: Checkout
-        uses: actions/checkout@v4
+        uses: actions/checkout@v7
 
       - name: Show docker info
         run: |
           docker version
           docker info
 
+      - name: Provide zfs so the snapshot suite covers every filesystem
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y --no-install-recommends zfsutils-linux
+          sudo modprobe zfs
+          zpool version
+
       - name: Run all tests via Makefile
+        env:
+          E2E_REQUIRE_FILESYSTEMS: "btrfs ext4 zfs"
         run: |
           make test
 
       - name: Upload E2E artifacts (always)
         if: always()
-        uses: actions/upload-artifact@v4
+        uses: actions/upload-artifact@v7
         with:
           name: e2e-artifacts
           path: artifacts
@@ -49,7 +58,7 @@ jobs:
 
     steps:
       - name: Checkout (full history for tags)
-        uses: actions/checkout@v4
+        uses: actions/checkout@v7
         with:
           fetch-depth: 0
 
@@ -67,7 +76,7 @@ jobs:
           git push -f origin stable
 
       - name: Login to GHCR
-        uses: docker/login-action@v3
+        uses: docker/login-action@v4
         with:
           registry: ${{ env.REGISTRY }}
           username: ${{ github.actor }}

.github/workflows/dependabot-auto-merge.yml

@@ -0,0 +1,29 @@
+---
+name: Dependabot auto-merge
+
+on: pull_request
+
+permissions:
+  contents: write
+  pull-requests: write
+
+jobs:
+  auto-merge:
+    # Enable auto-merge for minor/patch dependency bumps; majors stay manual.
+    # The merge only happens once every required status check (make test) is
+    # green, so the CI stays the gate.
+    if: github.actor == 'dependabot[bot]'
+    runs-on: ubuntu-latest
+    steps:
+      - name: Fetch dependency metadata
+        id: metadata
+        uses: dependabot/fetch-metadata@v3
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Enable auto-merge (minor + patch)
+        if: steps.metadata.outputs.update-type != 'version-update:semver-major'
+        run: gh pr merge --auto --squash "$PR_URL"
+        env:
+          PR_URL: ${{ github.event.pull_request.html_url }}
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.gitignore

@@ -1,3 +1,6 @@
 __pycache__
 artifacts/
-*.egg-info
+*.egg-info
+dist/
+build/
+.mcp.json

CHANGELOG.md

@@ -1,3 +1,564 @@
+# Changelog
+
+## [7.0.0] - 2026-08-18
+
+Breaking:
+- Backup: *mariadb* and *mysql* join the suffix tokens, so a container named
+  *<app>-mariadb* or *<app>-mysql* now resolves to the instance *<app>* instead
+  of to its own name. A *databases.csv* keyed on the full container name has to
+  move to the application name, or name the container in
+  *--database-containers*. This narrows what 6.0.0 broke rather than widening
+  it: a container named exactly *postgres*, *mariadb*, *mysql*, *db* or
+  *database* resolves again without any declaration, which is the shape a
+  compose file writes as *container_name: postgres* and the most common
+  configuration there is.
+
+Fixed:
+- Backup: a container named exactly after its engine is dumped again. The
+  suffix match needs a hyphen or underscore in front of the token, which a bare
+  name does not carry, so 6.0.0 resolved *container_name: postgres* to nothing
+  and stopped dumping it without saying so. *ENGINE_NAMES* now states the set
+  once and serves both readings — carried as a suffix it makes the rest the
+  instance, being one outright makes the container its own instance.
+- Backup: a central MariaDB under swarm is dumped for the first time. Swarm
+  names its task *mariadb_mariadb.1.<id>*, which matches neither the static
+  *mariadb* passed through *--database-containers* nor any token the suffix
+  match knew, since *_mariadb* is not *_db*. The database was silently absent
+  from every swarm backup this tool has ever written, before 6.0.0 as well.
+- Backup: an application container is no longer recorded as a database.
+  *container_engine* recognises an engine by its client tools, which an
+  application image frequently ships, so refusing its dump alone would have
+  written the volume to the manifest as *database: true, dumped: false* — the
+  exact shape a restore drill reads as a database that was missed. Without an
+  instance there is no database to record, and the volume is a file backup like
+  any other.
+
+## [6.0.0] - 2026-08-18
+
+Breaking:
+- Backup: a database container whose name carries no *database*, *db* or
+  *postgres* token — preceded by a hyphen or underscore — must now be named in
+  *--database-containers*. Without that declaration its *databases.csv* rows no
+  longer match and no dump is written, silently, because nothing fails. The
+  shape this hits hardest is a container named exactly *postgres* or *mariadb*:
+  the token needs a separator in front of it, which a bare name does not have.
+  *app-database*, *app_database.1.<task>* from swarm and *app-postgres-1* are
+  unaffected, as is any container already declared.
+
+Fixed:
+- Backup: *docker exec* now forwards *PGPASSWORD* into the container.
+  *execute_to_file* set the variable on baudolo's own process, but nothing
+  carried it across the container boundary, so an engine whose *pg_hba* demands
+  a password on TCP loopback refused every dump — which is every dedicated
+  Postgres instance on a real host. The name travels as a bare *-e NAME* so
+  docker copies the value out of this process's environment; spelling
+  *-e NAME=value* instead would publish the secret in the host's process list.
+- Backup: *get_instance* no longer claims an instance it never derived. It
+  returned the container name unchanged when that name carried no database
+  token, so an application container answered the same *databases.csv* row as
+  its own dedicated engine. Application images frequently ship the engine's
+  client tools, so the dump command started and wrote a file that looked like a
+  backup and held none of the data: measured against Discourse, 1,680 bytes
+  from the application where the engine produced 10,469,439. The regex stays a
+  normaliser — *<app>-database* from compose and *<app>_database.1.<task>* from
+  swarm still resolve to one instance. Only the fallthrough changed.
+
+New:
+- Tests: *get_instance* has unit coverage for the first time. Eleven cases pin
+  the container names that compose, swarm and explicitly-named engines produce,
+  so a future change to the regex has to state which shape it gives up.
+- Tests: two e2e modules cover shapes the suite structurally could not see.
+  Every fixture passed its container in *--database-containers*, which left the
+  regex branch — the only one a dedicated database ever takes — dead code under
+  test, and no scenario made a password mandatory, because stock
+  *postgres:alpine* grants trust on loopback.
+  *test_e2e_postgres_password_required* starts an engine with
+  *--auth-host=scram-sha-256* and carries a negative control asserting the
+  server refuses an unauthenticated dump; without it the module would pass
+  whether or not the password is forwarded at all.
+  *test_e2e_app_container_ships_client_tools* places an application container
+  beside its engine with neither declared, and requires the engine dumped, the
+  application volume copied as files, and no dump written from the application.
+
+## [5.0.0] - 2026-08-18
+
+**[5.0.0] - 2026-08-18**
+
+Breaking:
+- Library: *BackupException* is now *BackupError* and *VersionMismatch* is now
+  *VersionMismatchError*. Both names violated the convention that an exception
+  class ends in *Error*; the first is imported by six modules, so the rename is
+  atomic across the package.
+- Library: *backup_dumps_for_volume* and *backup_mariadb_or_postgres* return a
+  *VolumeOutcome* instead of a *(bool, bool)* tuple. The pair could not carry
+  the detected engine, which the caller needs for the manifest.
+
+New:
+- Backup: every generation carries a *manifest.json* stating its layout and,
+  per volume, *database* (it held one), *dumped* (a dump was produced) and
+  *engine* (which one was detected). A volume with *database* and no *dumped*
+  was copied as raw engine files — under *--only-sql* that fallback is the
+  documented behaviour, and until now nothing in the finished tree said it had
+  happened. Restoring such a volume replays engine files instead of a dump.
+- Library: *baudolo.generation* states the generation layout once — *files*,
+  *sql*, the dump suffixes, the manifest name. *BackupPaths*, the dump writer
+  and the volume copier stop spelling them out separately. The module is
+  import-free on purpose, so a consumer can read a manifest with nothing but
+  *json* on a host where this package is not installed.
+
+Changed:
+- Build: the test targets no longer depend on *clean*. *clean* is
+  *git clean -fdX .*, so running the unit tests deleted every git-ignored file
+  in the working tree. It was compensating for a missing *.dockerignore*, which
+  now keeps *__pycache__*, egg-info and build output out of the image context
+  where that belongs. *clean* remains available as its own target.
+- Lint: a ruff configuration is declared. The package ran on ruff's defaults
+  while its consumer held itself to a far wider selection; measured against
+  that selection the tree had 224 findings and now has none. Includes a full
+  *os.path* to *pathlib* migration, with two deliberate exceptions: *abspath*
+  stays where *Path.resolve()* would follow symlinks and let a symlinked volume
+  test as inside the snapshot subject, and the rsync trailing separator is kept
+  explicit where *Path* would drop it.
+
+## [4.0.0] - 2026-08-17
+
+Breaking:
+- CLI: *--repo-name* and *--databases-csv* are required. One default was the
+  literal *backup-docker-to-local* while its help promised the git folder name;
+  the other pointed into the installed package, so a forgotten flag ran the
+  whole backup silently without a single dump.
+- CLI: *--dump-only-sql* is now *--only-sql*; the old spelling exits 2.
+- CLI: *--everything* is withdrawn. Its only real effect was to ignore
+  *--images-no-stop-required*, which leaving that list empty already does.
+- Library: the runner injected into *volume_snapshot* receives an argv list
+  instead of a command string.
+
+New:
+- Backup: *--only-files* — no dumps at all, every volume as files, for hosts
+  that hold no database credentials. Needs no *--databases-csv*; mutually
+  exclusive with *--only-sql*.
+- Backup: the engine is detected by executing the dump tool in the container
+  (*pg_dumpall*, *mariadb-dump*, *mysqldump*), not by reading the image name.
+  A dedicated Postgres tagged *<app>-database* is finally dumped; an image
+  merely named like an engine no longer kills the run with exit 127. Probed
+  once per image ID, and an image shipping only *mysqldump* is dumped with it.
+- Library: *baudolo.databases* states the databases.csv contract once —
+  columns, delimiter, cluster marker, validator, *read_rows()* — for the seed,
+  the backup, and external consumers.
+
+Changed:
+- Backup: every command is an argv list; *shell=True* is gone. A database name
+  is validated on read as strictly as the seed writes it, *PGPASSWORD* travels
+  in the child's environment instead of the command string, and a failing dump
+  deletes its partial file instead of leaving it behind.
+
+## [3.6.1] - 2026-08-17
+
+- Restore: *--empty* on a cluster dump is a catalog-wide sweep — it drops every
+  non-template database and every non-pg_ role of the instance. On a dedicated
+  instance that is right, because the dump recreates all of it; on a shared one
+  it destroys databases the dump does not carry, with nothing to restore them
+  from. No test had ever executed that sweep: the e2e dropped the cluster by
+  hand first and left the pre-clean with zero rows to generate.
+- Restore: the instance is checked instead of the sweep being narrowed.
+  *--empty* refuses when the instance holds a database the dump does not carry,
+  names it, and touches nothing. Narrowing the sweep is the obvious fix and is
+  worse — a surviving database that owns or merely grants to one of the dump's
+  roles pins it in *pg_shdepend*, *DROP OWNED BY* reaches only the control
+  database the pre-clean is connected to, so *DROP ROLE* fails after the dump's
+  own databases are already gone and the replay never starts.
+- Restore: the dump's inventory is read with a real identifier parser. A quoted
+  name may hold spaces, and psql options precede the target of a *\connect*
+  line, so a character class that stops at whitespace read *"odd name"* as
+  *odd* and *-reuse-previous=on* as a database.
+- Tests: the cluster e2e no longer empties the instance itself, so *--empty*
+  has to do it and the replay has to put it back. A second pass adds a foreign
+  database and requires the refusal to leave both it and the restored data
+  untouched.
+
+## [3.6.0] - 2026-08-17
+
+- Restore: *--empty* drops the schema in one session and replays in the next,
+  with no rollback across the two, so a dump the engine could not parse left an
+  emptied database behind. The dump's header is now checked against the running
+  engine before anything is dropped, and a newer dump is refused.
+  Forward across a major version stays allowed; *--no-version-check* is the way out.
+- Restore: a volume with driver options — NFS, a bind device, tmpfs — keeps the
+  usual *_data* path, but docker mounts its real storage over it only while a
+  container holds it. Restoring meanwhile landed under the mount, stayed hidden
+  there, and rsync reported success. That volume is now refused until something
+  mounts it.
+- Backup: the same volume sits in a snapshot as an empty directory, so it was
+  copied empty and the generation stamped complete. Capture is decided per volume
+  now — an uncaptured one is copied live, the rest keep their snapshot. A single
+  NFS volume no longer costs the whole host its consistent backup.
+
+## [3.5.0] - 2026-08-17
+
+- Restore: a *database = '*'* row makes the backup write
+  *<instance>.cluster.backup.sql* via *pg_dumpall*, and nothing could read it
+  back — the CLI knew *files*, *postgres* and *mariadb*, so that dump was
+  stored and unrestorable. *baudolo-restore cluster* replays it against the
+  control database, deliberately without *--single-transaction* because
+  CREATE DATABASE is forbidden inside a transaction block, and filters out the
+  CREATE ROLE of the connecting role, which the pre-clean cannot drop while it
+  holds the session. *--empty* drops the cluster's databases first, then
+  releases what its roles still own, then the roles themselves.
+
+- Lint: ruff was never wired into the repository — no target, no CI step, no
+  pin — and reported 45 findings across sources and tests. *make ruff* and
+  *make lint* now run it over every file, *make test* gates on a clean run as a
+  fourth parallel spur, and the linter is pinned in a *lint* extra because a
+  minor bump changes which rules fire.
+
+## [3.4.3] - 2026-08-16
+
+- Backup: *create_version_directory* carried *exist_ok=True*, so a run starting
+  in the same wall-clock second as its predecessor claimed that predecessor's
+  generation. Generation names carry seconds, and a host with little to copy
+  finishes inside one — rsync *--delete* then overwrote a finished generation,
+  and only afterwards did *create_stamp_file* refuse the already-stamped
+  directory and exit 2. The guard reported the damage instead of preventing it.
+- Backup: the generation directory is claimed exclusively. Claiming it is the
+  first filesystem action of a run, so a collision aborts before the first
+  write and names the second it collided on.
+- Tests: the idempotence test asserted the reuse and gave way to one that
+  requires the refusal.
+
+## [3.4.2] - 2026-08-15
+
+- Backup: *has_image* matched the raw *.Config.Image*, so the registry host and
+  the tag decided the dump tool. On a swarm node named after the app under test,
+  *svc-db-mariadb-swarm-mgr-01:5000/postgres_custom* read as MariaDB and
+  *mariadb-dump* ran inside a Postgres container: exit 127, and the
+  *BackupException* took the backup unit with it.
+- Backup: *image_name* strips digest, tag and registry host, so the engine rests
+  on the repository path alone — the exact-matching intent of 3.0.0 applied to
+  the one place that change did not reach.
+- Tests: both false-positive directions on *has_image*, plus an e2e that
+  reproduces the shape with a *docker tag* and asserts a real *pg_dump* lands.
+
+## [3.4.1] - 2026-08-05
+
+- Backup: each volume is copied twice into the same destination — once hot,
+  once cold after the container is stopped — and rsync ran with *-b*, so
+  *--delete* renamed rather than removed a file the source had dropped between
+  the passes. Stopping a container is what makes the source drop files: a
+  graceful shutdown flushes and the format rolls its commit point. The
+  superseded file survived as *name~* beside the real one and was restored into
+  live data.
+- Backup: for an opaque payload that is stale bytes nobody reads; for a format
+  that enumerates its own directory it is corruption. Lucene resolves the
+  current commit by parsing every file starting with *segments* as a radix-36
+  generation, so a restored *segments_3~* leaves the shard store unreadable and
+  the primary at *NO_VALID_SHARD_COPY*. With *.security-7* unallocatable the
+  reserved *elastic* user has no password hash, every probe answers 401 and the
+  container never turns healthy.
+- Backup: *--link-dest* already provides the incrementals and nothing reads the
+  twins — the restore path is an unfiltered *rsync -avv --delete* into the live
+  volume. Dropping *-b* leaves the predecessor generation byte-identical, keeps
+  the hardlinks intact and makes generations smaller, never larger.
+- Tests: the absence of *--backup* is asserted on the rsync invocation.
+
+## [3.4.0] - 2026-08-02
+
+- Backup: *-a* implies *-D*, so a generation was written with
+  *--devices --specials* and rsync recreated every unix socket and fifo found in
+  a volume. Where the backup root is an nfs-ganesha export, ganesha accepts the
+  socket on write but cannot serve it back, and the remote pull's sender then
+  fails with *readdir* / *readlink_stat* "Invalid argument (22)" and exits 23 —
+  deterministically, for every retry. *--no-D* keeps them out of the generation.
+- Backup: nothing restorable is lost. Sockets and fifos are recreated by the
+  daemons that own them, and the postfix queue itself — *incoming*, *active*,
+  *deferred*, *hold*, *maildrop* — is unaffected, so accepted-but-undelivered
+  mail stays in the backup. Device nodes go too; the only volume that could hold
+  them is a nested docker data root, which does not belong in a backup anyway.
+- Tests: the flag is asserted on the rsync invocation.
+
+## [3.3.0] - 2026-08-02
+
+- Backup: *--volumes-no-backup-required* excludes a volume by name.
+  *--images-no-backup-required* resolves through *volume_is_fully_ignored*,
+  which skips a volume only when every container using it is ignored — a
+  container holding a derived tree beside state that must be kept cannot
+  express the exclusion at all. A docker-in-docker data root is exactly that
+  shape, and excluding by image would drop all three of its volumes.
+- Backup: the name check runs before *containers_using_volume*, so an excluded
+  volume costs no docker inspection and the decision does not depend on which
+  containers exist when the run starts.
+- Tests: two volumes off one container, asserting the sibling survives — the
+  property the image lever cannot provide — as unit and end-to-end.
+
+## [3.2.2] - 2026-07-31
+
+- Backup: the btrfs snapshot is carved inside its subject, as
+  *<data root>/.baudolo-<tag>*, not beside it. The kernel refuses a snapshot
+  whose destination is on another filesystem, which is exactly what the parent
+  directory is when the data root is a mountpoint of its own — a dedicated disk
+  mounted onto */var/lib/docker* failed every run with EXDEV. Placing it inside
+  makes source and destination the same filesystem by construction, and aligns
+  btrfs with the zfs path, which already resolves its snapshot inside the
+  subject at *<subject>/.zfs/snapshot/<tag>*. A leftover from an interrupted run
+  appears in the next snapshot as an empty directory rather than recursing,
+  since btrfs does not include nested subvolumes.
+
+## [3.2.1] - 2026-07-31
+
+- Backup: the snapshot resolver keeps the trailing separator *get_storage_path*
+  puts on a volume path — *os.path.abspath* stripped it. rsync reads *dir* as
+  "copy the directory" where *dir/* means "copy its contents", so every snapshot
+  generation landed at *<volume>/files/_data/...* while the live path lands at
+  *<volume>/files/...*. Restores read the live layout, and *--link-dest* had
+  nothing to match against the previous generation.
+- Backup: snapshot teardown no longer fails a completed run. A busy
+  *btrfs subvolume delete* raised out of the *finally*, skipping the generation
+  stamp and the compose handling on a run whose data was already copied, and
+  masking whatever the body had raised. The leftover is reported instead.
+- Backup: a volume created after the snapshot was taken is copied live with a
+  warning instead of aborting the run. Nothing is stopped in snapshot mode, so
+  the host keeps creating volumes for the whole copy.
+- Backup: the snapshot pass compares by content (*--checksum*) again. 3.2.0
+  dropped it because a snapshot source cannot move, which is true, but the
+  comparison that matters is against *--link-dest*: a file that changed while
+  keeping its size and whole-second mtime was hard-linked stale out of the
+  previous generation, and the single pass had no authoritative pass to repair
+  it. Still one pass where the live path takes two.
+- Backup: *--hard-restart-projects* is refused alongside *--snapshot*, like
+  *--shutdown* already is. It exists for stacks whose database cannot be backed
+  up hot, which is what a snapshot removes.
+- Tests: the trailing separator, both teardown behaviours, the new refusal, and
+  *app.main* driving the snapshot branch — the caller that runs in production,
+  which no test had exercised, which is why the layout defect shipped.
+
+## [3.2.0] - 2026-07-31
+
+- Backup: *--snapshot {btrfs,zfs}* with *--snapshot-subject* captures every
+  volume from one atomic filesystem snapshot. The subject — the btrfs subvolume
+  or zfs dataset holding the docker volumes, e.g. */var/lib/docker* — is frozen
+  once per run, so a generation shares a single point in time and no container
+  is stopped. Restoring such a copy is an ordinary crash recovery, which every
+  supported engine performs at startup. This is the mode 3.1.4 pointed to for
+  volumes where two rsync passes over a live tree stop being affordable.
+- Backup: snapshot passes copy once and drop *--checksum*. Verification exists
+  because a hot pass writes its destination from a moving source; a snapshot
+  source cannot move, so size and mtime cannot race and the second full read is
+  pure cost.
+- Backup: an unsupported filesystem or unknown kind fails with *SnapshotError*.
+  The kind is stated, not probed — an inconclusive probe would fall back to a
+  live copy and hand out the torn backup the mode prevents. *--shutdown* is
+  rejected alongside *--snapshot* rather than ignored, since nothing is stopped.
+- Refactor: *backup/app.py* splits into *layout.py*, *policy.py* and *dumps.py*
+  along the lines it already had; 276 lines down to 124.
+- Tests: unit coverage for snapshot, layout, policy, volume and CLI validation.
+  E2E cases drive real btrfs, zfs and ext4 on loop devices, one proving the
+  loud refusal; another writes a MariaDB across the snapshot and requires the
+  restored server to recover on its own with every committed row and none of
+  the later ones. CI installs zfs and sets *E2E_REQUIRE_FILESYSTEMS*, so a
+  missing kernel module fails the build instead of skipping a filesystem.
+
+## [3.1.4] - 2026-07-31
+
+- Backup: the post-stop volume pass now compares by content (*--checksum*), so
+  it can no longer skip a file the hot pass had copied from a live source. Each
+  volume is rsynced twice into the same destination — once with the container
+  running, once after it is stopped — and the second pass used rsync's quick
+  check (size plus whole-second mtime). A pre-allocated 16 MiB WAL segment never
+  changes size, so when its last pre-stop write and postgres' shutdown
+  checkpoint fell in the same whole second, the cold pass skipped it while still
+  replacing *global/pg_control*, whose previous write was seconds earlier. The
+  generation then paired a post-shutdown *pg_control* with a WAL segment still
+  zeroed at the recorded checkpoint LSN, and restoring it crash-looped postgres
+  with "invalid record length … expected at least 24, got 0" followed by "PANIC:
+  could not locate a valid checkpoint record". *backup_volume* takes
+  *authoritative* as a required keyword rather than an optional flag, so each of
+  the four call sites states which pass it is; the hot passes keep the quick
+  check. *--ignore-times* was rejected because it destroys the *--link-dest*
+  hardlink dedup (measured 20/20 to 0/20, generation size doubled), and
+  *--modify-window=-1* because it makes correctness depend on the destination
+  filesystem preserving sub-second mtimes, which degrades silently on NFS or
+  ext3.
+- Cost: the quick check scales with file count, *--checksum* with bytes read on
+  both sides, and it runs while the container is stopped. The extra stop time
+  stays under a minute up to roughly 4 GB on spinning disk, 15 GB on a SATA SSD
+  and 60 GB on NVMe; at 1 TB it is 17 minutes on NVMe and over three hours on
+  spinning disk. No size threshold is built in, since a guessed one would drop
+  the guarantee exactly where an unrestorable backup costs most — volumes at
+  that scale want filesystem snapshots or *pg_basebackup* rather than two rsync
+  passes over a live tree.
+
+## [3.1.3] - 2026-07-20
+
+- Restore: the postgres dump replay now runs under *--single-transaction*,
+  so a concurrent writer on a live database can no longer interleave a row
+  between the replay's table re-create and its *COPY* and trip a "duplicate
+  key value violates unique constraint" abort under ON_ERROR_STOP. This is
+  the discourse restore-drill race (*mini_scheduler* upserting
+  *scheduler_stats(id=1)* mid-restore) that failed the whole restore. The
+  *--empty* pre-clean stays multi-statement (*\gexec*, one DROP per
+  statement) because a single DROP transaction exhausts
+  *max_locks_per_transaction* on large schemas (e.g. gitlab).
+- Refactor: the *--empty* pre-clean SQL moves out of the inline Python
+  string into *src/baudolo/restore/db/empty_preclean.sql* (loaded via
+  *dirname(__file__)*, declared as package-data so it ships in the wheel).
+- Tests: a unit test guards the single-transaction / multi-statement split
+  (replay carries *--single-transaction*, pre-clean does not); a new e2e
+  reproduces the live-writer race and asserts the restore survives it.
+
+## [3.1.2] - 2026-07-18
+
+- Restore: the postgres *--empty* pre-clean also drops user-owned text
+  search configurations and dictionaries (*pg_ts_config*, *pg_ts_dict*),
+  so a schema shipping a custom dictionary (e.g. taiga's
+  *english_stem_nostop*) no longer aborts the replay with "duplicate key
+  value violates unique constraint pg_ts_dict_dictname_index" under
+  ON_ERROR_STOP.
+- Tests: the string-assertion unit test for the pre-clean SQL is replaced
+  by real scenario data in the e2e: the seeded schema contains an
+  overloaded *f()/f(int)* pair and the nostop dictionary plus
+  configuration, and the restored database is queried to prove each
+  survives the backup, pre-clean and replay cycle exactly once.
+
+## [3.1.1] - 2026-07-17
+
+- Restore: the postgres *--empty* pre-clean drops functions and procedures
+  by their identity signature (*pg_get_function_identity_arguments*), so a
+  schema that overloads a function name (e.g. discourse) no longer aborts
+  the replay with "function name is not unique" under ON_ERROR_STOP.
+  Identifier quoting moves from the outer DROP format into each object
+  branch, since the *name(args)* compound must not be quoted as a whole; a
+  unit test pins the per-branch *%I* quoting so future branches cannot
+  regress unquoted.
+
+## [3.1.0] - 2026-07-15
+
+- Restore: the postgres *--empty* pre-clean emits one DROP per object and
+  runs them via *\gexec* instead of a single DO-block, so large schemas
+  (e.g. gitlab) no longer exhaust *max_locks_per_transaction* in one
+  transaction. It also drops user-owned non-public schemas, so dumps that
+  CREATE SCHEMA (e.g. discourse's *discourse_functions*) no longer abort
+  on the already-existing schema under ON_ERROR_STOP.
+- Backup: *--database-containers* and *--images-no-stop-required* are now
+  optional and default to an empty list, so a pure file backup needs no
+  dummy arguments; an empty stop whitelist keeps the conservative
+  stop-all behavior.
+- Tests: new e2e test restores *--empty* against a fully populated
+  database containing a non-public schema and every dropped object class.
+  *make test* runs the three suites concurrently after a single
+  clean+build; *E2E_TEST_PATTERN* runs an e2e subset.
+
+## [3.0.0] - 2026-07-12
+
+- Backup: *--images-no-stop-required* and *--images-no-backup-required* now
+  match a container's exact *.Config.Image* (full *repo:tag*, registry
+  prefix included) instead of a substring, so a near-miss image name no
+  longer flips the stop/skip decision. Callers must pass exact image
+  references. **Breaking.**
+- Backup: renamed *--hard-compose-restart* to *--hard-restart-projects*
+  (its value stays a list of compose project dir names). **Breaking:** the
+  old flag name is removed.
+
+## [2.0.0] - 2026-07-12
+
+- Backup: renamed *--docker-compose-hard-restart-required* to
+  *--hard-compose-restart* and changed its default from *["mailu"]* to *[]*
+  (nargs="*"). The compose down/up is now opt-in: compose hosts pass
+  *mailu* explicitly, while swarm hosts pass nothing, since there the dir is
+  a stack whose overlay network collides with *compose up*. **Breaking:** the
+  old flag name is removed and the implicit mailu default is gone.
+- Backup: *--backups-dir* is now required (no */var/lib/backup/* default) so
+  a run can never silently target the wrong backup root. **Breaking.**
+- Restore: volume files are rsynced directly into the target volume's
+  mountpoint (resolved via *docker volume inspect*), mirroring the backup
+  path; the *alpine-rsync* helper image and the *--rsync-image* flag are
+  gone. The caller needs write access to the docker volume root (root on the
+  host, baudolo's normal privilege). **Breaking:** the restore *files*
+  subcommand no longer accepts *--rsync-image*.
+- Tests: the e2e suite tracks *postgres:alpine* (18+, mounted at
+  */var/lib/postgresql*) and *mariadb:latest* from a single source of truth.
+
+## [1.8.1] - 2026-07-12
+
+- Restore: the postgres empty mode also drops user-owned collations in
+  public; dumps containing CREATE COLLATION (e.g. OpenProject's ICU
+  collation versions_name) no longer abort the replay with 'collation
+  already exists'.
+- Maintenance: base image bumped from python 3.11-slim to 3.14-slim.
+
+## [1.8.0] - 2026-07-11
+
+Swarm-aware backups and replayable restores.
+
+- Backup: swarm task containers are never stopped or started manually
+  anymore; they are skipped visibly and backed up hot, while the sql dump
+  stays the consistent database backup.
+- Backup: a container that vanishes between listing and inspect no longer
+  aborts the run; a failing inspect on a container that still exists keeps
+  failing loudly.
+- Backup: pg_dump runs with the no-owner and no-privileges flags so dumps
+  are replayable by the owning app user.
+- Restore: the mariadb empty mode drops all tables in one client session
+  with FOREIGN_KEY_CHECKS disabled; FK-linked parent tables no longer abort
+  the replay with ERROR 1451.
+- Restore: the postgres empty mode drops only current-user-owned objects,
+  and the replay skips superuser-only dump lines without ever touching
+  COPY data blocks.
+- Restore: the replay streams the dump through a temp file instead of
+  buffering it in memory; multi-GB dumps no longer OOM the restore.
+- Tooling: the e2e runner reaches the DinD daemon via docker exec instead
+  of a host-published unencrypted API port.
+- Tooling: new end-to-end test reproducing the swarm stop flake, plus unit
+  tests for the restore filters and the swarm probes; the suite is 36 unit,
+  9 integration and 30 e2e tests.
+- Tooling: Dependabot with auto-merge for minor and patch updates.
+
+## [1.7.1] - 2026-05-26
+
+* 🔌 MariaDB SQL backups now connect over TCP loopback so the dump always matches the same wildcard-host grant the application uses — no more surprise `ERROR 1045 Access denied` when a localhost-bound auth row preempts.
+* 🧪 New regression and bug-repro tests pin the TCP behaviour and prove it under the exact preemption setup that caused the production failure on MariaDB 12.
+* 🩺 E2E test infrastructure: DinD bridge and inner daemon now default to MTU 1280 so registry pulls survive host paths with broken PMTUD (override via `E2E_DIND_MTU`).
+
+
+## [1.7.0] - 2026-02-07
+
+* 🚀 Backup jobs now support all valid Docker Compose file names – case-insensitive and hassle-free.
+
+
+## [1.6.0] - 2026-02-06
+
+* Compose handling is now fully delegated to the Infinito.Nexus compose wrapper or plain docker compose, removing all custom env and file detection to ensure a single, consistent source of truth.
+
+
+## [1.5.0] - 2026-01-31
+
+* * Make `databases.csv` optional: missing or empty files now emit warnings and no longer break backups
+* Fix Docker CLI compatibility by switching to `docker-ce-cli` and required build tools
+
+
+## [1.4.0] - 2026-01-31
+
+* Baudolo now restarts Docker Compose stacks in a wrapper-aware way (with a `docker compose` fallback), ensuring that all Compose overrides and env files are applied identically to the Infinito.Nexus workflow.
+
+
+## [1.3.0] - 2026-01-10
+
+* Empty databases.csv no longer causes baudolo-seed to fail
+
+
+## [1.2.0] - 2025-12-29
+
+* * Introduced **`--dump-only-sql`** mode for reliable, SQL-only database backups (replaces `--dump-only`).
+* Database configuration in `databases.csv` is now **strict and explicit** (`*` or concrete database name only).
+* **PostgreSQL cluster backups** are supported via `*`.
+* SQL dumps are written **atomically** to avoid corrupted or empty files.
+* Backups are **smarter and faster**: ignored volumes are skipped early, file backups run only when needed.
+* Improved reliability through expanded end-to-end tests and safer defaults.
+
+
+## [1.1.1] - 2025-12-28
+
+* * **Backup:** In ***--dump-only-sql*** mode, fall back to file backups with a warning when no database dump can be produced (e.g. missing `databases.csv` entry).
+
+
 ## [1.1.0] - 2025-12-28
 
 * * **Backup:** Log a warning and skip database dumps when no databases.csv entry is present instead of raising an exception; introduce module-level logging and apply formatting cleanups across backup/restore code and tests.

Dockerfile

@@ -1,34 +1,37 @@
 # syntax=docker/dockerfile:1
-FROM python:3.11-slim
+FROM python:3.14-slim
 
 WORKDIR /app
 
-# Runtime + build essentials:
+# Base deps for build/runtime + docker repo key
-# - rsync: required for file backup/restore
-# - ca-certificates: TLS
-# - docker-cli: needed if you want to control the host Docker engine (via /var/run/docker.sock mount)
-# - make: to delegate install logic to Makefile
-#
-# Notes:
-# - On Debian slim, the docker client package is typically "docker.io".
-# - If you only want restore-without-docker, you can drop docker.io later.
 RUN apt-get update && apt-get install -y --no-install-recommends \
     make \
     rsync \
     ca-certificates \
-    docker-cli \
+    bash \
+    curl \
+    gnupg \
   && rm -rf /var/lib/apt/lists/*
 
+# Install Docker CLI (docker-ce-cli) from Docker's official apt repo
+RUN bash -lc "set -euo pipefail \
+  && install -m 0755 -d /etc/apt/keyrings \
+  && curl -fsSL https://download.docker.com/linux/debian/gpg \
+     | gpg --dearmor -o /etc/apt/keyrings/docker.gpg \
+  && chmod a+r /etc/apt/keyrings/docker.gpg \
+  && . /etc/os-release \
+  && echo \"deb [arch=\$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian \${VERSION_CODENAME} stable\" \
+     > /etc/apt/sources.list.d/docker.list \
+  && apt-get update \
+  && apt-get install -y --no-install-recommends docker-ce-cli \
+  && rm -rf /var/lib/apt/lists/*"
+
 # Fail fast if docker client is missing
+RUN docker version || true
 RUN command -v docker
 
 COPY . .
-
-# All install decisions are handled by the Makefile.
 RUN make install
 
-# Sensible defaults (can be overridden at runtime)
 ENV PYTHONUNBUFFERED=1
-
-# Default: show CLI help
 CMD ["baudolo", "--help"]

Makefile

@@ -1,5 +1,6 @@
-.PHONY: install build \
+.PHONY: install install-lint build clean lint ruff ruff-fix \
-		test-e2e test test-unit test-integration
+		test test-unit test-integration test-e2e \
+		test-unit-run test-integration-run test-e2e-run
 
 # Default python if no venv is active
 PY_DEFAULT ?= python3
@@ -33,25 +34,49 @@ build:
 clean:
 	git clean -fdX .
 
-# ------------------------------------------------------------
+# Separate from `install` so the test image does not have to carry the linter.
-# Run E2E tests inside the container (Docker socket required)
+install-lint:
-# ------------------------------------------------------------
+	@$(PY_DEFAULT) -m pip install -q -e ".[lint]"
-# E2E via isolated Docker-in-Docker (DinD)
-# - depends on local image build
-# - starts a DinD daemon container on a dedicated network
-# - loads the freshly built image into DinD
-# - runs the unittest suite inside a container that talks to DinD via DOCKER_HOST
-test-e2e: clean build
-	@bash scripts/test-e2e.sh
 
-test: test-unit test-integration test-e2e
+# Runs on the host, not in the image, so it also covers what the Dockerfile
+# does not copy.
+ruff: install-lint
+	@echo ">> Running ruff over the whole repository"
+	@$(PY_DEFAULT) -m ruff check .
+	@$(PY_DEFAULT) -m ruff format --check .
 
-test-unit: clean build
+ruff-fix: install-lint
+	@$(PY_DEFAULT) -m ruff check --fix .
+	@$(PY_DEFAULT) -m ruff format .
+
+lint: ruff
+
+# build runs once, then lint and the three suites run concurrently via -j4; the
+# *-run targets carry no build prereq so the sub-make cannot race a second build.
+# `clean` is deliberately not a prerequisite; .dockerignore keeps the image
+# context clean instead.
+test:
+	@$(MAKE) build
+	@$(MAKE) -j4 lint test-unit-run test-integration-run test-e2e-run
+
+test-unit: build test-unit-run
+
+test-integration: build test-integration-run
+
+test-e2e: build test-e2e-run
+
+test-unit-run:
 	@echo ">> Running unit tests"
 	@docker run --rm -t $(IMAGE) \
-	  sh -lc 'python -m unittest discover -t . -s tests/unit -p "test_*.py" -v'
+	  bash -lc 'python -m unittest discover -t . -s tests/unit -p "test_*.py" -v'
 
-test-integration: clean build
+test-integration-run:
 	@echo ">> Running integration tests"
 	@docker run --rm -t $(IMAGE) \
-	  sh -lc 'python -m unittest discover -t . -s tests/integration -p "test_*.py" -v'
+	  bash -lc 'python -m unittest discover -t . -s tests/integration -p "test_*.py" -v'
+
+# E2E via isolated Docker-in-Docker (DinD): starts a DinD daemon on a dedicated
+# network, loads the freshly built image into it, and runs tests/e2e inside a
+# container that talks to DinD via DOCKER_HOST.
+test-e2e-run:
+	@bash scripts/test-e2e.sh

README.md

@@ -123,6 +123,8 @@ This information is used by `baudolo` to execute
 ```bash
 baudolo \
   --compose-dir /srv/docker \
+  --backups-dir /Backups \
+  --repo-name my-repo \
   --databases-csv /etc/baudolo/databases.csv \
   --database-containers central-postgres central-mariadb \
   --images-no-stop-required alpine postgres mariadb mysql \
@@ -133,11 +135,12 @@ baudolo \
 
 | Flag            | Description                                 |
 | --------------- | ------------------------------------------- |
-| `--everything`  | Always stop containers and re-run rsync     |
+| `--only-sql`    | Skip file backups only for DB volumes when dumps succeed; non-DB volumes are still backed up; fallback to files if no dump.    |
-| `--dump-only`   | Only create SQL dumps, skip file backups    |
+| `--only-files`  | Take no dumps at all; every volume is backed up as files. Needs no `--databases-csv`. Mutually exclusive with `--only-sql`. |
 | `--shutdown`    | Do not restart containers after backup      |
-| `--backups-dir` | Backup root directory (default: `/Backups`) |
+| `--backups-dir` | Backup root directory (required)            |
-| `--repo-name`   | Backup namespace under machine hash         |
+| `--repo-name`   | Backup namespace under machine hash (required) |
+| `--databases-csv`| Path to `databases.csv` (required)         |
 
 ## ♻️ Restore Operations
 

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "backup-docker-to-local"
-version = "1.1.0"
+version = "7.0.0"
 description = "Backup Docker volumes to local with rsync and optional DB dumps."
 readme = "README.md"
 requires-python = ">=3.9"
@@ -16,6 +16,11 @@ dependencies = [
   "dirval",
 ]
 
+[project.optional-dependencies]
+# Pinned: a ruff minor bump changes which rules fire, and `make test` gates on
+# a clean run, so an unpinned lint would fail the suite on an unrelated day.
+lint = ["ruff==0.16.1"]
+
 [project.scripts]
 baudolo = "baudolo.backup.__main__:main"
 baudolo-restore = "baudolo.restore.__main__:main"
@@ -27,3 +32,55 @@ package-dir = { "" = "src" }
 [tool.setuptools.packages.find]
 where = ["src"]
 exclude = ["tests*"]
+
+[tool.setuptools.package-data]
+"baudolo.restore.db" = ["*.sql"]
+
+[tool.ruff]
+respect-gitignore = true
+# The package still declares >=3.9, so pyupgrade must not propose 3.10+ syntax.
+target-version = "py39"
+exclude = ["build", "dist", "*.egg-info", ".venv", "venv"]
+
+[tool.ruff.lint]
+# Adopted from infinito-nexus-core so both repositories are held to one bar;
+# see that project's pyproject.toml for what each selector buys.
+select = [
+    "E", "F", "I", "B", "UP", "RUF", "SIM", "C4", "PERF", "RET", "PIE",
+    "T10", "PGH", "EXE", "RSE", "ICN", "DTZ",
+    "TID", "LOG", "G",
+    "S",
+    "PTH",
+    "FURB", "W", "FA", "YTT", "A", "ISC", "SLOT", "FLY",
+    "PYI",
+    "TC", "N",
+    "PLE0605",
+    "PLW1510", "PLW2901", "PLW0108", "PLW0603",
+    "PLR5501", "PLC0207", "PLR1722", "PLR1714",
+    "TRY002", "TRY004", "TRY300", "TRY301",
+    "BLE001",
+]
+
+# E501: `ruff format` reflows what it can; the rest is unsplittable literals.
+# RUF001/002/003: the prose uses em-dashes deliberately, not homoglyphs.
+# S603/S607: this tool's whole job is running `docker` / dump binaries from
+#   PATH in list form, which is already injection-safe.
+# PTH207/PTH208: changing `glob.glob`/`os.listdir` return shapes needs a
+#   per-call-site review, not a blanket rewrite.
+ignore = [
+    "E501",
+    "RUF001", "RUF002", "RUF003",
+    "S603", "S607",
+    "PTH207", "PTH208",
+]
+
+[tool.ruff.lint.per-file-ignores]
+# Test code legitimately uses what flake8-bandit flags in production code:
+# asserts, dummy credentials, /tmp fixtures, broad excepts in teardown, and
+# SQL built from fixture names (S608) to set the databases under test up.
+"tests/**" = [
+    "S101", "S102", "S105", "S106", "S108", "S110", "S112", "S608", "BLE001",
+]
+
+# The e2e helpers package is a deliberate re-export aggregator.
+"tests/e2e/helpers/__init__.py" = ["F403"]

scripts/test-e2e.sh

@@ -16,19 +16,30 @@ DIND="${E2E_DIND_NAME:-baudolo-e2e-dind}"
 DIND_VOL="${E2E_DIND_VOL:-baudolo-e2e-dind-data}"
 E2E_TMP_VOL="${E2E_TMP_VOL:-baudolo-e2e-tmp}"
 
-DIND_HOST="${E2E_DIND_HOST:-tcp://127.0.0.1:2375}"
+# Host-side access to the DinD daemon goes through `docker exec` (dind()
+# below) instead of a host-published port: port publishing is not reachable
+# from every environment (sandboxed runners, hosts with broken loopback
+# publishing), while exec only needs the outer docker socket. The TCP
+# listener stays for the test container inside the dedicated network.
 DIND_HOST_IN_NET="${E2E_DIND_HOST_IN_NET:-tcp://${DIND}:2375}"
 
+dind() { docker exec "${DIND}" docker "$@"; }
+dind_stdin() { docker exec -i "${DIND}" docker "$@"; }
+
 IMG="${E2E_IMAGE:-baudolo:local}"
-RSYNC_IMG="${E2E_RSYNC_IMAGE:-ghcr.io/kevinveenbirkenbach/alpine-rsync}"
 
 READY_TIMEOUT_SECONDS="${E2E_READY_TIMEOUT_SECONDS:-120}"
 ARTIFACTS_DIR="${E2E_ARTIFACTS_DIR:-./artifacts}"
 
+DIND_MTU="${E2E_DIND_MTU:-1280}"
+
 KEEP_ON_FAIL="${E2E_KEEP_ON_FAIL:-0}"
 KEEP_VOLUMES="${E2E_KEEP_VOLUMES:-0}"
 DEBUG_SHELL="${E2E_DEBUG_SHELL:-0}"
 
+# Override to run a subset, e.g. E2E_TEST_PATTERN=test_e2e_postgres_empty_drop_hard.py
+TEST_PATTERN="${E2E_TEST_PATTERN:-test_*.py}"
+
 FAILED=0
 TS="$(date +%Y%m%d%H%M%S)"
 
@@ -46,8 +57,8 @@ dump_debug() {
     echo "=== Host docker info ==="
     docker info || true
     echo
-    echo "=== DinD reachable? (docker -H ${DIND_HOST} version) ==="
+    echo "=== DinD reachable? (docker exec ${DIND} docker version) ==="
-    docker -H "${DIND_HOST}" version || true
+    dind version || true
     echo
   } > "${ARTIFACTS_DIR}/debug-host-${TS}.txt" 2>&1 || true
 
@@ -56,48 +67,34 @@ dump_debug() {
 
   # DinD state
   {
-    echo "=== docker -H ps -a ==="
+    echo "=== dind ps -a ==="
-    docker -H "${DIND_HOST}" ps -a || true
+    dind ps -a || true
     echo
-    echo "=== docker -H images ==="
+    echo "=== dind images ==="
-    docker -H "${DIND_HOST}" images || true
+    dind images || true
     echo
-    echo "=== docker -H network ls ==="
+    echo "=== dind network ls ==="
-    docker -H "${DIND_HOST}" network ls || true
+    dind network ls || true
     echo
-    echo "=== docker -H volume ls ==="
+    echo "=== dind volume ls ==="
-    docker -H "${DIND_HOST}" volume ls || true
+    dind volume ls || true
     echo
-    echo "=== docker -H system df ==="
+    echo "=== dind system df ==="
-    docker -H "${DIND_HOST}" system df || true
+    dind system df || true
   } > "${ARTIFACTS_DIR}/debug-dind-${TS}.txt" 2>&1 || true
 
   # Try to capture recent events (best effort; might be noisy)
-  docker -H "${DIND_HOST}" events --since 10m --until 0s \
+  dind events --since 10m --until 0s \
     > "${ARTIFACTS_DIR}/dind-events-${TS}.txt" 2>&1 || true
 
-  # Dump shared /tmp content from the tmp volume:
+  # The shared tmp volume is mounted at /tmp inside the DinD container
-  # We create a temporary container that mounts the volume, then tar its content.
+  # itself, so tar it there and copy it out with the outer daemon.
-  # (Does not rely on host filesystem paths.)
   log "DEBUG: archiving shared /tmp (volume ${E2E_TMP_VOL})"
-  docker -H "${DIND_HOST}" run --rm \
+  docker exec "${DIND}" tar -czf "/tmpdump-${TS}.tar.gz" -C /tmp . >/dev/null 2>&1 || true
-    -v "${E2E_TMP_VOL}:/tmp" \
+  docker cp "${DIND}:/tmpdump-${TS}.tar.gz" "${ARTIFACTS_DIR}/e2e-tmp-${TS}.tar.gz" >/dev/null 2>&1 || true
-    alpine:3.20 \
-    sh -lc 'cd /tmp && tar -czf /out.tar.gz . || true' \
-    >/dev/null 2>&1 || true
-
-  # The above writes inside the container FS, not to host. So do it properly:
-  # Use "docker cp" from a temp container.
-  local tmpc="baudolo-e2e-tmpdump-${TS}"
-  docker -H "${DIND_HOST}" rm -f "${tmpc}" >/dev/null 2>&1 || true
-  docker -H "${DIND_HOST}" create --name "${tmpc}" -v "${E2E_TMP_VOL}:/tmp" alpine:3.20 \
-    sh -lc 'cd /tmp && tar -czf /tmpdump.tar.gz . || true' >/dev/null
-  docker -H "${DIND_HOST}" start -a "${tmpc}" >/dev/null 2>&1 || true
-  docker -H "${DIND_HOST}" cp "${tmpc}:/tmpdump.tar.gz" "${ARTIFACTS_DIR}/e2e-tmp-${TS}.tar.gz" >/dev/null 2>&1 || true
-  docker -H "${DIND_HOST}" rm -f "${tmpc}" >/dev/null 2>&1 || true
 
   log "DEBUG: artifacts written:"
-  ls -la "${ARTIFACTS_DIR}" | sed 's/^/   /' || true
+  find "${ARTIFACTS_DIR}" -maxdepth 1 -mindepth 1 -print | sed 's/^/   /' || true
 }
 
 cleanup() {
@@ -105,9 +102,9 @@ cleanup() {
     log "KEEP_ON_FAIL=1 and failure detected -> skipping cleanup."
     log "Next steps:"
     echo "   - Inspect DinD logs: docker logs ${DIND} | less"
-    echo "   - Use DinD daemon:  docker -H ${DIND_HOST} ps -a"
+    echo "   - Use DinD daemon:  docker exec ${DIND} docker ps -a"
-    echo "   - Shared tmp vol:   docker -H ${DIND_HOST} run --rm -v ${E2E_TMP_VOL}:/tmp alpine:3.20 ls -la /tmp"
+    echo "   - Shared tmp vol:   docker exec ${DIND} ls -la /tmp"
-    echo "   - DinD docker root: docker -H ${DIND_HOST} run --rm -v ${DIND_VOL}:/var/lib/docker alpine:3.20 ls -la /var/lib/docker/volumes"
+    echo "   - DinD docker root: docker exec ${DIND} ls -la /var/lib/docker/volumes"
     return 0
   fi
 
@@ -124,8 +121,11 @@ cleanup() {
 }
 trap cleanup EXIT INT TERM
 
-log "Creating network ${NET} (if missing)"
+log "(Re)creating network ${NET} with MTU ${DIND_MTU}"
-docker network inspect "${NET}" >/dev/null 2>&1 || docker network create "${NET}" >/dev/null
+docker network rm "${NET}" >/dev/null 2>&1 || true
+docker network create \
+  --opt com.docker.network.driver.mtu="${DIND_MTU}" \
+  "${NET}" >/dev/null
 
 log "Removing old ${DIND} (if any)"
 docker rm -f "${DIND}" >/dev/null 2>&1 || true
@@ -145,14 +145,14 @@ docker run -d --privileged \
   -e DOCKER_TLS_CERTDIR="" \
   -v "${DIND_VOL}:/var/lib/docker" \
   -v "${E2E_TMP_VOL}:/tmp" \
-  -p 2375:2375 \
   docker:dind \
   --host=tcp://0.0.0.0:2375 \
-  --tls=false >/dev/null
+  --tls=false \
+  --mtu="${DIND_MTU}" >/dev/null
 
 log "Waiting for DinD to be ready..."
 for i in $(seq 1 "${READY_TIMEOUT_SECONDS}"); do
-  if docker -H "${DIND_HOST}" version >/dev/null 2>&1; then
+  if dind version >/dev/null 2>&1; then
     log "DinD is ready."
     break
   fi
@@ -166,15 +166,11 @@ for i in $(seq 1 "${READY_TIMEOUT_SECONDS}"); do
   fi
 done
 
-log "Pre-pulling helper images in DinD..."
-log " - Pulling: ${RSYNC_IMG}"
-docker -H "${DIND_HOST}" pull "${RSYNC_IMG}"
-
 log "Ensuring alpine exists in DinD (for debug helpers)"
-docker -H "${DIND_HOST}" pull alpine:3.20 >/dev/null
+dind pull alpine:3.20 >/dev/null
 
 log "Loading ${IMG} image into DinD..."
-docker save "${IMG}" | docker -H "${DIND_HOST}" load >/dev/null
+docker save "${IMG}" | dind_stdin load >/dev/null
 
 log "Running E2E tests inside DinD"
 set +e
@@ -183,11 +179,10 @@ if [ "${DEBUG_SHELL}" = "1" ]; then
   docker run --rm -it \
     --network "${NET}" \
     -e DOCKER_HOST="${DIND_HOST_IN_NET}" \
-    -e E2E_RSYNC_IMAGE="${RSYNC_IMG}" \
+    -v "${DIND_VOL}:/var/lib/docker" \
-    -v "${DIND_VOL}:/var/lib/docker:ro" \
     -v "${E2E_TMP_VOL}:/tmp" \
     "${IMG}" \
-    sh -lc '
+    bash -lc '
       set -e
       if [ ! -f /etc/machine-id ]; then
         mkdir -p /etc
@@ -195,18 +190,19 @@ if [ "${DEBUG_SHELL}" = "1" ]; then
       fi
       echo ">> DOCKER_HOST=${DOCKER_HOST}"
       docker ps -a || true
-      exec sh
+      exec bash
     '
   rc=$?
 else
   docker run --rm \
     --network "${NET}" \
     -e DOCKER_HOST="${DIND_HOST_IN_NET}" \
-    -e E2E_RSYNC_IMAGE="${RSYNC_IMG}" \
+    -e E2E_TEST_PATTERN="${TEST_PATTERN}" \
-    -v "${DIND_VOL}:/var/lib/docker:ro" \
+    -e E2E_REQUIRE_FILESYSTEMS="${E2E_REQUIRE_FILESYSTEMS:-}" \
+    -v "${DIND_VOL}:/var/lib/docker" \
     -v "${E2E_TMP_VOL}:/tmp" \
     "${IMG}" \
-    sh -lc '
+    bash -lc '
     set -euo pipefail
     set -x
     export PYTHONUNBUFFERED=1
@@ -218,7 +214,7 @@ else
         cat /proc/sys/kernel/random/uuid > /etc/machine-id
     fi
 
-    python -m unittest discover -t . -s tests/e2e -p "test_*.py" -v -f
+    python -m unittest discover -t . -s tests/e2e -p "${E2E_TEST_PATTERN}" -v -f
     '
   rc=$?
 fi

src/baudolo/backup/__main__.py

@@ -1,9 +1,6 @@
-#!/usr/bin/env python3
-
 from __future__ import annotations
 
 from .app import main
 
-
 if __name__ == "__main__":
     raise SystemExit(main())

src/baudolo/backup/app.py

@@ -1,187 +1,143 @@
+"""Back up every Docker volume of a host into a timestamped generation."""
+
 from __future__ import annotations
 
-import os
+from contextlib import ExitStack
-import pathlib
 from datetime import datetime
-
+from pathlib import Path
-import pandas
-from dirval import create_stamp_file
 
 from .cli import parse_args
 from .compose import handle_docker_compose_services
-from .db import backup_database
 from .docker import (
     change_containers_status,
     containers_using_volume,
     docker_volume_names,
-    get_image_info,
+    filter_stoppable,
-    has_image,
 )
-from .shell import execute_shell_command
+from .dumps import VolumeOutcome, backup_dumps_for_volume, load_databases_df
-from .volume import backup_volume
+from .layout import (
-
+    create_version_directory,
-
+    create_volume_directory,
-def get_machine_id() -> str:
+    get_machine_id,
-    return execute_shell_command("sha256sum /etc/machine-id")[0][0:64]
+    stamp_directory,
-
+    write_manifest,
-
+)
-def stamp_directory(version_dir: str) -> None:
+from .policy import requires_stop, volume_is_fully_ignored
-    """
+from .snapshot import snapshot_source, volume_snapshot
-    Use dirval as a Python library to stamp the directory (no CLI dependency).
+from .volume import backup_volume, inspect_backing
-    """
-    create_stamp_file(version_dir)
-
-
-def create_version_directory(versions_dir: str, backup_time: str) -> str:
-    version_dir = os.path.join(versions_dir, backup_time)
-    pathlib.Path(version_dir).mkdir(parents=True, exist_ok=True)
-    return version_dir
-
-
-def create_volume_directory(version_dir: str, volume_name: str) -> str:
-    path = os.path.join(version_dir, volume_name)
-    pathlib.Path(path).mkdir(parents=True, exist_ok=True)
-    return path
-
-
-def is_image_ignored(container: str, images_no_backup_required: list[str]) -> bool:
-    if not images_no_backup_required:
-        return False
-    img = get_image_info(container)
-    return any(pat in img for pat in images_no_backup_required)
-
-
-def volume_is_fully_ignored(
-    containers: list[str], images_no_backup_required: list[str]
-) -> bool:
-    """
-    Skip file backup only if all containers linked to the volume are ignored.
-    """
-    if not containers:
-        return False
-    return all(is_image_ignored(c, images_no_backup_required) for c in containers)
-
-
-def requires_stop(containers: list[str], images_no_stop_required: list[str]) -> bool:
-    """
-    Stop is required if ANY container image is NOT in the whitelist patterns.
-    """
-    for c in containers:
-        img = get_image_info(c)
-        if not any(pat in img for pat in images_no_stop_required):
-            return True
-    return False
-
-
-def backup_mariadb_or_postgres(
-    *,
-    container: str,
-    volume_dir: str,
-    databases_df: "pandas.DataFrame",
-    database_containers: list[str],
-) -> bool:
-    """
-    Returns True if the container is a DB container we handled.
-    """
-    for img in ["mariadb", "postgres"]:
-        if has_image(container, img):
-            backup_database(
-                container=container,
-                volume_dir=volume_dir,
-                db_type=img,
-                databases_df=databases_df,
-                database_containers=database_containers,
-            )
-            return True
-    return False
-
-
-def _backup_dumps_for_volume(
-    *,
-    containers: list[str],
-    vol_dir: str,
-    databases_df: "pandas.DataFrame",
-    database_containers: list[str],
-) -> bool:
-    """
-    Create DB dumps for any mariadb/postgres containers attached to this volume.
-    Returns True if at least one dump was produced.
-    """
-    dumped_any = False
-    for c in containers:
-        if backup_mariadb_or_postgres(
-            container=c,
-            volume_dir=vol_dir,
-            databases_df=databases_df,
-            database_containers=database_containers,
-        ):
-            dumped_any = True
-    return dumped_any
 
 
 def main() -> int:
     args = parse_args()
 
     machine_id = get_machine_id()
-    backup_time = datetime.now().strftime("%Y%m%d%H%M%S")
+    # Local wall clock on purpose: generations sort by this name, and UTC would
+    # order new ones before the existing ones wherever the offset is positive.
+    backup_time = datetime.now().strftime("%Y%m%d%H%M%S")  # noqa: DTZ005
 
-    versions_dir = os.path.join(args.backups_dir, machine_id, args.repo_name)
+    versions_dir = str(Path(args.backups_dir) / machine_id / args.repo_name)
     version_dir = create_version_directory(versions_dir, backup_time)
 
-    databases_df = pandas.read_csv(args.databases_csv, sep=";")
+    databases_df = None if args.only_files else load_databases_df(args.databases_csv)
 
     print("💾 Start volume backups...", flush=True)
 
-    for volume_name in docker_volume_names():
+    outcomes: dict[str, VolumeOutcome] = {}
-        print(f"Start backup routine for volume: {volume_name}", flush=True)
-        containers = containers_using_volume(volume_name)
 
-        vol_dir = create_volume_directory(version_dir, volume_name)
+    with ExitStack() as stack:
-
+        resolve_source = None
-        # Old behavior: DB dumps are additional to file backups.
+        if args.snapshot:
-        _backup_dumps_for_volume(
+            resolve_source = stack.enter_context(
-            containers=containers,
+                volume_snapshot(args.snapshot, args.snapshot_subject, backup_time)
-            vol_dir=vol_dir,
-            databases_df=databases_df,
-            database_containers=args.database_containers,
-        )
-
-        # dump-only: skip ALL file rsync backups
-        if args.dump_only:
-            continue
-
-        # skip file backup if all linked containers are ignored
-        if volume_is_fully_ignored(containers, args.images_no_backup_required):
-            print(
-                f"Skipping file backup for volume '{volume_name}' (all linked containers are ignored).",
-                flush=True,
             )
-            continue
 
-        if args.everything:
+        for volume_name in docker_volume_names():
-            # "everything": always do pre-rsync, then stop + rsync again
+            print(f"Start backup routine for volume: {volume_name}", flush=True)
-            backup_volume(versions_dir, volume_name, vol_dir)
-            change_containers_status(containers, "stop")
-            backup_volume(versions_dir, volume_name, vol_dir)
-            if not args.shutdown:
-                change_containers_status(containers, "start")
-            continue
 
-        # default: rsync, and if needed stop + rsync
+            if volume_name in args.volumes_no_backup_required:
-        backup_volume(versions_dir, volume_name, vol_dir)
+                print(
-        if requires_stop(containers, args.images_no_stop_required):
+                    f"Skipping volume '{volume_name}' entirely (declared no-backup).",
-            change_containers_status(containers, "stop")
+                    flush=True,
-            backup_volume(versions_dir, volume_name, vol_dir)
+                )
-            if not args.shutdown:
+                continue
-                change_containers_status(containers, "start")
 
-    # Stamp the backup version directory using dirval (python lib)
+            containers = containers_using_volume(volume_name)
+
+            if volume_is_fully_ignored(containers, args.images_no_backup_required):
+                print(
+                    f"Skipping volume '{volume_name}' entirely (all linked containers are ignored).",
+                    flush=True,
+                )
+                continue
+
+            vol_dir = create_volume_directory(version_dir, volume_name)
+
+            outcome = VolumeOutcome(database=False, dumped=False)
+            if not args.only_files:
+                outcome = backup_dumps_for_volume(
+                    containers=containers,
+                    vol_dir=vol_dir,
+                    databases_df=databases_df,
+                    database_containers=args.database_containers,
+                )
+            outcomes[volume_name] = outcome
+
+            if args.only_sql and outcome.database:
+                if not outcome.dumped:
+                    print(
+                        f"WARNING: only-sql requested but no DB dump was produced for DB volume '{volume_name}'. "
+                        "Falling back to file backup.",
+                        flush=True,
+                    )
+                else:
+                    continue
+
+            backing = inspect_backing(volume_name)
+            live_source = backing.source
+
+            def copy(
+                *,
+                authoritative: bool,
+                source: str = live_source,
+                volume: str = volume_name,
+                target: str = vol_dir,
+            ) -> None:
+                backup_volume(
+                    versions_dir,
+                    volume,
+                    target,
+                    authoritative=authoritative,
+                    source=source,
+                )
+
+            if resolve_source is not None:
+                source, reason = snapshot_source(
+                    resolve_source, backing, args.snapshot_subject
+                )
+                if source is not None:
+                    copy(authoritative=True, source=source)
+                else:
+                    print(
+                        f"WARNING: volume '{volume_name}' is not in the snapshot "
+                        f"({reason}); copying it live instead.",
+                        flush=True,
+                    )
+                    copy(authoritative=False)
+                continue
+
+            copy(authoritative=False)
+            if requires_stop(containers, args.images_no_stop_required):
+                stoppable = filter_stoppable(containers)
+                change_containers_status(stoppable, "stop")
+                copy(authoritative=True)
+                if not args.shutdown:
+                    change_containers_status(stoppable, "start")
+
+    write_manifest(version_dir, outcomes)
     stamp_directory(version_dir)
     print("Finished volume backups.", flush=True)
 
     print("Handling Docker Compose services...", flush=True)
-    handle_docker_compose_services(
+    handle_docker_compose_services(args.compose_dir, args.hard_restart_projects)
-        args.compose_dir, args.docker_compose_hard_restart_required
-    )
 
     return 0

src/baudolo/backup/cli.py

@@ -1,13 +1,9 @@
 from __future__ import annotations
 
 import argparse
-import os
 
 
 def parse_args() -> argparse.Namespace:
-    dirname = os.path.dirname(__file__)
-    default_databases_csv = os.path.join(dirname, "databases.csv")
-
     p = argparse.ArgumentParser(description="Backup Docker volumes.")
 
     p.add_argument(
@@ -17,61 +13,102 @@ def parse_args() -> argparse.Namespace:
         help="Path to the parent directory containing docker-compose setups",
     )
     p.add_argument(
-        "--docker-compose-hard-restart-required",
+        "--hard-restart-projects",
-        nargs="+",
+        nargs="*",
-        default=["mailu"],
+        default=[],
-        help="Compose dir names that require 'docker-compose down && up -d' (default: mailu)",
+        help="Compose dir names that require 'docker-compose down && up -d' (default: none; pass e.g. 'mailu' under compose where the DB cannot be backed up hot)",
     )
 
     p.add_argument(
         "--repo-name",
-        default="backup-docker-to-local",
+        required=True,
-        help="Backup repo folder name under <backups-dir>/<machine-id>/ (default: git repo folder name)",
+        help="Backup repo folder name under <backups-dir>/<machine-id>/",
     )
     p.add_argument(
         "--databases-csv",
-        default=default_databases_csv,
+        help="Path to databases.csv; required unless --only-files is given",
-        help=f"Path to databases.csv (default: {default_databases_csv})",
     )
     p.add_argument(
         "--backups-dir",
-        default="/var/lib/backup/",
+        required=True,
-        help="Backup root directory (default: /var/lib/backup/)",
+        help="Backup root directory (e.g. /var/lib/backup/)",
+    )
+
+    p.add_argument(
+        "--snapshot",
+        choices=["btrfs", "zfs"],
+        help="Capture every volume from one atomic filesystem snapshot instead of copying the live tree. Containers are not stopped, and the copy is a single pass. Requires --snapshot-subject. Omit to keep the live two-pass copy.",
+    )
+    p.add_argument(
+        "--snapshot-subject",
+        help="Btrfs subvolume or zfs dataset mountpoint holding the docker volumes, e.g. /var/lib/docker. Required with --snapshot.",
     )
 
     p.add_argument(
         "--database-containers",
         nargs="+",
-        required=True,
+        default=[],
         help="Container names treated as special instances for database backups",
     )
     p.add_argument(
         "--images-no-stop-required",
         nargs="+",
-        required=True,
+        default=[],
-        help="Image name patterns for which containers should not be stopped during file backup",
+        help="Exact image references (repo:tag, incl. any registry prefix) whose containers must not be stopped during file backup",
     )
     p.add_argument(
         "--images-no-backup-required",
         nargs="+",
         default=[],
-        help="Image name patterns for which no backup should be performed",
+        help="Exact image references (repo:tag, incl. any registry prefix) for which no backup should be performed",
     )
 
     p.add_argument(
-        "--everything",
+        "--volumes-no-backup-required",
-        action="store_true",
+        nargs="+",
-        help="Force file backup for all volumes and also execute database dumps (like old script)",
+        default=[],
+        help="Exact volume names that are never backed up, whatever containers use them. For derived trees a restore cannot reproduce, above all a nested docker data root",
     )
+
     p.add_argument(
         "--shutdown",
         action="store_true",
         help="Do not restart containers after backup",
     )
-    p.add_argument(
-        "--dump-only",
-        action="store_true",
-        help="Only create DB dumps (skip ALL file rsync backups)",
-    )
 
-    return p.parse_args()
+    scope = p.add_mutually_exclusive_group()
+    scope.add_argument(
+        "--only-sql",
+        action="store_true",
+        help=(
+            "Create database dumps only for DB volumes. "
+            "File backups are skipped for DB volumes if a dump succeeds, "
+            "but non-DB volumes are still backed up. "
+            "If a DB dump cannot be produced, baudolo falls back to a file backup."
+        ),
+    )
+    scope.add_argument(
+        "--only-files",
+        action="store_true",
+        help=(
+            "Take no database dumps at all and back up every volume as files. "
+            "For hosts that hold no database credentials. A database's files "
+            "are only consistent if its containers are stopped for the second "
+            "pass, so keep its image off --images-no-stop-required."
+        ),
+    )
+    args = p.parse_args()
+    if not args.only_files and not args.databases_csv:
+        p.error("--databases-csv is required unless --only-files is given")
+    if bool(args.snapshot) != bool(args.snapshot_subject):
+        p.error("--snapshot and --snapshot-subject must be given together")
+    if args.snapshot and args.shutdown:
+        p.error(
+            "--shutdown is meaningless with --snapshot: containers are never stopped"
+        )
+    if args.snapshot and args.hard_restart_projects:
+        p.error(
+            "--hard-restart-projects is meaningless with --snapshot: the flag exists "
+            "for stacks whose database cannot be backed up hot, which a snapshot solves"
+        )
+    return args

src/baudolo/backup/compose.py

@@ -1,29 +1,94 @@
 from __future__ import annotations
 
 import os
+import shutil
 import subprocess
+from pathlib import Path
+
+
+def _build_compose_cmd(project_dir: str, passthrough: list[str]) -> list[str]:
+    """
+    Build the compose command for this project directory.
+
+    Policy:
+    - If `compose` wrapper exists (Infinito.Nexus): use it and delegate ALL logic to it.
+    - Else: use plain `docker compose` with --chdir.
+    - NO custom compose file/env detection in this project.
+    """
+    pdir = Path(project_dir).resolve()
+
+    wrapper = shutil.which("compose")
+    if wrapper:
+        # "--" ensures wrapper stops parsing its own args.
+        return [wrapper, "--chdir", str(pdir), "--", *passthrough]
+
+    docker = shutil.which("docker")
+    if docker:
+        return [docker, "compose", "--chdir", str(pdir), *passthrough]
+
+    raise RuntimeError("Neither 'compose' nor 'docker' found in PATH")
+
+
+def _find_compose_file(project_dir: str) -> Path | None:
+    """
+    Detect a compose file in `project_dir` (case-insensitive).
+
+    Supported names:
+    - compose.yml / compose.yaml
+    - docker-compose.yml / docker-compose.yaml
+    """
+    pdir = Path(project_dir)
+    if not pdir.is_dir():
+        return None
+
+    # Map lowercase filename -> actual Path (preserves original casing)
+    by_lower = {p.name.lower(): p for p in pdir.iterdir() if p.is_file()}
+
+    # Preferred order (policy decision)
+    candidates = [
+        "docker-compose.yml",
+        "docker-compose.yaml",
+        "compose.yml",
+        "compose.yaml",
+    ]
+
+    for name in candidates:
+        found = by_lower.get(name)
+        if found is not None:
+            return found
+
+    return None
 
 
 def hard_restart_docker_services(dir_path: str) -> None:
-    print(f"Hard restart docker-compose services in: {dir_path}", flush=True)
+    print(f"Hard restart compose services in: {dir_path}", flush=True)
-    subprocess.run(["docker-compose", "down"], cwd=dir_path, check=True)
+
-    subprocess.run(["docker-compose", "up", "-d"], cwd=dir_path, check=True)
+    down_cmd = _build_compose_cmd(dir_path, ["down"])
+    up_cmd = _build_compose_cmd(dir_path, ["up", "-d"])
+
+    print(">>> " + " ".join(down_cmd), flush=True)
+    subprocess.run(down_cmd, check=True)
+
+    print(">>> " + " ".join(up_cmd), flush=True)
+    subprocess.run(up_cmd, check=True)
 
 
 def handle_docker_compose_services(
-    parent_directory: str, hard_restart_required: list[str]
+    parent_directory: str,
+    hard_restart_required: list[str],
 ) -> None:
     for entry in os.scandir(parent_directory):
         if not entry.is_dir():
             continue
 
         dir_path = entry.path
-        name = os.path.basename(dir_path)
+        name = Path(dir_path).name
-        compose_file = os.path.join(dir_path, "docker-compose.yml")
 
         print(f"Checking directory: {dir_path}", flush=True)
-        if not os.path.isfile(compose_file):
+
-            print("No docker-compose.yml found. Skipping.", flush=True)
+        compose_file = _find_compose_file(dir_path)
+        if compose_file is None:
+            print("No supported compose file found. Skipping.", flush=True)
             continue
 
         if name in hard_restart_required:

src/baudolo/backup/db.py

@@ -1,31 +1,68 @@
 from __future__ import annotations
 
-import os
+import logging
 import pathlib
 import re
+from typing import TYPE_CHECKING
 
-import pandas
+from baudolo.databases import CLUSTER_ROW, validate_database
-import logging
+from baudolo.generation import CLUSTER_SUFFIX, DUMP_SUFFIX, SQL_DIR
 
-from .shell import BackupException, execute_shell_command
+from .docker import docker_exec_argv
+from .shell import BackupError, execute_to_file
+
+if TYPE_CHECKING:
+    import pandas as pd
 
 log = logging.getLogger(__name__)
 
+ENGINE_NAMES = ("database", "postgres", "mariadb", "mysql", "db")
+_SUFFIX_RE = re.compile(rf"(_|-)({'|'.join(ENGINE_NAMES)})")
 
-def get_instance(container: str, database_containers: list[str]) -> str:
+
+def get_instance(container: str, database_containers: list[str]) -> str | None:
+    """The databases.csv instance a container serves, or None for no database.
+
+    A declared container is its own instance. Every other name is read against
+    ENGINE_NAMES: carrying one as a suffix makes the rest the instance, which
+    maps `<app>-database` from compose and `<app>_database.1.<task>` from swarm
+    onto the same one; being one outright makes the container its own instance,
+    the shape a compose file writes as `container_name: postgres`.
+
+    Args:
+        container: the running container's name.
+        database_containers: names passed via --database-containers, taken as
+            declared engines whatever they are called.
+
+    Returns:
+        The instance name, or None when the name neither carries nor is an
+        engine name: an application container is not an engine, even when it
+        ships the client tools that would let a dump command start.
+    """
     if container in database_containers:
         return container
-    return re.split(r"(_|-)(database|db|postgres)", container)[0]
+    parts = _SUFFIX_RE.split(container)
+    if len(parts) > 1:
+        return parts[0]
+    return container if container in ENGINE_NAMES else None
 
 
 def fallback_pg_dumpall(
     container: str, username: str, password: str, out_file: str
 ) -> None:
-    cmd = (
+    """
-        f"PGPASSWORD={password} docker exec -i {container} "
+    Perform a full Postgres cluster dump using pg_dumpall.
-        f"pg_dumpall -U {username} -h localhost > {out_file}"
+    """
+    execute_to_file(
+        docker_exec_argv(
+            container,
+            ["pg_dumpall", "-U", username, "-h", "localhost"],
+            interactive=True,
+            forward_env=["PGPASSWORD"],
+        ),
+        out_file,
+        env={"PGPASSWORD": password},
     )
-    execute_shell_command(cmd)
 
 
 def backup_database(
@@ -33,50 +70,107 @@ def backup_database(
     container: str,
     volume_dir: str,
     db_type: str,
-    databases_df: "pandas.DataFrame",
+    dump_tool: str,
+    databases_df: pd.DataFrame,
     database_containers: list[str],
-) -> None:
+) -> bool:
+    """
+    Backup databases for a given DB container.
+
+    Args:
+        dump_tool: the MariaDB client found in the container, so an image
+            that ships only mysqldump is dumped with the tool it has.
+
+    Returns True if at least one dump was produced.
+    """
     instance_name = get_instance(container, database_containers)
-    entries = databases_df.loc[databases_df["instance"] == instance_name]
+    if instance_name is None:
+        log.debug("Container '%s' carries no database token", container)
+        return False
+
+    entries = databases_df[databases_df["instance"] == instance_name]
     if entries.empty:
-        log.warning("No entry found for instance '%s'", instance_name)
+        log.debug("No database entries for instance '%s'", instance_name)
-        return
+        return False
 
-    out_dir = os.path.join(volume_dir, "sql")
+    out_dir = pathlib.Path(volume_dir) / SQL_DIR
-    pathlib.Path(out_dir).mkdir(parents=True, exist_ok=True)
+    out_dir.mkdir(parents=True, exist_ok=True)
 
-    for row in entries.iloc:
+    produced = False
-        db_name = row["database"]
-        user = row["username"]
-        password = row["password"]
 
-        dump_file = os.path.join(out_dir, f"{db_name}.backup.sql")
+    for row in entries.itertuples(index=False):
+        raw_db = getattr(row, "database", "")
+        user = (getattr(row, "username", "") or "").strip()
+        password = (getattr(row, "password", "") or "").strip()
+
+        db_value = validate_database(raw_db, instance=instance_name)
+
+        if db_value == CLUSTER_ROW:
+            if db_type != "postgres":
+                raise ValueError(
+                    f"databases.csv entry for instance '{instance_name}': "
+                    f"'{CLUSTER_ROW}' is currently only supported for Postgres."
+                )
+
+            cluster_file = str(out_dir / f"{instance_name}{CLUSTER_SUFFIX}")
+            fallback_pg_dumpall(container, user, password, cluster_file)
+            produced = True
+            continue
+
+        db_name = db_value
+        dump_file = str(out_dir / f"{db_name}{DUMP_SUFFIX}")
 
         if db_type == "mariadb":
-            cmd = (
+            # Force TCP so auth matches '<user>'@'%' instead of socket -> 'localhost'.
-                f"docker exec {container} /usr/bin/mariadb-dump "
+            execute_to_file(
-                f"-u {user} -p{password} {db_name} > {dump_file}"
+                docker_exec_argv(
+                    container,
+                    [
+                        dump_tool,
+                        "-h",
+                        "127.0.0.1",
+                        "--protocol=tcp",
+                        "-u",
+                        user,
+                        f"-p{password}",
+                        db_name,
+                    ],
+                ),
+                dump_file,
             )
-            execute_shell_command(cmd)
+            produced = True
             continue
 
         if db_type == "postgres":
-            cluster_file = os.path.join(out_dir, f"{instance_name}.cluster.backup.sql")
-            if not db_name:
-                fallback_pg_dumpall(container, user, password, cluster_file)
-                return
-
             try:
-                cmd = (
+                execute_to_file(
-                    f"PGPASSWORD={password} docker exec -i {container} "
+                    docker_exec_argv(
-                    f"pg_dump -U {user} -d {db_name} -h localhost > {dump_file}"
+                        container,
+                        [
+                            "pg_dump",
+                            "-U",
+                            user,
+                            "-d",
+                            db_name,
+                            "-h",
+                            "localhost",
+                            "--no-owner",
+                            "--no-privileges",
+                        ],
+                        interactive=True,
+                        forward_env=["PGPASSWORD"],
+                    ),
+                    dump_file,
+                    env={"PGPASSWORD": password},
                 )
-                execute_shell_command(cmd)
+                produced = True
-            except BackupException as e:
+            except BackupError as e:
-                print(f"pg_dump failed: {e}", flush=True)
+                raise BackupError(
-                print(
+                    f"Postgres dump failed for instance '{instance_name}', "
-                    f"Falling back to pg_dumpall for instance '{instance_name}'",
+                    f"database '{db_name}'. This database was explicitly configured "
-                    flush=True,
+                    "and therefore must succeed.\n"
-                )
+                    f"{e}"
-                fallback_pg_dumpall(container, user, password, cluster_file)
+                ) from e
             continue
+
+    return produced

src/baudolo/backup/docker.py

@@ -1,45 +1,143 @@
 from __future__ import annotations
 
-from .shell import execute_shell_command
+from typing import TYPE_CHECKING
+
+from .shell import BackupError, execute_shell_command
+
+if TYPE_CHECKING:
+    from collections.abc import Sequence
+
+
+def docker_exec_argv(
+    container: str,
+    argv: Sequence[str],
+    *,
+    interactive: bool = False,
+    forward_env: Sequence[str] = (),
+) -> list[str]:
+    """The argv that runs *argv* inside *container*.
+
+    Args:
+        container: the container to run in.
+        argv: the command, already split.
+        interactive: keep stdin open, for a command that is fed a dump.
+        forward_env: names of environment variables to hand to the container.
+            Passed as bare ``-e NAME``, so docker copies the value out of this
+            process's own environment; spelling ``-e NAME=value`` instead would
+            publish a secret in the host's process list.
+
+    Returns:
+        The argv list.
+    """
+    forwarded = [arg for name in forward_env for arg in ("-e", name)]
+    return [
+        "docker",
+        "exec",
+        *(["-i"] if interactive else []),
+        *forwarded,
+        container,
+        *argv,
+    ]
 
 
 def get_image_info(container: str) -> str:
     return execute_shell_command(
-        f"docker inspect --format '{{{{.Config.Image}}}}' {container}"
+        ["docker", "inspect", "--format", "{{.Config.Image}}", container]
     )[0]
 
 
-def has_image(container: str, pattern: str) -> bool:
+def image_id(container: str) -> str:
-    """Return True if container's image contains the pattern."""
+    """The container's image ID, identical for every replica of one image."""
-    return pattern in get_image_info(container)
+    return execute_shell_command(
+        ["docker", "inspect", "--format", "{{.Image}}", container]
+    )[0].strip()
+
+
+def has_tool(container: str, tool: str) -> bool:
+    """Whether *tool* runs inside the container.
+
+    Executes the binary rather than asking a shell for it: a distroless image
+    has no shell, and `sh -c 'command -v'` would answer "absent" for every
+    tool it ships.
+    """
+    try:
+        execute_shell_command(docker_exec_argv(container, [tool, "--version"]))
+    except BackupError:
+        return False
+    return True
 
 
 def docker_volume_names() -> list[str]:
-    return execute_shell_command("docker volume ls --format '{{.Name}}'")
+    return execute_shell_command(["docker", "volume", "ls", "--format", "{{.Name}}"])
 
 
 def containers_using_volume(volume_name: str) -> list[str]:
     return execute_shell_command(
-        f"docker ps --filter volume=\"{volume_name}\" --format '{{{{.Names}}}}'"
+        [
+            "docker",
+            "ps",
+            "--filter",
+            f"volume={volume_name}",
+            "--format",
+            "{{.Names}}",
+        ]
     )
 
 
+def is_swarm_task(container: str) -> bool:
+    """Swarm-managed task containers must never be stopped or started
+    manually: the orchestrator replaces the stopped task and a later
+    `docker start` fails on the detached overlay network. A container that
+    vanished between listing and inspect (--rm one-shots, task-history GC)
+    counts as not stoppable instead of aborting the whole backup run; if the
+    container still exists the inspect failure re-raises, so a broken daemon
+    keeps failing the run loudly instead of silently skipping the stop."""
+    try:
+        out = execute_shell_command(
+            [
+                "docker",
+                "inspect",
+                "--format",
+                '{{index .Config.Labels "com.docker.swarm.task.id"}}',
+                container,
+            ]
+        )
+    except BackupError:
+        still_listed = execute_shell_command(
+            [
+                "docker",
+                "ps",
+                "-a",
+                "--filter",
+                f"name=^{container}$",
+                "--format",
+                "{{.Names}}",
+            ]
+        )
+        if still_listed and still_listed[0].strip():
+            raise
+        return True
+    return bool(out and out[0].strip())
+
+
+def filter_stoppable(containers: list[str]) -> list[str]:
+    """Containers baudolo may stop/start itself (everything but swarm tasks)."""
+    stoppable = []
+    for container in containers:
+        if is_swarm_task(container):
+            print(
+                f"Skipping stop/start for swarm task container '{container}'.",
+                flush=True,
+            )
+            continue
+        stoppable.append(container)
+    return stoppable
+
+
 def change_containers_status(containers: list[str], status: str) -> None:
     """Stop or start a list of containers."""
     if not containers:
         print(f"No containers to {status}.", flush=True)
         return
-    names = " ".join(containers)
+    print(f"{status.capitalize()} containers: {' '.join(containers)}...", flush=True)
-    print(f"{status.capitalize()} containers: {names}...", flush=True)
+    execute_shell_command(["docker", status, *containers])
-    execute_shell_command(f"docker {status} {names}")
-
-
-def docker_volume_exists(volume: str) -> bool:
-    # Avoid throwing exceptions for exists checks.
-    try:
-        execute_shell_command(
-            f"docker volume inspect {volume} >/dev/null 2>&1 && echo OK"
-        )
-        return True
-    except Exception:
-        return False

src/baudolo/backup/dumps.py

@@ -0,0 +1,155 @@
+"""Database dumps taken before a volume's files are copied."""
+
+from __future__ import annotations
+
+import sys
+from typing import NamedTuple
+
+import pandas as pd
+from pandas.errors import EmptyDataError
+
+from baudolo.databases import COLUMNS, DELIMITER
+
+from .db import backup_database, get_instance
+from .docker import has_tool, image_id
+
+DUMP_TOOLS: tuple[tuple[str, str], ...] = (
+    ("postgres", "pg_dumpall"),
+    ("mariadb", "mariadb-dump"),
+    ("mariadb", "mysqldump"),
+)
+
+_ENGINE_BY_IMAGE: dict[str, tuple[str, str] | None] = {}
+
+
+class VolumeOutcome(NamedTuple):
+    """What a dump attempt established about one volume.
+
+    ``database`` says a container serving the volume speaks an engine this
+    tool can dump; ``dumped`` says a dump was actually written. ``engine`` is
+    the engine that was detected, or None when none was.
+    """
+
+    database: bool
+    dumped: bool
+    engine: str | None = None
+
+
+def container_engine(container: str) -> tuple[str, str] | None:
+    """The (engine, dump tool) a container can serve, or None for neither.
+
+    Asks the container what it can run instead of reading its image name. A
+    dedicated Postgres is tagged `<app>-database` or `postgis/postgis` and
+    carries no engine token at all, while a swarm registry host such as
+    `svc-db-mariadb-swarm-mgr-01:5000` carries the wrong one.
+
+    Args:
+        container: must be running - `docker exec` is the probe, and a
+            stopped container would be cached as "no engine" for its whole
+            image. The only caller feeds it `docker ps` output.
+
+    Returns:
+        The engine and the tool that dumps it, cached per image ID so that
+        replicas of one image are probed once.
+    """
+    image = image_id(container)
+    if image not in _ENGINE_BY_IMAGE:
+        _ENGINE_BY_IMAGE[image] = next(
+            (
+                (engine, tool)
+                for engine, tool in DUMP_TOOLS
+                if has_tool(container, tool)
+            ),
+            None,
+        )
+    return _ENGINE_BY_IMAGE[image]
+
+
+def backup_mariadb_or_postgres(
+    *,
+    container: str,
+    volume_dir: str,
+    databases_df: pd.DataFrame,
+    database_containers: list[str],
+) -> VolumeOutcome:
+    """What this container contributes to its volume's outcome."""
+    engine = container_engine(container)
+    if engine is None:
+        return VolumeOutcome(database=False, dumped=False)
+    if get_instance(container, database_containers) is None:
+        return VolumeOutcome(database=False, dumped=False)
+    db_type, dump_tool = engine
+    dumped = backup_database(
+        container=container,
+        volume_dir=volume_dir,
+        db_type=db_type,
+        dump_tool=dump_tool,
+        databases_df=databases_df,
+        database_containers=database_containers,
+    )
+    return VolumeOutcome(database=True, dumped=dumped, engine=db_type)
+
+
+def _empty_databases_df() -> pd.DataFrame:
+    """
+    Create an empty DataFrame with the expected schema for databases.csv.
+
+    This allows the backup to continue without DB dumps when the CSV is missing
+    or empty (pandas EmptyDataError).
+    """
+    return pd.DataFrame(columns=list(COLUMNS))
+
+
+def load_databases_df(csv_path: str) -> pd.DataFrame:
+    """
+    Load databases.csv robustly.
+
+    - Missing file     -> warn, continue with empty df
+    - Empty file       -> warn, continue with empty df
+    - Valid CSV        -> return dataframe
+    """
+    try:
+        return pd.read_csv(csv_path, sep=DELIMITER, keep_default_na=False, dtype=str)
+    except FileNotFoundError:
+        print(
+            f"WARNING: databases.csv not found: {csv_path}. Continuing without database dumps.",
+            file=sys.stderr,
+            flush=True,
+        )
+        return _empty_databases_df()
+    except EmptyDataError:
+        print(
+            f"WARNING: databases.csv exists but is empty: {csv_path}. Continuing without database dumps.",
+            file=sys.stderr,
+            flush=True,
+        )
+        return _empty_databases_df()
+
+
+def backup_dumps_for_volume(
+    *,
+    containers: list[str],
+    vol_dir: str,
+    databases_df: pd.DataFrame,
+    database_containers: list[str],
+) -> VolumeOutcome:
+    """The volume's outcome across every container that mounts it."""
+    found_db = False
+    dumped_any = False
+    engine: str | None = None
+
+    for c in containers:
+        outcome = backup_mariadb_or_postgres(
+            container=c,
+            volume_dir=vol_dir,
+            databases_df=databases_df,
+            database_containers=database_containers,
+        )
+        if outcome.database:
+            found_db = True
+        if outcome.dumped:
+            dumped_any = True
+        if engine is None:
+            engine = outcome.engine
+
+    return VolumeOutcome(database=found_db, dumped=dumped_any, engine=engine)

src/baudolo/backup/layout.py

@@ -0,0 +1,61 @@
+"""Where a backup run puts its files, and how a finished run is stamped."""
+
+from __future__ import annotations
+
+import json
+import pathlib
+
+from dirval import create_stamp_file
+
+from baudolo.generation import MANIFEST_FILE, manifest_document
+
+from .shell import BackupError, execute_shell_command
+
+
+def get_machine_id() -> str:
+    return execute_shell_command(["sha256sum", "/etc/machine-id"])[0][0:64]
+
+
+def stamp_directory(version_dir: str) -> None:
+    """
+    Use dirval as a Python library to stamp the directory (no CLI dependency).
+    """
+    create_stamp_file(version_dir)
+
+
+def create_version_directory(versions_dir: str, backup_time: str) -> str:
+    version_dir = str(pathlib.Path(versions_dir) / backup_time)
+    try:
+        pathlib.Path(version_dir).mkdir(parents=True)
+    except FileExistsError:
+        raise BackupError(
+            f"generation {backup_time} already exists at {version_dir}; "
+            "another run claimed this second - refusing to write into it, "
+            "since rsync --delete would overwrite that generation"
+        ) from None
+    return version_dir
+
+
+def create_volume_directory(version_dir: str, volume_name: str) -> str:
+    path = pathlib.Path(version_dir) / volume_name
+    path.mkdir(parents=True, exist_ok=True)
+    return str(path)
+
+
+def write_manifest(version_dir: str, volumes: dict[str, dict[str, bool]]) -> str:
+    """Record the generation's layout and per-volume outcome.
+
+    Written before the directory is stamped, so the stamp covers it.
+
+    Args:
+        version_dir: the generation directory.
+        volumes: per volume name, ``database`` and ``dumped``.
+
+    Returns:
+        The path written.
+    """
+    path = pathlib.Path(version_dir) / MANIFEST_FILE
+    with path.open("w", encoding="utf-8") as handle:
+        json.dump(manifest_document(volumes), handle, indent=2, sort_keys=True)
+        handle.write("\n")
+    return str(path)

src/baudolo/backup/policy.py

@@ -0,0 +1,38 @@
+"""Which volumes are backed up, and which containers must stop for it."""
+
+from __future__ import annotations
+
+from .docker import get_image_info, is_swarm_task
+
+
+def is_image_ignored(container: str, images_no_backup_required: list[str]) -> bool:
+    if not images_no_backup_required:
+        return False
+    img = get_image_info(container)
+    return img in images_no_backup_required
+
+
+def volume_is_fully_ignored(
+    containers: list[str], images_no_backup_required: list[str]
+) -> bool:
+    """
+    Skip file backup only if all containers linked to the volume are ignored.
+    """
+    if not containers:
+        return False
+    return all(is_image_ignored(c, images_no_backup_required) for c in containers)
+
+
+def requires_stop(containers: list[str], images_no_stop_required: list[str]) -> bool:
+    """
+    Stop is required if ANY stoppable container image is NOT in the exact
+    image whitelist. Swarm task containers never count: baudolo must
+    not cycle them (see docker.is_swarm_task).
+    """
+    for c in containers:
+        if is_swarm_task(c):
+            continue
+        img = get_image_info(c)
+        if img not in images_no_stop_required:
+            return True
+    return False

src/baudolo/backup/shell.py

@@ -1,26 +1,75 @@
+"""Running external commands without a shell.
+
+Every command is an argv list. A database name, a password or a container name
+therefore cannot close a quote and start a second command, which a formatted
+string handed to ``shell=True`` allowed.
+"""
+
 from __future__ import annotations
 
+import os
 import subprocess
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from collections.abc import Mapping, Sequence
 
 
-class BackupException(Exception):
+class BackupError(Exception):
     """Generic exception for backup errors."""
 
 
-def execute_shell_command(command: str) -> list[str]:
+def _child_env(env: Mapping[str, str] | None) -> dict[str, str] | None:
-    """Execute a shell command and return its output lines."""
+    return None if env is None else {**os.environ, **env}
-    print(command, flush=True)
+
+
+def _fail(command: Sequence[str], returncode: int, out: bytes, err: bytes) -> None:
+    raise BackupError(
+        f"Error in command: {' '.join(command)}\n"
+        f"Output: {out}\nError: {err}\n"
+        f"Exit code: {returncode}"
+    )
+
+
+def execute_shell_command(
+    command: Sequence[str], *, env: Mapping[str, str] | None = None
+) -> list[str]:
+    """Run *command* and return its stdout lines.
+
+    Args:
+        command: argv, the program first.
+        env: variables added to the child's environment, for values that must
+            not appear in the argv of a process listing.
+    """
+    command = list(command)
+    print(" ".join(command), flush=True)
     process = subprocess.Popen(
-        [command],
+        command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=_child_env(env)
-        stdout=subprocess.PIPE,
-        stderr=subprocess.PIPE,
-        shell=True,
     )
     out, err = process.communicate()
     if process.returncode != 0:
-        raise BackupException(
+        _fail(command, process.returncode, out, err)
-            f"Error in command: {command}\n"
-            f"Output: {out}\nError: {err}\n"
-            f"Exit code: {process.returncode}"
-        )
     return [line.decode("utf-8") for line in out.splitlines()]
+
+
+def execute_to_file(
+    command: Sequence[str], out_file: str, *, env: Mapping[str, str] | None = None
+) -> None:
+    """Run *command*, writing its stdout to *out_file* only once it succeeded.
+
+    The output goes to a sibling temporary file first, so a partial or empty
+    stream from a failing dump never takes the place of a valid backup.
+    """
+    command = list(command)
+    print(" ".join(command), flush=True)
+    tmp = Path(f"{out_file}.tmp")
+    with tmp.open("wb") as handle:
+        process = subprocess.Popen(
+            command, stdout=handle, stderr=subprocess.PIPE, env=_child_env(env)
+        )
+        _, err = process.communicate()
+    if process.returncode != 0:
+        tmp.unlink()
+        _fail(command, process.returncode, b"", err)
+    tmp.replace(out_file)

src/baudolo/backup/snapshot.py

@@ -0,0 +1,176 @@
+"""Capture every volume from one atomic filesystem snapshot.
+
+Copying a live tree file by file cannot produce a point in time: a database can
+write between two files and leave a control file and its write-ahead log
+disagreeing, which no recovery can repair. A snapshot freezes the whole subject
+at once, so a database reads it as a crash and replays its log - a case it is
+built for. That also removes the reason to stop containers at all.
+
+The snapshot kind is stated by the caller rather than probed, because falling
+back to a live copy when a probe is inconclusive would hand out backups that
+look consistent and are not.
+
+Which volumes a snapshot of the subject contains is a different question, and
+it is decided per volume: a volume with a backing store of its own appears
+inside the snapshot as an existing empty directory, so copying from there
+succeeds and stores nothing. Such a volume is copied live instead - correct
+data without the point in time - while every other volume of the same run
+keeps its snapshot.
+"""
+
+from __future__ import annotations
+
+import os
+from contextlib import contextmanager
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+from .shell import BackupError, execute_shell_command
+
+if TYPE_CHECKING:
+    from collections.abc import Callable, Iterator
+
+    from .volume import Backing
+
+KINDS = ("btrfs", "zfs")
+
+
+class SnapshotError(RuntimeError):
+    """A snapshot could not be created, resolved or removed."""
+
+
+def _resolver(subject: str, root: str) -> Callable[[str], str]:
+    def resolve(path: str) -> str:
+        # Exception: abspath, not Path.resolve() - resolve() follows symlinks,
+        # which would let a symlinked volume test as inside the subject.
+        relative = os.path.relpath(
+            os.path.abspath(path),  # noqa: PTH100
+            os.path.abspath(subject),  # noqa: PTH100
+        )
+        if relative.startswith(".."):
+            raise SnapshotError(f"{path} lies outside the snapshot subject {subject}")
+        resolved = root if relative == "." else str(Path(root) / relative)
+
+        # abspath drops a trailing separator, and rsync reads "dir/" as its
+        # contents where "dir" means the directory itself.
+        return resolved + os.sep if path.endswith(os.sep) else resolved
+
+    return resolve
+
+
+def _btrfs(
+    subject: str, name: str, run: Callable[[list[str]], list[str]]
+) -> tuple[str, list[str]]:
+    # The snapshot goes inside the subject, never beside it: the kernel rejects
+    # a snapshot whose destination is on another filesystem, which is exactly
+    # what the parent directory is when the subject is a mountpoint of its own.
+    target = str(Path(os.path.abspath(subject)) / f".{name}")  # noqa: PTH100 - see _resolver
+    run(["btrfs", "subvolume", "snapshot", "-r", subject, target])
+    return target, ["btrfs", "subvolume", "delete", target]
+
+
+def _zfs(
+    subject: str, name: str, run: Callable[[list[str]], list[str]]
+) -> tuple[str, list[str]]:
+    output = run(["zfs", "list", "-H", "-o", "name", subject])
+    dataset = (output[0] if output else "").strip()
+    if not dataset:
+        raise SnapshotError(f"no zfs dataset is mounted at {subject}")
+    run(["zfs", "snapshot", f"{dataset}@{name}"])
+    root = str(Path(subject) / ".zfs" / "snapshot" / name)
+    return root, ["zfs", "destroy", f"{dataset}@{name}"]
+
+
+_CREATE = {"btrfs": _btrfs, "zfs": _zfs}
+
+
+def unsnapshotted(backing: Backing, subject: str) -> str | None:
+    """Return why a snapshot of ``subject`` does not hold this volume's data.
+
+    Docker mounts a volume's own backing store lazily and unmounts it when the
+    last consumer stops, so the declaration is what gets checked: it is true at
+    every moment, where the mount table is only true while a container happens
+    to hold the volume.
+
+    Args:
+        backing: the volume as the daemon describes it.
+        subject: the snapshot subject, e.g. ``/var/lib/docker``.
+
+    Returns:
+        The reason, or None when the snapshot holds the volume.
+    """
+    if backing.driver != "local":
+        return f"it uses the {backing.driver} driver"
+    if backing.options:
+        return f"it declares its own backing store {backing.options}"
+    if not backing.mountpoint:
+        return "it reports no mountpoint"
+    real = os.path.realpath(backing.mountpoint)
+    if os.path.ismount(real):
+        return f"its mountpoint {backing.mountpoint} sits on its own mount"
+    try:
+        crosses = (
+            Path(real).stat().st_dev != Path(os.path.realpath(subject)).stat().st_dev
+        )
+    except OSError as error:
+        return f"its mountpoint {backing.mountpoint} could not be read: {error}"
+    if crosses:
+        return f"its mountpoint {backing.mountpoint} crosses a filesystem boundary"
+    return None
+
+
+def snapshot_source(
+    resolve: Callable[[str], str], backing: Backing, subject: str
+) -> tuple[str | None, str]:
+    """Resolve where to read a volume from, and why if not from the snapshot.
+
+    Returns:
+        ``(path, "")`` to copy from the snapshot, or ``(None, reason)`` to copy
+        it live.
+    """
+    reason = unsnapshotted(backing, subject)
+    if reason:
+        return None, reason
+    try:
+        source = resolve(backing.source)
+    except SnapshotError as error:
+        return None, str(error)
+    if not Path(source).is_dir():
+        return None, "it was created after the snapshot was taken"
+    return source, ""
+
+
+@contextmanager
+def volume_snapshot(
+    kind: str,
+    subject: str,
+    tag: str,
+    run: Callable[[list[str]], list[str]] = execute_shell_command,
+) -> Iterator[Callable[[str], str]]:
+    """Yield a resolver mapping a path under ``subject`` into a snapshot of it.
+
+    Args:
+        kind: ``btrfs`` or ``zfs``; the caller states it, nothing is probed.
+        subject: the btrfs subvolume or zfs dataset mountpoint holding the
+            volumes, e.g. ``/var/lib/docker``.
+        tag: unique suffix for the snapshot name, e.g. the backup timestamp.
+        run: shell runner, injected so the mechanics are testable.
+
+    Raises:
+        SnapshotError: the kind is unknown, or the snapshot cannot be created.
+            Removal failure is reported, not raised: a leftover snapshot is a
+            cleanup problem and must not discard a generation that is complete.
+    """
+    create = _CREATE.get(kind)
+    if create is None:
+        raise SnapshotError(f"unknown snapshot kind {kind!r}; expected one of {KINDS}")
+
+    root, remove = create(subject, f"baudolo-{tag}", run)
+    try:
+        yield _resolver(subject, root)
+    finally:
+        try:
+            run(remove)
+        except BackupError as error:
+            # Raising here would also mask whatever the body raised.
+            print(f"WARNING: {root} could not be removed: {error}", flush=True)

src/baudolo/backup/volume.py

@@ -1,16 +1,45 @@
 from __future__ import annotations
 
+import json
 import os
 import pathlib
+from dataclasses import dataclass, field
 
-from .shell import BackupException, execute_shell_command
+from baudolo.generation import FILES_DIR
+
+from .shell import BackupError, execute_shell_command
 
 
-def get_storage_path(volume_name: str) -> str:
+@dataclass(frozen=True)
-    path = execute_shell_command(
+class Backing:
-        f"docker volume inspect --format '{{{{ .Mountpoint }}}}' {volume_name}"
+    """Where a docker volume actually keeps its data.
+
+    Args:
+        mountpoint: the path the daemon reports.
+        driver: the volume driver, ``local`` for the built-in one.
+        options: the driver options; a non-empty map means the mountpoint is a
+            mount target rather than the storage itself.
+    """
+
+    mountpoint: str
+    driver: str = "local"
+    options: dict = field(default_factory=dict)
+
+    @property
+    def source(self) -> str:
+        return f"{self.mountpoint}/"
+
+
+def inspect_backing(volume_name: str) -> Backing:
+    reported = execute_shell_command(
+        ["docker", "volume", "inspect", "--format", "{{json .}}", volume_name]
     )[0]
-    return f"{path}/"
+    data = json.loads(reported)
+    return Backing(
+        data.get("Mountpoint") or "",
+        data.get("Driver") or "",
+        data.get("Options") or {},
+    )
 
 
 def get_last_backup_dir(
@@ -18,26 +47,44 @@ def get_last_backup_dir(
 ) -> str | None:
     versions = sorted(os.listdir(versions_dir), reverse=True)
     for version in versions:
-        candidate = os.path.join(versions_dir, version, volume_name, "files", "")
+        candidate = f"{pathlib.Path(versions_dir) / version / volume_name / FILES_DIR}/"
-        if candidate != current_backup_dir and os.path.isdir(candidate):
+        if candidate != current_backup_dir and pathlib.Path(candidate).is_dir():
             return candidate
     return None
 
 
-def backup_volume(versions_dir: str, volume_name: str, volume_dir: str) -> None:
+def backup_volume(
-    """Perform incremental file backup of a Docker volume."""
+    versions_dir: str,
-    dest = os.path.join(volume_dir, "files") + "/"
+    volume_name: str,
+    volume_dir: str,
+    *,
+    authoritative: bool,
+    source: str,
+) -> None:
+    """Perform incremental file backup of a Docker volume.
+
+    Args:
+        authoritative: compare source and destination by content instead of by
+            size and whole-second mtime. Required on a pass whose destination was
+            already written from a live source, where a file can differ while
+            both attributes still agree.
+        source: directory to read from - the volume's mountpoint, or its path
+            inside a snapshot.
+    """
+    dest = f"{pathlib.Path(volume_dir) / FILES_DIR}/"
     pathlib.Path(dest).mkdir(parents=True, exist_ok=True)
 
     last = get_last_backup_dir(versions_dir, volume_name, dest)
-    link_dest = f"--link-dest='{last}'" if last else ""
+    cmd = ["rsync", "-aP", "--no-D", "--delete", "--delete-excluded"]
-    source = get_storage_path(volume_name)
+    if authoritative:
-
+        cmd.append("--checksum")
-    cmd = f"rsync -abP --delete --delete-excluded {link_dest} {source} {dest}"
+    if last:
+        cmd.append(f"--link-dest={last}")
+    cmd += [source, dest]
 
     try:
         execute_shell_command(cmd)
-    except BackupException as e:
+    except BackupError as e:
         if "file has vanished" in str(e):
             print(
                 "Warning: Some files vanished before transfer. Continuing.", flush=True

src/baudolo/databases.py

@@ -0,0 +1,103 @@
+"""The databases.csv contract: its columns, its delimiter, and what a row means.
+
+``baudolo-seed`` writes the file, the backup reads it to learn which dumps to
+take, and a restore consumer reads it again to replay them. Stating the schema
+once keeps a column or a convention added here from being invisible to the
+other two.
+
+Field values are handed back exactly as they stand in the file. A password may
+legitimately begin or end with a space, so stripping belongs to the caller that
+compares, never to the reader.
+"""
+
+from __future__ import annotations
+
+import csv
+import re
+from pathlib import Path
+from typing import NamedTuple
+
+COLUMNS = ("instance", "database", "username", "password")
+DELIMITER = ";"
+CLUSTER_ROW = "*"
+
+_NAME_RE = re.compile(r"^[a-zA-Z0-9_][a-zA-Z0-9_-]*$")
+
+
+class DatabasesCsvError(ValueError):
+    """A row does not match the contract."""
+
+
+class Row(NamedTuple):
+    """One databases.csv row, verbatim.
+
+    ``database`` holds :data:`CLUSTER_ROW` when the whole instance is dumped.
+    """
+
+    instance: str
+    database: str
+    username: str
+    password: str
+
+    @property
+    def is_cluster(self) -> bool:
+        return self.database.strip() == CLUSTER_ROW
+
+
+def validate_database(value: str | None, *, instance: str) -> str:
+    """The database column of one row, or raise.
+
+    The name reaches a shell as part of the dump command, so it is checked
+    where it is read as well as where it is written: a file edited by hand
+    never passed the seed.
+
+    Args:
+        value: the raw column.
+        instance: named in the error, so a bad row can be found.
+
+    Raises:
+        DatabasesCsvError: the column is empty, literally ``nan``, or holds
+            anything but letters, numbers, ``_`` and ``-``.
+    """
+    text = (value or "").strip()
+    if not text:
+        raise DatabasesCsvError(
+            f"Invalid databases.csv entry for instance '{instance}': column "
+            f"'database' must be '{CLUSTER_ROW}' or a concrete database name "
+            "(not empty)."
+        )
+    if text == CLUSTER_ROW:
+        return CLUSTER_ROW
+    if text.lower() == "nan":
+        raise DatabasesCsvError(
+            f"Invalid databases.csv entry for instance '{instance}': "
+            "database must not be 'nan'."
+        )
+    if not _NAME_RE.match(text):
+        raise DatabasesCsvError(
+            f"Invalid databases.csv entry for instance '{instance}': invalid "
+            f"database name '{text}'. Allowed: letters, numbers, '_' and '-'."
+        )
+    return text
+
+
+def read_rows(csv_path: str) -> list[Row]:
+    """Every row of the file in file order, header skipped, blank rows dropped.
+
+    Raises:
+        DatabasesCsvError: a row holds fewer columns than :data:`COLUMNS`.
+    """
+    rows: list[Row] = []
+    with Path(csv_path).open(newline="", encoding="utf-8") as handle:
+        reader = csv.reader(handle, delimiter=DELIMITER)
+        next(reader, None)
+        for raw in reader:
+            if not any(field.strip() for field in raw):
+                continue
+            if len(raw) < len(COLUMNS):
+                raise DatabasesCsvError(
+                    f"{csv_path} has a row with {len(raw)} column(s), "
+                    f"expected {len(COLUMNS)}"
+                )
+            rows.append(Row(*raw[: len(COLUMNS)]))
+    return rows

src/baudolo/generation.py

@@ -0,0 +1,54 @@
+"""The on-disk shape of a generation, and the manifest that states it.
+
+Every name a reader needs to find payload in a generation is declared here
+once, and written into each generation's own manifest. A consumer therefore
+never has to hardcode the layout or match this package's version: it reads
+what the run that produced the tree recorded.
+
+The manifest also carries what only the run itself can know: per volume,
+``database`` (it held one), ``dumped`` (a dump was produced for it) and
+``engine`` (which one was detected). Both flags true is a replayable dump;
+``database`` without ``dumped`` is a raw copy of live engine files.
+
+Kept import-free: consumers read the manifest with nothing but ``json``, on
+hosts that do not have this package installed.
+"""
+
+from __future__ import annotations
+
+FILES_DIR = "files"
+SQL_DIR = "sql"
+DUMP_SUFFIX = ".backup.sql"
+CLUSTER_SUFFIX = ".cluster.backup.sql"
+
+MANIFEST_FILE = "manifest.json"
+MANIFEST_SCHEMA = 1
+
+
+def manifest_document(volumes: dict[str, object]) -> dict[str, object]:
+    """The manifest a finished run writes.
+
+    Args:
+        volumes: per volume name, an object carrying ``database``, ``dumped``
+            and ``engine`` -- a ``baudolo.backup.dumps.VolumeOutcome``.
+
+    Returns:
+        The document, ready for ``json.dump``.
+    """
+    return {
+        "schema": MANIFEST_SCHEMA,
+        "layout": {
+            "files_dir": FILES_DIR,
+            "sql_dir": SQL_DIR,
+            "dump_suffix": DUMP_SUFFIX,
+            "cluster_suffix": CLUSTER_SUFFIX,
+        },
+        "volumes": {
+            name: {
+                "database": bool(outcome.database),
+                "dumped": bool(outcome.dumped),
+                "engine": outcome.engine,
+            }
+            for name, outcome in sorted(volumes.items())
+        },
+    }

src/baudolo/restore/__main__.py

@@ -3,10 +3,11 @@ from __future__ import annotations
 import argparse
 import sys
 
-from .paths import BackupPaths
+from .db.cluster import restore_cluster_sql
-from .files import restore_volume_files
-from .db.postgres import restore_postgres_sql
 from .db.mariadb import restore_mariadb_sql
+from .db.postgres import restore_postgres_sql
+from .files import restore_volume_files
+from .paths import BackupPaths
 
 
 def _add_common_backup_args(p: argparse.ArgumentParser) -> None:
@@ -21,8 +22,22 @@ def _add_common_backup_args(p: argparse.ArgumentParser) -> None:
     )
     p.add_argument(
         "--repo-name",
-        default="backup-docker-to-local",
+        required=True,
-        help="Backup repo folder name under <backups-dir>/<hash>/ (default: backup-docker-to-local)",
+        help="Backup repo folder name under <backups-dir>/<hash>/",
+    )
+
+
+def _add_common_engine_args(p: argparse.ArgumentParser) -> None:
+    p.add_argument("--container", required=True)
+    p.add_argument("--db-password", required=True)
+    p.add_argument("--empty", action="store_true")
+    p.add_argument(
+        "--no-version-check",
+        action="store_true",
+        help=(
+            "Replay even if the dump comes from a newer engine than the target. "
+            "With --empty this can leave an emptied database behind."
+        ),
     )
 
 
@@ -33,15 +48,8 @@ def main(argv: list[str] | None = None) -> int:
     )
     sub = parser.add_subparsers(dest="cmd", required=True)
 
-    # ------------------------------------------------------------------
-    # files
-    # ------------------------------------------------------------------
     p_files = sub.add_parser("files", help="Restore files into a docker volume")
     _add_common_backup_args(p_files)
-    p_files.add_argument(
-        "--rsync-image",
-        default="ghcr.io/kevinveenbirkenbach/alpine-rsync",
-    )
     p_files.add_argument(
         "--source-volume",
         default=None,
@@ -52,36 +60,40 @@ def main(argv: list[str] | None = None) -> int:
         ),
     )
 
-    # ------------------------------------------------------------------
-    # postgres
-    # ------------------------------------------------------------------
     p_pg = sub.add_parser("postgres", help="Restore a single PostgreSQL database dump")
     _add_common_backup_args(p_pg)
-    p_pg.add_argument("--container", required=True)
+    _add_common_engine_args(p_pg)
     p_pg.add_argument("--db-name", required=True)
     p_pg.add_argument("--db-user", default=None, help="Defaults to db-name if omitted")
-    p_pg.add_argument("--db-password", required=True)
-    p_pg.add_argument("--empty", action="store_true")
 
-    # ------------------------------------------------------------------
+    p_cluster = sub.add_parser(
-    # mariadb
+        "cluster", help="Restore a full PostgreSQL cluster dump (pg_dumpall)"
-    # ------------------------------------------------------------------
+    )
+    _add_common_backup_args(p_cluster)
+    _add_common_engine_args(p_cluster)
+    p_cluster.add_argument(
+        "--instance",
+        required=True,
+        help="Instance the dump was taken from; names <instance>.cluster.backup.sql",
+    )
+    p_cluster.add_argument(
+        "--db-user",
+        required=True,
+        help="Superuser of the instance; the dump creates roles and databases",
+    )
+
     p_mdb = sub.add_parser(
         "mariadb", help="Restore a single MariaDB/MySQL-compatible dump"
     )
     _add_common_backup_args(p_mdb)
-    p_mdb.add_argument("--container", required=True)
+    _add_common_engine_args(p_mdb)
     p_mdb.add_argument("--db-name", required=True)
     p_mdb.add_argument("--db-user", default=None, help="Defaults to db-name if omitted")
-    p_mdb.add_argument("--db-password", required=True)
-    p_mdb.add_argument("--empty", action="store_true")
 
     args = parser.parse_args(argv)
 
     try:
         if args.cmd == "files":
-            # target volume = args.volume_name
-            # source volume (backup key) defaults to target volume
             source_volume = args.source_volume or args.volume_name
 
             bp_files = BackupPaths(
@@ -95,7 +107,6 @@ def main(argv: list[str] | None = None) -> int:
             return restore_volume_files(
                 args.volume_name,
                 bp_files.files_dir(),
-                rsync_image=args.rsync_image,
             )
 
         if args.cmd == "postgres":
@@ -113,6 +124,24 @@ def main(argv: list[str] | None = None) -> int:
                     backups_dir=args.backups_dir,
                 ).sql_file(args.db_name),
                 empty=args.empty,
+                check_version=not args.no_version_check,
+            )
+            return 0
+
+        if args.cmd == "cluster":
+            restore_cluster_sql(
+                container=args.container,
+                user=args.db_user,
+                password=args.db_password,
+                sql_path=BackupPaths(
+                    args.volume_name,
+                    args.backup_hash,
+                    args.version,
+                    repo_name=args.repo_name,
+                    backups_dir=args.backups_dir,
+                ).cluster_file(args.instance),
+                empty=args.empty,
+                check_version=not args.no_version_check,
             )
             return 0
 
@@ -131,13 +160,14 @@ def main(argv: list[str] | None = None) -> int:
                     backups_dir=args.backups_dir,
                 ).sql_file(args.db_name),
                 empty=args.empty,
+                check_version=not args.no_version_check,
             )
             return 0
 
         parser.error("Unhandled command")
-        return 2
+        return 2  # noqa: TRY300 - the try wraps the whole dispatch on purpose
 
-    except Exception as e:
+    except Exception as e:  # noqa: BLE001 - CLI boundary: any failure becomes exit 1
         print(f"ERROR: {e}", file=sys.stderr)
         return 1
 

src/baudolo/restore/db/cluster.py

@@ -0,0 +1,252 @@
+"""Replay a full PostgreSQL cluster dump produced by ``pg_dumpall``.
+
+The backup side writes one when a databases.csv row asks for every database of
+an instance (``database = '*'``, see ``backup/db.py``). Until now nothing read
+it back, so that dump was stored and unrestorable - a format whose producer has
+no consumer.
+
+A cluster stream differs from a single-database one in three ways that decide
+the implementation:
+
+* it recreates roles and databases, so it must be replayed against the control
+  database rather than into a target database;
+* ``CREATE DATABASE`` cannot run inside a transaction block, so unlike
+  :mod:`baudolo.restore.db.postgres` the replay must not be wrapped in
+  ``--single-transaction``;
+* it is replayed as a superuser, so the superuser-only statements that the
+  single-database path filters out are exactly the ones that have to survive.
+"""
+
+from __future__ import annotations
+
+import re
+import tempfile
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+from baudolo.restore.run import docker_exec
+
+from .version import guard
+
+if TYPE_CHECKING:
+    from collections.abc import Iterable, Iterator
+
+CONTROL_DB = "postgres"
+_CLUSTER_PRECLEAN_SQL = Path(__file__).parent / "cluster_preclean.sql"
+_CREATE_ROLE = re.compile(rb'^CREATE ROLE "?([^";]+)"?;\s*$')
+_CREATE_DATABASE = re.compile(rb"^CREATE DATABASE\s+(.*)$")
+_CREATE_ROLE_LINE = re.compile(rb"^CREATE ROLE\s+(.*)$")
+_CONNECT = re.compile(rb"^\\connect\s+(.*)$")
+_NO_ROWS = "SELECT ''::text WHERE false"
+
+
+def _first_identifier(rest: str) -> str | None:
+    """The first SQL identifier in *rest*, quoted or bare.
+
+    A quoted identifier may hold spaces and doubled quotes, so it cannot be
+    read with a character class that stops at whitespace - which is how a
+    database called ``odd name`` used to leave the inventory as ``odd``.
+    """
+    text = rest.strip()
+    if not text:
+        return None
+    if text.startswith('"'):
+        out = []
+        index = 1
+        while index < len(text):
+            char = text[index]
+            if char == '"':
+                if index + 1 < len(text) and text[index + 1] == '"':
+                    out.append('"')
+                    index += 2
+                    continue
+                return "".join(out)
+            out.append(char)
+            index += 1
+        return None
+    return re.split(r"[\s;(]", text, maxsplit=1)[0] or None
+
+
+def _connect_target(rest: str) -> str | None:
+    """The database a ``\\connect`` line switches to.
+
+    psql options precede the name (``\\connect -reuse-previous=on dbname=x``),
+    and the name may arrive as a ``dbname=`` assignment rather than bare.
+    """
+    for token in rest.strip().split():
+        if token.startswith("-"):
+            continue
+        if token.startswith("dbname="):
+            return _first_identifier(token[len("dbname=") :])
+        return _first_identifier(rest.strip()[rest.strip().index(token) :])
+    return None
+
+
+def dump_inventory(sql_path: str) -> tuple[list[str], list[str]]:
+    """The databases and roles a cluster dump recreates.
+
+    Args:
+        sql_path: the ``pg_dumpall`` stream.
+
+    Returns:
+        ``(databases, roles)``, each in the order the dump names them. The
+        pre-clean is scoped to these: everything else in the instance belongs
+        to no backup this restore holds, and dropping it would destroy data
+        the replay cannot bring back.
+    """
+    databases: list[str] = []
+    roles: list[str] = []
+    with Path(sql_path).open("rb") as handle:
+        for raw in handle:
+            line = raw.decode("utf-8", "replace")
+            for pattern, sink, read in (
+                (_CREATE_DATABASE, databases, _first_identifier),
+                (_CONNECT, databases, _connect_target),
+                (_CREATE_ROLE_LINE, roles, _first_identifier),
+            ):
+                found = pattern.match(raw)
+                if not found:
+                    continue
+                name = read(line[found.start(1) :])
+                if name and name not in sink:
+                    sink.append(name)
+    return databases, roles
+
+
+def preclean_sql() -> str:
+    """The catalog-wide pre-clean, safe only behind the instance check."""
+    with _CLUSTER_PRECLEAN_SQL.open(encoding="utf-8") as preclean:
+        return preclean.read()
+
+
+def instance_databases(container: str, user: str, docker_env: dict) -> list[str]:
+    """The instance's own databases, templates and control database aside."""
+    listed = docker_exec(
+        container,
+        [
+            "psql",
+            "-U",
+            user,
+            "-d",
+            CONTROL_DB,
+            "-tAc",
+            (
+                "SELECT datname FROM pg_database "
+                "WHERE NOT datistemplate AND datname <> current_database()"
+            ),
+        ],
+        capture=True,
+        docker_env=docker_env,
+    ).stdout
+    text = listed.decode() if isinstance(listed, bytes) else listed
+    return [name for name in text.split() if name]
+
+
+def assert_instance_matches_dump(
+    container: str, user: str, sql_path: str, docker_env: dict
+) -> None:
+    """Refuse ``--empty`` on an instance holding anything the dump lacks.
+
+    The pre-clean is a catalog-wide sweep, so a foreign database would be
+    destroyed with no way back. Scoping the sweep instead is not a fix: a
+    surviving database that owns or grants to one of the dump's roles pins
+    that role in pg_shdepend, and DROP ROLE then fails after the dump's own
+    databases are already gone.
+
+    Raises:
+        RuntimeError: the instance carries databases this dump cannot restore.
+    """
+    dumped, _roles = dump_inventory(sql_path)
+    present = instance_databases(container, user, docker_env)
+    foreign = sorted(set(present) - set(dumped))
+    if foreign:
+        raise RuntimeError(
+            f"{container} also holds {', '.join(foreign)}, which "
+            f"{Path(sql_path).name} does not carry. --empty wipes the "
+            "instance, so those would be destroyed with nothing to restore "
+            "them from. Move them off this instance, or drop them yourself if "
+            "they are disposable."
+        )
+
+
+def _psql(user: str) -> list[str]:
+    """The replay client: no --single-transaction, CREATE DATABASE forbids it."""
+    return ["psql", "-v", "ON_ERROR_STOP=1", "-U", user, "-d", CONTROL_DB]
+
+
+def filter_own_role_creation(lines: Iterable[bytes], user: str) -> Iterator[bytes]:
+    """Drop the ``CREATE ROLE`` of the role holding this session.
+
+    A pg_dumpall stream recreates every role of the cluster, the bootstrap
+    superuser included, and the pre-clean cannot drop the one it is connected
+    as - so that single statement always collides. Its ``ALTER ROLE`` is kept:
+    that is what re-applies the attributes and the password the dump captured.
+
+    Args:
+        lines: dump lines including their trailing newlines.
+        user: the connecting role.
+
+    Yields:
+        Every line except that one CREATE.
+    """
+    for line in lines:
+        found = _CREATE_ROLE.match(line)
+        if found and found.group(1).decode() == user:
+            continue
+        yield line
+
+
+def restore_cluster_sql(
+    *,
+    container: str,
+    user: str,
+    password: str,
+    sql_path: str,
+    empty: bool,
+    check_version: bool = True,
+) -> None:
+    """Replay a pg_dumpall stream into a running instance.
+
+    Args:
+        container: the running engine to replay into.
+        user: a superuser of that instance; the dump creates roles and
+            databases, which an application role may not do.
+        password: its password, handed to psql through the container's env.
+        sql_path: the ``<instance>.cluster.backup.sql`` of a generation.
+        empty: drop the cluster's databases and roles first. Without it the
+            replay stops at the first object that already exists, which is the
+            honest outcome: recreating a cluster over a populated one is a
+            decision, not a default.
+        check_version: refuse a dump from a newer major version than the
+            running engine before anything is dropped.
+    """
+    if not Path(sql_path).is_file():
+        raise FileNotFoundError(sql_path)
+
+    if check_version:
+        guard(
+            sql_path=sql_path,
+            engine="postgres",
+            container=container,
+            user=user,
+            password=password,
+        )
+
+    docker_env = {"PGPASSWORD": password}
+
+    if empty:
+        assert_instance_matches_dump(container, user, sql_path, docker_env)
+        docker_exec(
+            container,
+            _psql(user),
+            stdin=preclean_sql().encode(),
+            docker_env=docker_env,
+        )
+
+    with Path(sql_path).open("rb") as src, tempfile.TemporaryFile() as filtered:
+        for line in filter_own_role_creation(src, user):
+            filtered.write(line)
+        filtered.seek(0)
+        docker_exec(container, _psql(user), stdin=filtered, docker_env=docker_env)
+
+    print(f"PostgreSQL cluster restore complete from '{Path(sql_path).name}'.")

src/baudolo/restore/db/cluster_preclean.sql

@@ -0,0 +1,33 @@
+-- Pre-clean for `restore cluster --empty`. A pg_dumpall stream recreates roles
+-- and databases, so replaying it into a populated cluster dies on the first
+-- CREATE ROLE. Emitted as one DROP per row and run via \gexec so each executes
+-- as its own top-level statement: DROP DATABASE cannot run inside a
+-- transaction block, which rules out a single DO block.
+-- The phase column pins the order: databases must be gone before their owners
+-- can be dropped, and DROP OWNED BY releases what a role still holds in the
+-- control database. Template databases, the control database itself, the pg_*
+-- system roles and the connecting role are kept - the dump does not recreate
+-- them and dropping them would end the session.
+-- The sweep stays catalog-wide on purpose: a scoped one leaves databases that
+-- pin a dumped role in pg_shdepend, and phase 3 then fails after phase 1 has
+-- already dropped. assert_instance_matches_dump refuses before this runs.
+SELECT statement
+  FROM (
+    SELECT 1 AS phase,
+           format('DROP DATABASE IF EXISTS %I', datname) AS statement
+      FROM pg_database
+     WHERE NOT datistemplate
+       AND datname <> current_database()
+    UNION ALL
+    SELECT 2, format('DROP OWNED BY %I', rolname)
+      FROM pg_roles
+     WHERE NOT starts_with(rolname, 'pg_')
+       AND rolname <> current_user
+    UNION ALL
+    SELECT 3, format('DROP ROLE IF EXISTS %I', rolname)
+      FROM pg_roles
+     WHERE NOT starts_with(rolname, 'pg_')
+       AND rolname <> current_user
+  ) drops
+ ORDER BY phase
+\gexec

src/baudolo/restore/db/empty_preclean.sql

@@ -0,0 +1,64 @@
+-- Owner-filtered pre-clean for `restore --empty`. Emitted as one DROP per row and
+-- run via \gexec so each executes as its own top-level statement: a single DO-block
+-- would run every DROP in one transaction and exhaust max_locks_per_transaction on
+-- large schemas (e.g. gitlab). Also drops user-owned non-public schemas so a dump
+-- that CREATE SCHEMAs (e.g. discourse's discourse_functions) does not fail on an
+-- already-existing schema. Extension members (pg_trgm's set_limit) are
+-- superuser-owned; IF EXISTS absorbs the CASCADE fallout.
+SELECT format('DROP %s IF EXISTS public.%s CASCADE', obj.type, obj.name)
+  FROM (
+    SELECT format('%I', c.relname) AS name,
+           CASE c.relkind
+             WHEN 'v' THEN 'VIEW'
+             WHEN 'm' THEN 'MATERIALIZED VIEW'
+             WHEN 'f' THEN 'FOREIGN TABLE'
+             ELSE 'TABLE'
+           END AS type
+      FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace
+     WHERE n.nspname = 'public' AND c.relkind IN ('r', 'p', 'v', 'm', 'f')
+       AND pg_get_userbyid(c.relowner) = current_user
+    UNION ALL
+    -- Overloaded functions share a proname; DROP needs the identity
+    -- signature or psql aborts with "function name is not unique".
+    SELECT format('%I(%s)', p.proname, pg_get_function_identity_arguments(p.oid)) AS name,
+           CASE p.prokind WHEN 'p' THEN 'PROCEDURE' ELSE 'FUNCTION' END AS type
+      FROM pg_proc p JOIN pg_namespace n ON n.oid = p.pronamespace
+     WHERE n.nspname = 'public' AND p.prokind IN ('f', 'p', 'w')
+       AND pg_get_userbyid(p.proowner) = current_user
+    UNION ALL
+    SELECT format('%I', c.relname) AS name, 'SEQUENCE' AS type
+      FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace
+     WHERE n.nspname = 'public' AND c.relkind = 'S'
+       AND pg_get_userbyid(c.relowner) = current_user
+    UNION ALL
+    SELECT format('%I', t.typname) AS name, 'TYPE' AS type
+      FROM pg_type t JOIN pg_namespace n ON n.oid = t.typnamespace
+     WHERE n.nspname = 'public'
+       AND pg_get_userbyid(t.typowner) = current_user
+       AND (t.typtype IN ('e', 'd')
+            OR (t.typtype = 'c' AND EXISTS (
+                  SELECT 1 FROM pg_class c2
+                   WHERE c2.oid = t.typrelid AND c2.relkind = 'c')))
+    UNION ALL
+    SELECT format('%I', col.collname) AS name, 'COLLATION' AS type
+      FROM pg_collation col JOIN pg_namespace n ON n.oid = col.collnamespace
+     WHERE n.nspname = 'public'
+       AND pg_get_userbyid(col.collowner) = current_user
+    UNION ALL
+    SELECT format('%I', ts.cfgname) AS name, 'TEXT SEARCH CONFIGURATION' AS type
+      FROM pg_ts_config ts JOIN pg_namespace n ON n.oid = ts.cfgnamespace
+     WHERE n.nspname = 'public'
+       AND pg_get_userbyid(ts.cfgowner) = current_user
+    UNION ALL
+    SELECT format('%I', d.dictname) AS name, 'TEXT SEARCH DICTIONARY' AS type
+      FROM pg_ts_dict d JOIN pg_namespace n ON n.oid = d.dictnamespace
+     WHERE n.nspname = 'public'
+       AND pg_get_userbyid(d.dictowner) = current_user
+  ) obj
+UNION ALL
+SELECT format('DROP SCHEMA IF EXISTS %I CASCADE', n.nspname)
+  FROM pg_namespace n
+ WHERE NOT starts_with(n.nspname, 'pg_')
+   AND n.nspname NOT IN ('public', 'information_schema')
+   AND pg_get_userbyid(n.nspowner) = current_user
+\gexec

src/baudolo/restore/db/mariadb.py

@@ -1,9 +1,13 @@
 from __future__ import annotations
 
-import os
 import sys
+from pathlib import Path
 
-from ..run import docker_exec, docker_exec_sh
+from baudolo.restore.run import docker_exec, docker_exec_sh
+
+from .version import guard
+
+_NO_CLIENT = "ERROR: neither 'mariadb' nor 'mysql' found in container."
 
 
 def _pick_client(container: str) -> str:
@@ -19,14 +23,13 @@ exit 42
 """
     try:
         out = docker_exec_sh(container, script, capture=True).stdout.decode().strip()
-        if not out:
+    except Exception:
-            raise RuntimeError("empty client detection output")
+        print(_NO_CLIENT, file=sys.stderr)
-        return out
+        raise
-    except Exception as e:
+    if not out:
-        print(
+        print(_NO_CLIENT, file=sys.stderr)
-            "ERROR: neither 'mariadb' nor 'mysql' found in container.", file=sys.stderr
+        raise RuntimeError("empty client detection output")
-        )
+    return out
-        raise e
 
 
 def restore_mariadb_sql(
@@ -37,28 +40,25 @@ def restore_mariadb_sql(
     password: str,
     sql_path: str,
     empty: bool,
+    check_version: bool = True,
 ) -> None:
     client = _pick_client(container)
 
-    if not os.path.isfile(sql_path):
+    if not Path(sql_path).is_file():
         raise FileNotFoundError(sql_path)
 
-    if empty:
+    if check_version:
-        # IMPORTANT:
+        guard(
-        # Do NOT hardcode 'mysql' here. Use the detected client.
+            sql_path=sql_path,
-        # MariaDB 11 images may not contain the mysql binary at all.
+            engine="mariadb",
-        docker_exec(
+            container=container,
-            container,
+            user=user,
-            [
+            password=password,
-                client,
+            client=client,
-                "-u",
-                user,
-                f"--password={password}",
-                "-e",
-                "SET FOREIGN_KEY_CHECKS=0;",
-            ],
         )
 
+    if empty:
+        # Do not hardcode 'mysql': MariaDB 11 images may not ship that binary.
         result = docker_exec(
             container,
             [
@@ -68,13 +68,22 @@ def restore_mariadb_sql(
                 f"--password={password}",
                 "-N",
                 "-e",
-                f"SELECT table_name FROM information_schema.tables WHERE table_schema = '{db_name}';",
+                f"SELECT table_name FROM information_schema.tables WHERE table_schema = '{db_name}';",  # noqa: S608 - validate_database() constrains the name to ^[a-zA-Z0-9_][a-zA-Z0-9_-]*$
             ],
             capture=True,
         )
         tables = result.stdout.decode().split()
 
-        for tbl in tables:
+        if tables:
+            # SET FOREIGN_KEY_CHECKS is session-scoped, so it must share one
+            # client session with the DROPs or FK constraints still fire.
+            drop_sql = (
+                "SET FOREIGN_KEY_CHECKS=0; "
+                + " ".join(
+                    f"DROP TABLE IF EXISTS `{db_name}`.`{tbl}`;" for tbl in tables
+                )
+                + " SET FOREIGN_KEY_CHECKS=1;"
+            )
             docker_exec(
                 container,
                 [
@@ -83,23 +92,11 @@ def restore_mariadb_sql(
                     user,
                     f"--password={password}",
                     "-e",
-                    f"DROP TABLE IF EXISTS `{db_name}`.`{tbl}`;",
+                    drop_sql,
                 ],
             )
 
-        docker_exec(
+    with Path(sql_path).open("rb") as f:
-            container,
-            [
-                client,
-                "-u",
-                user,
-                f"--password={password}",
-                "-e",
-                "SET FOREIGN_KEY_CHECKS=1;",
-            ],
-        )
-
-    with open(sql_path, "rb") as f:
         docker_exec(
             container, [client, "-u", user, f"--password={password}", db_name], stdin=f
         )

src/baudolo/restore/db/postgres.py

@@ -1,8 +1,46 @@
 from __future__ import annotations
 
-import os
+import tempfile
+from pathlib import Path
+from typing import TYPE_CHECKING
 
-from ..run import docker_exec
+from baudolo.restore.run import docker_exec
+
+from .version import guard
+
+if TYPE_CHECKING:
+    from collections.abc import Iterable, Iterator
+
+_SUPERUSER_ONLY_PREFIXES = (b"COMMENT ON EXTENSION", b"ALTER DEFAULT PRIVILEGES")
+_EMPTY_PRECLEAN_SQL = Path(__file__).parent / "empty_preclean.sql"
+
+
+def filter_superuser_only_lines(lines: Iterable[bytes]) -> Iterator[bytes]:
+    """Drop superuser-only statements an app-level psql replay cannot run.
+
+    Args:
+        lines: dump lines including their trailing newlines.
+
+    Yields:
+        Every line except top-level statements starting with a superuser-only
+        prefix. Lines inside COPY ... FROM stdin data blocks are passed
+        through untouched: a data row may legally start with the same bytes,
+        and dropping it would silently corrupt the restored table.
+    """
+    in_copy = False
+    for line in lines:
+        if in_copy:
+            yield line
+            if line.rstrip(b"\r\n") == b"\\.":
+                in_copy = False
+            continue
+        if line.startswith(b"COPY ") and line.rstrip(b"\r\n").endswith(b"FROM stdin;"):
+            in_copy = True
+            yield line
+            continue
+        if line.startswith(_SUPERUSER_ONLY_PREFIXES):
+            continue
+        yield line
 
 
 def restore_postgres_sql(
@@ -13,28 +51,25 @@ def restore_postgres_sql(
     password: str,
     sql_path: str,
     empty: bool,
+    check_version: bool = True,
 ) -> None:
-    if not os.path.isfile(sql_path):
+    if not Path(sql_path).is_file():
         raise FileNotFoundError(sql_path)
 
-    # Make password available INSIDE the container for psql.
+    if check_version:
+        guard(
+            sql_path=sql_path,
+            engine="postgres",
+            container=container,
+            user=user,
+            password=password,
+        )
+
     docker_env = {"PGPASSWORD": password}
 
     if empty:
-        drop_sql = r"""
+        with _EMPTY_PRECLEAN_SQL.open(encoding="utf-8") as preclean:
-DO $$ DECLARE r RECORD;
+            drop_sql = preclean.read()
-BEGIN
-  FOR r IN (
-    SELECT table_name AS name, 'TABLE' AS type FROM information_schema.tables WHERE table_schema='public'
-    UNION ALL
-    SELECT routine_name AS name, 'FUNCTION' AS type FROM information_schema.routines WHERE specific_schema='public'
-    UNION ALL
-    SELECT sequence_name AS name, 'SEQUENCE' AS type FROM information_schema.sequences WHERE sequence_schema='public'
-  ) LOOP
-    EXECUTE format('DROP %s public.%I CASCADE', r.type, r.name);
-  END LOOP;
-END $$;
-"""
         docker_exec(
             container,
             ["psql", "-v", "ON_ERROR_STOP=1", "-U", user, "-d", db_name],
@@ -42,11 +77,26 @@ END $$;
             docker_env=docker_env,
         )
 
-    with open(sql_path, "rb") as f:
+    # Filter into a spooled temp file instead of building the whole dump in
+    # memory: production dumps reach many GB and the previous read/splitlines/
+    # join needed roughly three times the dump size in RSS.
+    with Path(sql_path).open("rb") as src, tempfile.TemporaryFile() as filtered:
+        for line in filter_superuser_only_lines(src):
+            filtered.write(line)
+        filtered.seek(0)
         docker_exec(
             container,
-            ["psql", "-v", "ON_ERROR_STOP=1", "-U", user, "-d", db_name],
+            [
-            stdin=f,
+                "psql",
+                "--single-transaction",
+                "-v",
+                "ON_ERROR_STOP=1",
+                "-U",
+                user,
+                "-d",
+                db_name,
+            ],
+            stdin=filtered,
             docker_env=docker_env,
         )
 

src/baudolo/restore/db/version.py

@@ -0,0 +1,147 @@
+"""Refuse a dump the target engine is too old to read.
+
+A restore with ``--empty`` destroys before it replays: the pre-clean drops the
+schema in one session and the dump goes in the next, with no rollback across
+the two. A dump the engine cannot parse therefore does not fail harmlessly -
+it leaves an emptied database behind. Comparing the two versions first turns
+that into a refusal.
+
+Both engines state their origin in the dump's own header, and they do not
+state it the same way. Postgres writes ``-- Dumped from database version``
+around line seven. MariaDB opens line two with ``-- MariaDB dump 10.19-11.8.8``,
+where the first number is mariadb-dump's own version, and names the server only
+further down on the tab-separated ``-- Server version`` line. Matching the first
+number in the header would read the tool on one engine and the server on the
+other, so each engine gets its own pattern.
+
+A ``pg_dumpall`` cluster dump has no version line of its own: its header opens
+with the cluster banner and the roles section, and the first
+``-- Dumped from database version`` belongs to the first database's embedded
+``pg_dump`` output, arbitrarily far down. Hence the scan runs to
+``SCAN_LINES`` rather than to a header-sized handful.
+"""
+
+from __future__ import annotations
+
+import re
+from pathlib import Path
+
+from baudolo.restore.run import docker_exec, stdout_of
+
+SCAN_LINES = 2000
+DUMP_VERSION = {
+    "postgres": re.compile(r"^-- Dumped from database version (\S+)"),
+    "mariadb": re.compile(r"^-- Server version\s+(\S+)"),
+}
+
+
+class VersionMismatchError(Exception):
+    """The dump cannot be replayed into this engine."""
+
+
+def major_of(version: str) -> int:
+    """The major number of an engine version string.
+
+    Args:
+        version: as the engine spells it, e.g. ``17.11`` or
+            ``11.8.8-MariaDB-ubu2404``.
+
+    Raises:
+        VersionMismatchError: the string does not start with a number.
+    """
+    leading = re.match(r"(\d+)", version)
+    if not leading:
+        raise VersionMismatchError(f"cannot read a major version from '{version}'")
+    return int(leading.group(1))
+
+
+def dump_version(sql_path: str, engine: str) -> str:
+    """Read the engine version a dump was taken from, out of its own header.
+
+    Args:
+        sql_path: the dump to read.
+        engine: ``postgres`` or ``mariadb``.
+
+    Returns:
+        The version string as the dump spells it.
+
+    Raises:
+        VersionMismatchError: no version line within the first ``SCAN_LINES``.
+    """
+    pattern = DUMP_VERSION[engine]
+    with Path(sql_path).open(encoding="utf-8", errors="replace") as handle:
+        for _ in range(SCAN_LINES):
+            line = handle.readline()
+            if not line:
+                break
+            found = pattern.search(line)
+            if found:
+                return found.group(1)
+    raise VersionMismatchError(
+        f"{sql_path} carries no {engine} version header in its first {SCAN_LINES} lines"
+    )
+
+
+def server_version(
+    container: str, engine: str, user: str, password: str, client: str = ""
+) -> str:
+    """Ask the running engine which version it is."""
+    if engine == "postgres":
+        return stdout_of(
+            docker_exec(
+                container,
+                ["psql", "-U", user, "-tAc", "SHOW server_version"],
+                capture=True,
+                docker_env={"PGPASSWORD": password},
+            )
+        )
+    return stdout_of(
+        docker_exec(
+            container,
+            [
+                client or "mariadb",
+                "-u",
+                user,
+                f"--password={password}",
+                "-N",
+                "-B",
+                "-e",
+                "SELECT VERSION()",
+            ],
+            capture=True,
+        )
+    )
+
+
+def assert_replayable(sql_path: str, engine: str, dumped: str, serving: str) -> None:
+    """Refuse a dump from a newer major version than the target engine.
+
+    Restoring forward across a major version is the upgrade path and stays
+    allowed; backward is refused, because a newer dump uses syntax an older
+    server rejects and the pre-clean would already have dropped the schema.
+
+    Raises:
+        VersionMismatchError: the dump is newer than the engine.
+    """
+    if major_of(dumped) > major_of(serving):
+        raise VersionMismatchError(
+            f"{sql_path} came from {engine} {dumped} but {serving} is running; "
+            "a newer dump does not replay into an older engine, and --empty "
+            "would drop the schema before finding out"
+        )
+
+
+def guard(
+    *,
+    sql_path: str,
+    engine: str,
+    container: str,
+    user: str,
+    password: str,
+    client: str = "",
+) -> None:
+    """Compare the dump's origin against the running engine before replaying."""
+    dumped = dump_version(sql_path, engine)
+    serving = server_version(container, engine, user, password, client)
+    assert_replayable(sql_path, engine, dumped, serving)
+    print(f"OK: dump is from {engine} {dumped}, {serving} is serving.")

src/baudolo/restore/files.py

@@ -1,15 +1,28 @@
+"""Restore a volume's file tree by writing into its mountpoint.
+
+That shortcut only holds for a plain local volume, where the mountpoint *is*
+the storage. A volume with driver options - NFS, a bind device, tmpfs - keeps
+the same ``/var/lib/docker/volumes/<name>/_data`` path, but docker mounts the
+real backing store over it on demand and unmounts it again when the last
+consumer stops. Writing there while nothing has it mounted lands in the empty
+directory underneath, is hidden by the next mount, and rsync reports success.
+"""
+
 from __future__ import annotations
 
 import os
 import sys
+from pathlib import Path
 
-from .run import run, docker_volume_exists
+from .run import docker_volume_exists, run, stdout_of
+
+INSPECT_FORMAT = (
+    "{{ .Mountpoint }}|{{ .Driver }}|{{ if .Options }}opts{{ else }}plain{{ end }}"
+)
 
 
-def restore_volume_files(
+def restore_volume_files(volume_name: str, backup_files_dir: str) -> int:
-    volume_name: str, backup_files_dir: str, *, rsync_image: str
+    if not Path(backup_files_dir).is_dir():
-) -> int:
-    if not os.path.isdir(backup_files_dir):
         print(f"ERROR: backup files dir not found: {backup_files_dir}", file=sys.stderr)
         return 2
 
@@ -19,21 +32,33 @@ def restore_volume_files(
     else:
         print(f"Volume {volume_name} already exists.")
 
-    # Keep behavior close to the old script: rsync -avv --delete
+    cp = run(
-    run(
+        ["docker", "volume", "inspect", "--format", INSPECT_FORMAT, volume_name],
-        [
+        capture=True,
-            "docker",
-            "run",
-            "--rm",
-            "-v",
-            f"{volume_name}:/recover/",
-            "-v",
-            f"{backup_files_dir}:/backup/",
-            rsync_image,
-            "sh",
-            "-lc",
-            "rsync -avv --delete /backup/ /recover/",
-        ]
     )
+    fields = stdout_of(cp).split("|")
+    mountpoint = fields[0] if fields else ""
+    if not mountpoint:
+        print(
+            f"ERROR: could not resolve mountpoint for volume {volume_name}",
+            file=sys.stderr,
+        )
+        return 2
+
+    driver, options = ([*fields, "local", "plain"])[1:3]
+    if (driver != "local" or options == "opts") and not os.path.ismount(mountpoint):
+        print(
+            f"ERROR: volume {volume_name} has a backing store of its own "
+            f"(driver {driver}) but nothing has it mounted; writing to "
+            f"{mountpoint} now would land under the mount and be lost. "
+            "Start a container that mounts the volume, then restore again.",
+            file=sys.stderr,
+        )
+        return 2
+
+    # rsync reads "dir/" as its contents and "dir" as the directory itself.
+    src = f"{Path(backup_files_dir)}{os.sep}"
+    dest = f"{Path(mountpoint)}{os.sep}"
+    run(["rsync", "-avv", "--delete", src, dest])
     print("File restore complete.")
     return 0

src/baudolo/restore/paths.py

@@ -1,7 +1,9 @@
 from __future__ import annotations
 
-import os
 from dataclasses import dataclass
+from pathlib import Path
+
+from baudolo.generation import CLUSTER_SUFFIX, DUMP_SUFFIX, FILES_DIR, SQL_DIR
 
 
 @dataclass(frozen=True)
@@ -14,16 +16,20 @@ class BackupPaths:
 
     def root(self) -> str:
         # Always build an absolute path under backups_dir
-        return os.path.join(
+        return str(
-            self.backups_dir,
+            Path(self.backups_dir)
-            self.backup_hash,
+            / self.backup_hash
-            self.repo_name,
+            / self.repo_name
-            self.version,
+            / self.version
-            self.volume_name,
+            / self.volume_name
         )
 
     def files_dir(self) -> str:
-        return os.path.join(self.root(), "files")
+        return str(Path(self.root()) / FILES_DIR)
 
     def sql_file(self, db_name: str) -> str:
-        return os.path.join(self.root(), "sql", f"{db_name}.backup.sql")
+        return str(Path(self.root()) / SQL_DIR / f"{db_name}{DUMP_SUFFIX}")
+
+    def cluster_file(self, instance: str) -> str:
+        """The pg_dumpall stream a `database = '*'` row produces."""
+        return str(Path(self.root()) / SQL_DIR / f"{instance}{CLUSTER_SUFFIX}")

src/baudolo/restore/run.py

@@ -2,7 +2,6 @@ from __future__ import annotations
 
 import subprocess
 import sys
-from typing import Optional
 
 
 def run(
@@ -10,7 +9,7 @@ def run(
     *,
     stdin=None,
     capture: bool = False,
-    env: Optional[dict] = None,
+    env: dict | None = None,
 ) -> subprocess.CompletedProcess:
     try:
         kwargs: dict = {
@@ -26,32 +25,35 @@ def run(
         else:
             kwargs["stdin"] = stdin
 
-        return subprocess.run(cmd, **kwargs)
+        return subprocess.run(cmd, **kwargs)  # noqa: PLW1510 - check lives in kwargs
 
     except subprocess.CalledProcessError as e:
         msg = f"ERROR: command failed ({e.returncode}): {' '.join(cmd)}"
         print(msg, file=sys.stderr)
-        if e.stdout:
+        for stream in (e.stdout, e.stderr):
+            if not stream:
+                continue
             try:
-                print(e.stdout.decode(), file=sys.stderr)
+                print(stream.decode(), file=sys.stderr)
-            except Exception:
+            except (UnicodeDecodeError, AttributeError):
-                print(e.stdout, file=sys.stderr)
+                print(stream, file=sys.stderr)
-        if e.stderr:
-            try:
-                print(e.stderr.decode(), file=sys.stderr)
-            except Exception:
-                print(e.stderr, file=sys.stderr)
         raise
 
 
+def stdout_of(completed: subprocess.CompletedProcess) -> str:
+    """The captured stdout as stripped text, whether it came back bytes or str."""
+    raw = completed.stdout or b""
+    return (raw.decode() if isinstance(raw, bytes) else raw).strip()
+
+
 def docker_exec(
     container: str,
     argv: list[str],
     *,
     stdin=None,
     capture: bool = False,
-    env: Optional[dict] = None,
+    env: dict | None = None,
-    docker_env: Optional[dict[str, str]] = None,
+    docker_env: dict[str, str] | None = None,
 ) -> subprocess.CompletedProcess:
     cmd: list[str] = ["docker", "exec", "-i"]
     if docker_env:
@@ -67,8 +69,8 @@ def docker_exec_sh(
     *,
     stdin=None,
     capture: bool = False,
-    env: Optional[dict] = None,
+    env: dict | None = None,
-    docker_env: Optional[dict[str, str]] = None,
+    docker_env: dict[str, str] | None = None,
 ) -> subprocess.CompletedProcess:
     return docker_exec(
         container,
@@ -85,5 +87,6 @@ def docker_volume_exists(volume: str) -> bool:
         ["docker", "volume", "inspect", volume],
         stdout=subprocess.DEVNULL,
         stderr=subprocess.DEVNULL,
+        check=False,
     )
     return p.returncode == 0

src/baudolo/seed/__main__.py

@@ -1,67 +1,93 @@
-import pandas as pd
+from __future__ import annotations
+
 import argparse
-import os
+import sys
+from pathlib import Path
+
+import pandas as pd
+from pandas.errors import EmptyDataError
+
+from baudolo.databases import COLUMNS, DELIMITER, validate_database
 
 
-def check_and_add_entry(file_path, instance, database, username, password):
+def _empty_df() -> pd.DataFrame:
-    # Check if the file exists and is not empty
+    return pd.DataFrame(columns=list(COLUMNS))
-    if os.path.exists(file_path) and os.path.getsize(file_path) > 0:
+
-        # Read the existing CSV file with header
+
-        df = pd.read_csv(file_path, sep=";")
+def check_and_add_entry(
+    file_path: str,
+    instance: str,
+    database: str | None,
+    username: str,
+    password: str,
+) -> None:
+    """
+    Add or update an entry in databases.csv.
+
+    The function enforces strict validation:
+    - database MUST be set
+    - database MUST be '*' or a valid database name
+    """
+    database = validate_database(database, instance=instance)
+
+    if Path(file_path).exists():
+        try:
+            df = pd.read_csv(
+                file_path,
+                sep=DELIMITER,
+                dtype=str,
+                keep_default_na=False,
+            )
+        except EmptyDataError:
+            print(
+                f"WARNING: databases.csv exists but is empty: {file_path}. Creating header columns.",
+                file=sys.stderr,
+            )
+            df = _empty_df()
     else:
-        # Create a new DataFrame with columns if file does not exist
+        df = _empty_df()
-        df = pd.DataFrame(columns=["instance", "database", "username", "password"])
+    mask = (df["instance"] == instance) & (df["database"] == database)
 
-    # Check if the entry exists and remove it
+    if mask.any():
-    mask = (
+        print("Updating existing entry.")
-        (df["instance"] == instance)
+        df.loc[mask, ["username", "password"]] = [username, password]
-        & (
-            (df["database"] == database)
-            | (((df["database"].isna()) | (df["database"] == "")) & (database == ""))
-        )
-        & (df["username"] == username)
-    )
-
-    if not df[mask].empty:
-        print("Replacing existing entry.")
-        df = df[~mask]
     else:
         print("Adding new entry.")
+        new_entry = pd.DataFrame(
+            [[instance, database, username, password]],
+            columns=list(COLUMNS),
+        )
+        df = pd.concat([df, new_entry], ignore_index=True)
 
-    # Create a new DataFrame for the new entry
+    df.to_csv(file_path, sep=DELIMITER, index=False)
-    new_entry = pd.DataFrame(
-        [
-            {
-                "instance": instance,
-                "database": database,
-                "username": username,
-                "password": password,
-            }
-        ]
-    )
-
-    # Add (or replace) the entry using concat
-    df = pd.concat([df, new_entry], ignore_index=True)
-
-    # Save the updated CSV file
-    df.to_csv(file_path, sep=";", index=False)
 
 
-def main():
+def main() -> None:
     parser = argparse.ArgumentParser(
-        description="Check and replace (or add) a database entry in a CSV file."
+        description="Seed or update databases.csv for backup configuration."
     )
-    parser.add_argument("file_path", help="Path to the CSV file")
+    parser.add_argument("file", help="Path to databases.csv")
-    parser.add_argument("instance", help="Database instance")
+    parser.add_argument("instance", help="Instance name (e.g. bigbluebutton)")
-    parser.add_argument("database", help="Database name")
+    parser.add_argument(
-    parser.add_argument("username", help="Username")
+        "database",
-    parser.add_argument("password", nargs="?", default="", help="Password (optional)")
+        help="Database name or '*' to dump all databases",
+    )
+    parser.add_argument("username", help="Database username")
+    parser.add_argument("password", help="Database password")
 
     args = parser.parse_args()
 
-    check_and_add_entry(
+    try:
-        args.file_path, args.instance, args.database, args.username, args.password
+        check_and_add_entry(
-    )
+            file_path=args.file,
+            instance=args.instance,
+            database=args.database,
+            username=args.username,
+            password=args.password,
+        )
+    except Exception as exc:  # noqa: BLE001 - CLI boundary: any failure becomes exit 1
+        print(f"ERROR: {exc}", file=sys.stderr)
+        sys.exit(1)
 
 
 if __name__ == "__main__":

tests/e2e/faithful_driver.py

@@ -0,0 +1,90 @@
+"""Show what a real snapshot holds for a volume that has its own storage.
+
+Runs inside the privileged container that built the btrfs subject. Prints one
+PASS/FAIL line per assertion and exits non-zero on the first failure.
+"""
+
+from __future__ import annotations
+
+import subprocess
+import sys
+from pathlib import Path
+
+sys.path.insert(0, "/src")
+
+from baudolo.backup.snapshot import (
+    SnapshotError,
+    snapshot_source,
+    unsnapshotted,
+    volume_snapshot,
+)
+from baudolo.backup.volume import Backing
+
+SUBJECT = sys.argv[1]
+
+
+def shell(command: list[str]) -> list[str]:
+    proc = subprocess.run(command, capture_output=True, text=True, check=False)
+    if proc.returncode != 0:
+        raise SnapshotError(
+            f"{' '.join(command)} exited {proc.returncode}: {proc.stderr.strip()}"
+        )
+    return proc.stdout.splitlines()
+
+
+def check(label: str, condition: bool) -> None:
+    print(f"{'PASS' if condition else 'FAIL'} {label}", flush=True)
+    if not condition:
+        sys.exit(1)
+
+
+def volume(name: str, payload: str) -> Path:
+    path = Path(SUBJECT) / "volumes" / name / "_data"
+    path.mkdir(parents=True, exist_ok=True)
+    (path / "state").write_text(payload)
+    return path
+
+
+plain = volume("plain", "plain-payload")
+own = Path(SUBJECT) / "volumes" / "own" / "_data"
+own.mkdir(parents=True, exist_ok=True)
+shell(["mount", "-t", "tmpfs", "tmpfs", own])
+(own / "state").write_text("own-payload")
+
+check("a plain volume is captured", unsnapshotted(Backing(str(plain)), SUBJECT) is None)
+check(
+    "a volume on a mount of its own is not",
+    unsnapshotted(Backing(str(own)), SUBJECT) is not None,
+)
+check(
+    "a declared backing store is not, mounted or not",
+    unsnapshotted(Backing(str(plain), options={"type": "nfs"}), SUBJECT) is not None,
+)
+check(
+    "a foreign driver is not",
+    unsnapshotted(Backing(str(plain), driver="rexray"), SUBJECT) is not None,
+)
+
+with volume_snapshot("btrfs", SUBJECT, "e2e", run=shell) as resolve:
+    frozen_plain = Path(resolve(str(plain)))
+    frozen_own = Path(resolve(str(own)))
+
+    check(
+        "the snapshot carries the plain volume",
+        (frozen_plain / "state").read_text() == "plain-payload",
+    )
+    check(
+        "the snapshot shows the other volume as an empty directory",
+        frozen_own.is_dir() and not any(frozen_own.iterdir()),
+    )
+
+    source, reason = snapshot_source(resolve, Backing(str(plain)), SUBJECT)
+    check(
+        "the plain volume is read from the snapshot",
+        source is not None and source.rstrip("/") == str(frozen_plain),
+    )
+
+    source, reason = snapshot_source(resolve, Backing(str(own)), SUBJECT)
+    check(f"the other volume degrades to live: {reason[:60]}", source is None)
+
+print("ALL OK", flush=True)

tests/e2e/helpers/__init__.py

@@ -0,0 +1,4 @@
+"""Shared e2e helpers, re-exported so tests import one name."""
+
+from .fixtures import *
+from .process import *

tests/e2e/helpers/fixtures.py

@@ -0,0 +1,114 @@
+"""Fixtures and paths the e2e suite builds its scenarios from."""
+
+from __future__ import annotations
+
+import shutil
+import subprocess
+from pathlib import Path
+
+from .process import machine_hash, run
+
+# postgres 18+ mounts at /var/lib/postgresql, not /var/lib/postgresql/data.
+POSTGRES_IMAGE = "postgres:alpine"
+POSTGRES_DATA_DIR = "/var/lib/postgresql"
+MARIADB_IMAGE = "mariadb:latest"
+MARIADB_DATA_DIR = "/var/lib/mysql"
+
+
+def backup_run(
+    *,
+    backups_dir: str,
+    repo_name: str,
+    compose_dir: str,
+    databases_csv: str,
+    database_containers: list[str],
+    images_no_stop_required: list[str],
+    images_no_backup_required: list[str] | None = None,
+    only_sql: bool = False,
+) -> None:
+    cmd = [
+        "baudolo",
+        "--compose-dir",
+        compose_dir,
+        "--hard-restart-projects",
+        "mailu",
+        "--repo-name",
+        repo_name,
+        "--databases-csv",
+        databases_csv,
+        "--backups-dir",
+        backups_dir,
+        "--database-containers",
+        *database_containers,
+        "--images-no-stop-required",
+        *images_no_stop_required,
+    ]
+    if images_no_backup_required:
+        cmd += ["--images-no-backup-required", *images_no_backup_required]
+    if only_sql:
+        cmd += ["--only-sql"]
+
+    try:
+        run(cmd, capture=True, check=True)
+    except subprocess.CalledProcessError as e:
+        print(">>> baudolo failed (exit code:", e.returncode, ")")
+        if e.stdout:
+            print(">>> baudolo STDOUT:\n" + e.stdout)
+        if e.stderr:
+            print(">>> baudolo STDERR:\n" + e.stderr)
+        raise
+
+
+def latest_version_dir(backups_dir: str, repo_name: str) -> tuple[str, str]:
+    """
+    Returns (hash, version) for the latest backup.
+    """
+    h = machine_hash()
+    root = Path(backups_dir) / h / repo_name
+    if not root.is_dir():
+        raise FileNotFoundError(str(root))
+
+    versions = sorted([p.name for p in root.iterdir() if p.is_dir()])
+    if not versions:
+        raise RuntimeError(f"No versions found under {root}")
+    return h, versions[-1]
+
+
+def backup_path(backups_dir: str, repo_name: str, version: str, volume: str) -> Path:
+    h = machine_hash()
+    return Path(backups_dir) / h / repo_name / version / volume
+
+
+def create_minimal_compose_dir(base: str) -> str:
+    """
+    baudolo requires --compose-dir. Create an empty dir with one non-compose subdir.
+    """
+    p = Path(base) / "compose-root"
+    p.mkdir(parents=True, exist_ok=True)
+    (p / "noop").mkdir(parents=True, exist_ok=True)
+    return str(p)
+
+
+def write_databases_csv(path: str, rows: list[tuple[str, str, str, str]]) -> None:
+    """
+    rows: (instance, database, username, password)
+    database may be '' (empty) to trigger pg_dumpall behavior if you want, but here we use db name.
+    """
+    Path(path).parent.mkdir(parents=True, exist_ok=True)
+    with Path(path).open("w", encoding="utf-8") as f:
+        f.write("instance;database;username;password\n")
+        f.writelines(f"{inst};{db};{user};{pw}\n" for inst, db, user, pw in rows)
+
+
+def cleanup_docker(*, containers: list[str], volumes: list[str]) -> None:
+    for c in containers:
+        run(["docker", "rm", "-f", c], capture=True, check=False)
+    for v in volumes:
+        run(["docker", "volume", "rm", "-f", v], capture=True, check=False)
+
+
+def ensure_empty_dir(path: str) -> None:
+    p = Path(path)
+    if p.exists():
+        shutil.rmtree(p)
+    p.mkdir(parents=True, exist_ok=True)

tests/e2e/helpers/process.py

@@ -1,11 +1,10 @@
-# tests/e2e/helpers.py
+"""Process, docker and readiness helpers for the e2e suite."""
+
 from __future__ import annotations
 
-import shutil
 import subprocess
 import time
 import uuid
-from pathlib import Path
 
 
 def run(
@@ -155,103 +154,3 @@ def wait_for_mariadb_sql(
     raise TimeoutError(
         f"Timed out waiting for MariaDB SQL login readiness in container {container}"
     )
-
-
-def backup_run(
-    *,
-    backups_dir: str,
-    repo_name: str,
-    compose_dir: str,
-    databases_csv: str,
-    database_containers: list[str],
-    images_no_stop_required: list[str],
-    images_no_backup_required: list[str] | None = None,
-    dump_only: bool = False,
-) -> None:
-    cmd = [
-        "baudolo",
-        "--compose-dir",
-        compose_dir,
-        "--docker-compose-hard-restart-required",
-        "mailu",
-        "--repo-name",
-        repo_name,
-        "--databases-csv",
-        databases_csv,
-        "--backups-dir",
-        backups_dir,
-        "--database-containers",
-        *database_containers,
-        "--images-no-stop-required",
-        *images_no_stop_required,
-    ]
-    if images_no_backup_required:
-        cmd += ["--images-no-backup-required", *images_no_backup_required]
-    if dump_only:
-        cmd += ["--dump-only"]
-
-    try:
-        run(cmd, capture=True, check=True)
-    except subprocess.CalledProcessError as e:
-        print(">>> baudolo failed (exit code:", e.returncode, ")")
-        if e.stdout:
-            print(">>> baudolo STDOUT:\n" + e.stdout)
-        if e.stderr:
-            print(">>> baudolo STDERR:\n" + e.stderr)
-        raise
-
-
-def latest_version_dir(backups_dir: str, repo_name: str) -> tuple[str, str]:
-    """
-    Returns (hash, version) for the latest backup.
-    """
-    h = machine_hash()
-    root = Path(backups_dir) / h / repo_name
-    if not root.is_dir():
-        raise FileNotFoundError(str(root))
-
-    versions = sorted([p.name for p in root.iterdir() if p.is_dir()])
-    if not versions:
-        raise RuntimeError(f"No versions found under {root}")
-    return h, versions[-1]
-
-
-def backup_path(backups_dir: str, repo_name: str, version: str, volume: str) -> Path:
-    h = machine_hash()
-    return Path(backups_dir) / h / repo_name / version / volume
-
-
-def create_minimal_compose_dir(base: str) -> str:
-    """
-    baudolo requires --compose-dir. Create an empty dir with one non-compose subdir.
-    """
-    p = Path(base) / "compose-root"
-    p.mkdir(parents=True, exist_ok=True)
-    (p / "noop").mkdir(parents=True, exist_ok=True)
-    return str(p)
-
-
-def write_databases_csv(path: str, rows: list[tuple[str, str, str, str]]) -> None:
-    """
-    rows: (instance, database, username, password)
-    database may be '' (empty) to trigger pg_dumpall behavior if you want, but here we use db name.
-    """
-    Path(path).parent.mkdir(parents=True, exist_ok=True)
-    with open(path, "w", encoding="utf-8") as f:
-        f.write("instance;database;username;password\n")
-        for inst, db, user, pw in rows:
-            f.write(f"{inst};{db};{user};{pw}\n")
-
-
-def cleanup_docker(*, containers: list[str], volumes: list[str]) -> None:
-    for c in containers:
-        run(["docker", "rm", "-f", c], capture=True, check=False)
-    for v in volumes:
-        run(["docker", "volume", "rm", "-f", v], capture=True, check=False)
-
-
-def ensure_empty_dir(path: str) -> None:
-    p = Path(path)
-    if p.exists():
-        shutil.rmtree(p)
-    p.mkdir(parents=True, exist_ok=True)

tests/e2e/snapshot_db_driver.py

@@ -0,0 +1,43 @@
+"""Copy a live database's volume out of a snapshot, using the real backup path.
+
+Runs inside the privileged container built by test_e2e_snapshot_db.py, where a
+database is mid-write on a btrfs subvolume. Exercises volume_snapshot and
+backup_volume exactly as a backup run would.
+"""
+
+from __future__ import annotations
+
+import subprocess
+import sys
+
+sys.path.insert(0, "/src")
+
+from baudolo.backup.snapshot import SnapshotError, volume_snapshot
+from baudolo.backup.volume import backup_volume
+
+SUBJECT = "/subject/docker"
+VOLUME = "mariadb_data"
+DATADIR = f"{SUBJECT}/volumes/{VOLUME}/_data"
+VERSIONS = "/backups"
+GENERATION = f"{VERSIONS}/20260731"
+
+
+def shell(command: list[str]) -> list[str]:
+    proc = subprocess.run(command, capture_output=True, text=True, check=False)
+    if proc.returncode != 0:
+        raise SnapshotError(
+            f"{' '.join(command)} exited {proc.returncode}: {proc.stderr.strip()}"
+        )
+    return proc.stdout.splitlines()
+
+
+with volume_snapshot("btrfs", SUBJECT, "dbtest", run=shell) as resolve:
+    backup_volume(
+        VERSIONS,
+        VOLUME,
+        f"{GENERATION}/{VOLUME}",
+        authoritative=True,
+        source=resolve(f"{DATADIR}/"),
+    )
+
+print("SNAPSHOT COPY DONE", flush=True)

tests/e2e/snapshot_driver.py

@@ -0,0 +1,64 @@
+"""Exercise volume_snapshot against a real filesystem, from inside a container.
+
+Runs where loop devices exist. Prints one PASS/FAIL line per assertion and exits
+non-zero on the first failure, so the calling test can surface the reason.
+"""
+
+from __future__ import annotations
+
+import subprocess
+import sys
+from pathlib import Path
+
+sys.path.insert(0, "/src")
+
+from baudolo.backup.snapshot import SnapshotError, volume_snapshot
+
+KIND = sys.argv[1]
+SUBJECT = sys.argv[2]
+EXPECT = sys.argv[3]
+
+
+def shell(command: list[str]) -> list[str]:
+    proc = subprocess.run(command, capture_output=True, text=True, check=False)
+    if proc.returncode != 0:
+        raise SnapshotError(
+            f"{' '.join(command)} exited {proc.returncode}: {proc.stderr.strip()}"
+        )
+    return proc.stdout.splitlines()
+
+
+def check(label: str, condition: bool) -> None:
+    print(f"{'PASS' if condition else 'FAIL'} {label}", flush=True)
+    if not condition:
+        sys.exit(1)
+
+
+volume = Path(SUBJECT) / "volumes" / "demo" / "_data"
+volume.mkdir(parents=True, exist_ok=True)
+(volume / "state").write_text("before\n")
+
+if EXPECT == "unsupported":
+    try:
+        with volume_snapshot(KIND, SUBJECT, "e2e", run=shell):
+            check("snapshot on an unsupported filesystem must not succeed", False)
+    except SnapshotError as exc:
+        check(f"refused loudly: {str(exc)[:60]}", True)
+    sys.exit(0)
+
+with volume_snapshot(KIND, SUBJECT, "e2e", run=shell) as resolve:
+    frozen = Path(resolve(str(volume))) / "state"
+    check("the snapshot exposes the volume", frozen.is_file())
+    check("the snapshot carries the content", frozen.read_text() == "before\n")
+
+    (volume / "state").write_text("after\n")
+    check("a later write does not reach the snapshot", frozen.read_text() == "before\n")
+    check("the live tree did change", (volume / "state").read_text() == "after\n")
+
+    root = Path(resolve(SUBJECT))
+
+check(
+    "the snapshot is removed afterwards",
+    not root.exists() or not (root / "volumes").exists(),
+)
+print("ALL OK", flush=True)

tests/e2e/test_e2e_app_container_ships_client_tools.py

@@ -0,0 +1,184 @@
+"""An application container that ships the engine's client tools.
+
+This is the shape a dedicated database deploys in: the engine runs as
+`<app>-database` while the application itself runs as `<app>`, and neither is
+declared through --database-containers, so both names go through the instance
+regex. `<app>-database` loses its suffix and lands on the instance `<app>` -
+and `<app>` carries no database token at all, so a fallback that returns the
+name unchanged lands on that same instance and offers the application container
+as a second engine for the same row.
+
+Discourse is the live example: its application container is named `discourse`
+by its own launcher and ships pg_dumpall, so a dump command starts there and
+writes a file that looks like a backup and holds none of the data.
+"""
+
+import json
+import unittest
+from pathlib import Path
+
+from baudolo.generation import DUMP_SUFFIX, FILES_DIR, MANIFEST_FILE, SQL_DIR
+
+from .helpers import (
+    POSTGRES_DATA_DIR,
+    POSTGRES_IMAGE,
+    backup_path,
+    backup_run,
+    cleanup_docker,
+    create_minimal_compose_dir,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    wait_for_postgres,
+    write_databases_csv,
+)
+
+MARKER = "the-application-volume-holds-files"
+PAYLOAD = "shop-payload"
+
+
+class TestE2EAppContainerShipsClientTools(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        # uuid4 hex may begin with "db", which the instance regex would split
+        # on and turn the application container into a different instance,
+        # hiding exactly the collision this module is about.
+        cls.prefix = unique("baudolo-e2e-app-tools").replace("-db", "-xb")
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+
+        cls.engine = f"{cls.prefix}-shop-database"
+        cls.app = f"{cls.prefix}-shop"
+        cls.engine_volume = f"{cls.prefix}-shop-database-vol"
+        cls.app_volume = f"{cls.prefix}-shop-app-vol"
+        cls.containers = [cls.engine, cls.app]
+        cls.volumes = [cls.engine_volume, cls.app_volume]
+
+        run(["docker", "volume", "create", cls.engine_volume])
+        run(["docker", "volume", "create", cls.app_volume])
+
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.engine,
+                "-e",
+                "POSTGRES_PASSWORD=shoppw",
+                "-e",
+                "POSTGRES_DB=shopdb",
+                "-e",
+                "POSTGRES_USER=postgres",
+                "-v",
+                f"{cls.engine_volume}:{POSTGRES_DATA_DIR}",
+                POSTGRES_IMAGE,
+            ]
+        )
+
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.app,
+                "--entrypoint",
+                "sh",
+                "-v",
+                f"{cls.app_volume}:/data",
+                POSTGRES_IMAGE,
+                "-c",
+                f"echo '{MARKER}' > /data/marker.txt && sleep 3600",
+            ]
+        )
+
+        wait_for_postgres(cls.engine, user="postgres", timeout_s=90)
+        run(
+            [
+                "docker",
+                "exec",
+                cls.engine,
+                "sh",
+                "-lc",
+                (
+                    'psql -U postgres -d shopdb -c "CREATE TABLE orders (id int, '
+                    f"note text); INSERT INTO orders VALUES (1,'{PAYLOAD}');\""
+                ),
+            ],
+            check=True,
+        )
+
+        cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
+        write_databases_csv(
+            cls.databases_csv,
+            [(cls.app, "shopdb", "postgres", "shoppw")],
+        )
+
+        backup_run(
+            backups_dir=cls.backups_dir,
+            repo_name=cls.repo_name,
+            compose_dir=cls.compose_dir,
+            databases_csv=cls.databases_csv,
+            database_containers=["dummy-db"],
+            images_no_stop_required=[POSTGRES_IMAGE],
+        )
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=cls.containers, volumes=cls.volumes)
+
+    def volume_dir(self, volume: str) -> Path:
+        return backup_path(self.backups_dir, self.repo_name, self.version, volume)
+
+    def test_the_engine_volume_was_dumped(self) -> None:
+        dump = self.volume_dir(self.engine_volume) / SQL_DIR / f"shopdb{DUMP_SUFFIX}"
+        self.assertTrue(dump.is_file(), f"expected a dump at {dump}")
+        self.assertIn(PAYLOAD, dump.read_text(encoding="utf-8"))
+
+    def test_the_application_volume_produced_no_dump(self) -> None:
+        """The collision this module exists for: the application container
+        answers the same instance as the engine and starts a dump of its own."""
+        sql_dir = self.volume_dir(self.app_volume) / SQL_DIR
+        self.assertFalse(
+            sql_dir.exists(),
+            f"the application container was dumped: {sorted(sql_dir.iterdir())}"
+            if sql_dir.exists()
+            else "",
+        )
+
+    def test_the_application_volume_was_backed_up_as_files(self) -> None:
+        """Refusing the dump must not cost the volume its backup."""
+        marker = self.volume_dir(self.app_volume) / FILES_DIR / "marker.txt"
+        self.assertTrue(marker.is_file(), f"expected a file backup at {marker}")
+        self.assertIn(MARKER, marker.read_text(encoding="utf-8"))
+
+    def test_the_manifest_does_not_call_the_application_volume_a_database(self) -> None:
+        manifest = json.loads(
+            (self.volume_dir(self.app_volume).parent / MANIFEST_FILE).read_text(
+                encoding="utf-8"
+            )
+        )
+        entry = manifest["volumes"][self.app_volume]
+        self.assertFalse(entry["database"], entry)
+        self.assertFalse(entry["dumped"], entry)
+
+    def test_the_manifest_records_the_engine_volume_as_dumped(self) -> None:
+        manifest = json.loads(
+            (self.volume_dir(self.engine_volume).parent / MANIFEST_FILE).read_text(
+                encoding="utf-8"
+            )
+        )
+        entry = manifest["volumes"][self.engine_volume]
+        self.assertTrue(entry["database"], entry)
+        self.assertTrue(entry["dumped"], entry)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/e2e/test_e2e_cli_contract_only_sql.py

@@ -0,0 +1,33 @@
+import unittest
+
+from .helpers import run
+
+WITHDRAWN_FLAGS = ["--dump-only", "--dump-only-sql", "--everything"]
+
+
+class TestE2ECLIContractOnlySql(unittest.TestCase):
+    def test_help_mentions_the_flag(self) -> None:
+        cp = run(["baudolo", "--help"], capture=True, check=True)
+        out = (cp.stdout or "") + "\n" + (cp.stderr or "")
+        self.assertIn(
+            "--only-sql",
+            out,
+            f"Expected '--only-sql' to appear in --help output. Output:\n{out}",
+        )
+
+    def test_a_withdrawn_flag_is_rejected(self) -> None:
+        for flag in WITHDRAWN_FLAGS:
+            with self.subTest(flag=flag):
+                cp = run(["baudolo", flag], capture=True, check=False)
+                self.assertEqual(
+                    cp.returncode,
+                    2,
+                    f"Expected exitcode 2 for unknown args, got {cp.returncode}\n"
+                    f"STDOUT={cp.stdout}\nSTDERR={cp.stderr}",
+                )
+                err = (cp.stderr or "") + "\n" + (cp.stdout or "")
+                # Argparse typically prints "unrecognized arguments"
+                self.assertTrue(
+                    ("unrecognized arguments" in err) or ("usage:" in err.lower()),
+                    f"Expected argparse-style error output. Output:\n{err}",
+                )

tests/e2e/test_e2e_engine_detection_by_tool.py

@@ -0,0 +1,177 @@
+"""The engine comes from the tools a container ships, not from its image name.
+
+Two containers in one backup run, each lying in one direction:
+
+* a real Postgres tagged `<prefix>-database`, the way a dedicated database is
+  built inside an app's own stack - no engine token anywhere in the name;
+* an Alpine tagged `postgres:<prefix>`, carrying the token without shipping a
+  single Postgres binary.
+
+Reading the name gets both wrong, and the second one fatally: pg_dump exits 127
+inside Alpine and takes the whole run with it.
+"""
+
+import unittest
+
+from .helpers import (
+    POSTGRES_DATA_DIR,
+    POSTGRES_IMAGE,
+    backup_path,
+    backup_run,
+    cleanup_docker,
+    create_minimal_compose_dir,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    wait_for_postgres,
+    write_databases_csv,
+)
+
+IMPOSTOR_BASE_IMAGE = "alpine:3.20"
+MARKER = "engine-detection-by-tool"
+
+
+class TestE2EEngineDetectionByTool(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        cls.prefix = unique("baudolo-e2e-engine-by-tool")
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+
+        cls.engine_image = f"{cls.prefix}-database:17"
+        cls.impostor_image = f"postgres:{cls.prefix}"
+
+        cls.engine_container = f"{cls.prefix}-engine"
+        cls.impostor_container = f"{cls.prefix}-impostor"
+        cls.engine_volume = f"{cls.prefix}-engine-vol"
+        cls.impostor_volume = f"{cls.prefix}-impostor-vol"
+
+        cls.containers = [cls.engine_container, cls.impostor_container]
+        cls.volumes = [cls.engine_volume, cls.impostor_volume]
+
+        run(["docker", "pull", POSTGRES_IMAGE])
+        run(["docker", "pull", IMPOSTOR_BASE_IMAGE])
+        run(["docker", "tag", POSTGRES_IMAGE, cls.engine_image])
+        run(["docker", "tag", IMPOSTOR_BASE_IMAGE, cls.impostor_image])
+        run(["docker", "volume", "create", cls.engine_volume])
+        run(["docker", "volume", "create", cls.impostor_volume])
+
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.engine_container,
+                "-e",
+                "POSTGRES_PASSWORD=pgpw",
+                "-e",
+                "POSTGRES_DB=appdb",
+                "-e",
+                "POSTGRES_USER=postgres",
+                "-v",
+                f"{cls.engine_volume}:{POSTGRES_DATA_DIR}",
+                cls.engine_image,
+            ]
+        )
+        wait_for_postgres(cls.engine_container, user="postgres", timeout_s=90)
+        run(
+            [
+                "docker",
+                "exec",
+                cls.engine_container,
+                "sh",
+                "-lc",
+                (
+                    "psql -U postgres -d appdb -c "
+                    '"CREATE TABLE t (id int primary key, v text); '
+                    "INSERT INTO t VALUES (1,'ok');\""
+                ),
+            ]
+        )
+
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.impostor_container,
+                "-v",
+                f"{cls.impostor_volume}:/data",
+                cls.impostor_image,
+                "sh",
+                "-lc",
+                f"echo '{MARKER}' > /data/marker.txt && sleep 3600",
+            ]
+        )
+
+        cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
+        write_databases_csv(
+            cls.databases_csv,
+            [
+                (cls.engine_container, "appdb", "postgres", "pgpw"),
+                (cls.impostor_container, "appdb", "postgres", "pgpw"),
+            ],
+        )
+
+        backup_run(
+            backups_dir=cls.backups_dir,
+            repo_name=cls.repo_name,
+            compose_dir=cls.compose_dir,
+            databases_csv=cls.databases_csv,
+            database_containers=[cls.engine_container, cls.impostor_container],
+            images_no_stop_required=[cls.engine_image, cls.impostor_image],
+            only_sql=True,
+        )
+
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=cls.containers, volumes=cls.volumes)
+        run(["docker", "rmi", cls.engine_image], check=False)
+        run(["docker", "rmi", cls.impostor_image], check=False)
+
+    def _volume_dir(self, volume: str):
+        return backup_path(self.backups_dir, self.repo_name, self.version, volume)
+
+    def test_an_engine_without_an_engine_name_is_still_dumped(self) -> None:
+        dump = self._volume_dir(self.engine_volume) / "sql" / "appdb.backup.sql"
+        self.assertTrue(
+            dump.is_file(),
+            f"a Postgres tagged '{self.engine_image}' produced no dump at {dump}",
+        )
+        self.assertIn("Dumped by pg_dump", dump.read_text(encoding="utf-8"))
+
+    def test_an_engine_name_without_an_engine_is_not_dumped(self) -> None:
+        sql_dir = self._volume_dir(self.impostor_volume) / "sql"
+        dumps = list(sql_dir.glob("*.sql")) if sql_dir.exists() else []
+        self.assertEqual(
+            dumps,
+            [],
+            f"'{self.impostor_image}' ships no Postgres yet was dumped: {dumps}",
+        )
+
+    def test_the_recognised_engine_is_dumped_instead_of_copied(self) -> None:
+        files = self._volume_dir(self.engine_volume) / "files"
+        self.assertFalse(
+            files.exists(),
+            f"--only-sql still copied the engine's files to {files}",
+        )
+
+    def test_the_impostor_falls_through_to_a_file_backup(self) -> None:
+        files = self._volume_dir(self.impostor_volume) / "files"
+        self.assertTrue(files.is_dir(), f"expected a file backup at {files}")
+        self.assertEqual(
+            (files / "marker.txt").read_text(encoding="utf-8").strip(), MARKER
+        )
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/e2e/test_e2e_engine_detection_registry_prefix.py

@@ -0,0 +1,105 @@
+import unittest
+
+from .helpers import (
+    POSTGRES_DATA_DIR,
+    POSTGRES_IMAGE,
+    backup_path,
+    backup_run,
+    cleanup_docker,
+    create_minimal_compose_dir,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    wait_for_postgres,
+    write_databases_csv,
+)
+
+REGISTRY_HOST = "svc-db-mariadb-swarm-mgr-01:5000"
+
+
+class TestE2EEngineDetectionRegistryPrefix(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        cls.prefix = unique("baudolo-e2e-registry-prefix")
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+
+        cls.image = f"{REGISTRY_HOST}/postgres_custom:17-3.5"
+        cls.pg_container = f"{cls.prefix}-pg"
+        cls.pg_volume = f"{cls.prefix}-pg-vol"
+        cls.containers = [cls.pg_container]
+        cls.volumes = [cls.pg_volume]
+
+        run(["docker", "pull", POSTGRES_IMAGE])
+        run(["docker", "tag", POSTGRES_IMAGE, cls.image])
+        run(["docker", "volume", "create", cls.pg_volume])
+
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.pg_container,
+                "-e",
+                "POSTGRES_PASSWORD=pgpw",
+                "-e",
+                "POSTGRES_DB=appdb",
+                "-e",
+                "POSTGRES_USER=postgres",
+                "-v",
+                f"{cls.pg_volume}:{POSTGRES_DATA_DIR}",
+                cls.image,
+            ]
+        )
+        wait_for_postgres(cls.pg_container, user="postgres", timeout_s=90)
+
+        run(
+            [
+                "docker",
+                "exec",
+                cls.pg_container,
+                "sh",
+                "-lc",
+                "psql -U postgres -d appdb -c \"CREATE TABLE t (id int primary key, v text); INSERT INTO t VALUES (1,'ok');\"",
+            ]
+        )
+
+        cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
+        write_databases_csv(
+            cls.databases_csv, [(cls.pg_container, "appdb", "postgres", "pgpw")]
+        )
+
+        backup_run(
+            backups_dir=cls.backups_dir,
+            repo_name=cls.repo_name,
+            compose_dir=cls.compose_dir,
+            databases_csv=cls.databases_csv,
+            database_containers=[cls.pg_container],
+            images_no_stop_required=[cls.image],
+        )
+
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=cls.containers, volumes=cls.volumes)
+        run(["docker", "rmi", cls.image], check=False)
+
+    def test_the_registry_host_does_not_pick_the_engine(self) -> None:
+        p = (
+            backup_path(self.backups_dir, self.repo_name, self.version, self.pg_volume)
+            / "sql"
+            / "appdb.backup.sql"
+        )
+        self.assertTrue(p.is_file(), f"Expected a pg_dump at: {p}")
+        self.assertIn("Dumped by pg_dump", p.read_text(encoding="utf-8"))
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/e2e/test_e2e_files_full.py

@@ -1,16 +1,16 @@
 import unittest
 
 from .helpers import (
-    backup_run,
     backup_path,
+    backup_run,
     cleanup_docker,
     create_minimal_compose_dir,
     ensure_empty_dir,
     latest_version_dir,
     require_docker,
+    run,
     unique,
     write_databases_csv,
-    run,
 )
 
 
@@ -30,7 +30,6 @@ class TestE2EFilesFull(unittest.TestCase):
         cls.containers = []
         cls.volumes = [cls.volume_src, cls.volume_dst]
 
-        # create source volume with a file
         run(["docker", "volume", "create", cls.volume_src])
         run(
             [
@@ -50,14 +49,13 @@ class TestE2EFilesFull(unittest.TestCase):
         cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
         write_databases_csv(cls.databases_csv, [])
 
-        # Run backup (files should be copied)
         backup_run(
             backups_dir=cls.backups_dir,
             repo_name=cls.repo_name,
             compose_dir=cls.compose_dir,
             databases_csv=cls.databases_csv,
             database_containers=["dummy-db"],
-            images_no_stop_required=["alpine", "postgres", "mariadb", "mysql"],
+            images_no_stop_required=["alpine:3.20"],
         )
 
         cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
@@ -94,12 +92,9 @@ class TestE2EFilesFull(unittest.TestCase):
                 self.repo_name,
                 "--source-volume",
                 self.volume_src,
-                "--rsync-image",
-                "ghcr.io/kevinveenbirkenbach/alpine-rsync",
             ]
         )
 
-        # verify restored file exists in dst volume
         p = run(
             [
                 "docker",

tests/e2e/test_e2e_files_no_copy.py

@@ -1,16 +1,16 @@
 import unittest
 
 from .helpers import (
-    backup_run,
     backup_path,
+    backup_run,
     cleanup_docker,
     create_minimal_compose_dir,
     ensure_empty_dir,
     latest_version_dir,
     require_docker,
+    run,
     unique,
     write_databases_csv,
-    run,
 )
 
 
@@ -26,9 +26,8 @@ class TestE2EFilesNoCopy(unittest.TestCase):
         cls.repo_name = cls.prefix
 
         cls.volume_src = f"{cls.prefix}-vol-src"
-        cls.volume_dst = f"{cls.prefix}-vol-dst"
+        cls.containers: list[str] = []
-        cls.containers = []
+        cls.volumes = [cls.volume_src]
-        cls.volumes = [cls.volume_src, cls.volume_dst]
 
         run(["docker", "volume", "create", cls.volume_src])
         run(
@@ -48,36 +47,40 @@ class TestE2EFilesNoCopy(unittest.TestCase):
         cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
         write_databases_csv(cls.databases_csv, [])
 
-        # dump-only => NO file rsync backups
+        # only-sql => non-DB volumes are STILL backed up as files
         backup_run(
             backups_dir=cls.backups_dir,
             repo_name=cls.repo_name,
             compose_dir=cls.compose_dir,
             databases_csv=cls.databases_csv,
             database_containers=["dummy-db"],
-            images_no_stop_required=["alpine", "postgres", "mariadb", "mysql"],
+            images_no_stop_required=["alpine:3.20"],
-            dump_only=True,
+            only_sql=True,
         )
 
         cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
 
+        # Wipe the volume to ensure restore actually restores something
+        run(["docker", "volume", "rm", "-f", cls.volume_src])
+        run(["docker", "volume", "create", cls.volume_src])
+
     @classmethod
     def tearDownClass(cls) -> None:
         cleanup_docker(containers=cls.containers, volumes=cls.volumes)
 
-    def test_files_backup_not_present(self) -> None:
+    def test_files_backup_present_for_non_db_volume(self) -> None:
         p = (
             backup_path(self.backups_dir, self.repo_name, self.version, self.volume_src)
             / "files"
         )
-        self.assertFalse(p.exists(), f"Did not expect files backup dir at: {p}")
+        self.assertTrue(p.exists(), f"Expected files backup dir at: {p}")
 
-    def test_restore_files_fails_expected(self) -> None:
+    def test_restore_files_succeeds_and_restores_content(self) -> None:
         p = run(
             [
                 "baudolo-restore",
                 "files",
-                self.volume_dst,
+                self.volume_src,
                 self.hash,
                 self.version,
                 "--backups-dir",
@@ -89,6 +92,27 @@ class TestE2EFilesNoCopy(unittest.TestCase):
         )
         self.assertEqual(
             p.returncode,
-            2,
+            0,
-            f"Expected exitcode 2, got {p.returncode}\nSTDOUT={p.stdout}\nSTDERR={p.stderr}",
+            f"Expected exitcode 0, got {p.returncode}\nSTDOUT={p.stdout}\nSTDERR={p.stderr}",
+        )
+
+        cp = run(
+            [
+                "docker",
+                "run",
+                "--rm",
+                "-v",
+                f"{self.volume_src}:/data",
+                "alpine:3.20",
+                "sh",
+                "-lc",
+                "cat /data/hello.txt",
+            ],
+            capture=True,
+            check=True,
+        )
+        self.assertEqual(
+            cp.stdout.strip(),
+            "hello",
+            f"Unexpected restored content. STDOUT={cp.stdout}\nSTDERR={cp.stderr}",
         )

tests/e2e/test_e2e_images_no_backup_required_early_skip.py

@@ -0,0 +1,123 @@
+import unittest
+
+from .helpers import (
+    backup_path,
+    cleanup_docker,
+    create_minimal_compose_dir,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    write_databases_csv,
+)
+
+
+class TestE2EImagesNoBackupRequiredEarlySkip(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+
+        cls.prefix = unique("baudolo-e2e-early-skip-no-backup-required")
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+
+        # --- Docker resources ---
+        cls.redis_container = f"{cls.prefix}-redis"
+        cls.ignored_volume = f"{cls.prefix}-redis-vol"
+        cls.normal_volume = f"{cls.prefix}-files-vol"
+
+        cls.containers = [cls.redis_container]
+        cls.volumes = [cls.ignored_volume, cls.normal_volume]
+
+        run(["docker", "volume", "create", cls.ignored_volume])
+        run(["docker", "volume", "create", cls.normal_volume])
+
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.redis_container,
+                "-v",
+                f"{cls.ignored_volume}:/data",
+                "redis:alpine",
+            ]
+        )
+
+        # Put deterministic content into the normal volume
+        run(
+            [
+                "docker",
+                "run",
+                "--rm",
+                "-v",
+                f"{cls.normal_volume}:/data",
+                "alpine:3.20",
+                "sh",
+                "-lc",
+                "mkdir -p /data && echo 'hello' > /data/hello.txt",
+            ]
+        )
+
+        # databases.csv required by CLI (can be empty)
+        cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
+        write_databases_csv(cls.databases_csv, [])
+
+        cmd = [
+            "baudolo",
+            "--compose-dir",
+            cls.compose_dir,
+            "--hard-restart-projects",
+            "mailu",
+            "--repo-name",
+            cls.repo_name,
+            "--databases-csv",
+            cls.databases_csv,
+            "--backups-dir",
+            cls.backups_dir,
+            "--database-containers",
+            "dummy-db",
+            "--images-no-stop-required",
+            "redis:alpine",
+            "--images-no-backup-required",
+            "redis:alpine",
+        ]
+        cp = run(cmd, capture=True, check=True)
+        cls.stdout = cp.stdout or ""
+        cls.stderr = cp.stderr or ""
+
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=cls.containers, volumes=cls.volumes)
+
+    def test_ignored_volume_has_no_backup_directory_at_all(self) -> None:
+        p = backup_path(
+            self.backups_dir,
+            self.repo_name,
+            self.version,
+            self.ignored_volume,
+        )
+        self.assertFalse(
+            p.exists(),
+            f"Expected NO backup directory to be created for ignored volume, but found: {p}",
+        )
+
+    def test_normal_volume_is_still_backed_up(self) -> None:
+        p = (
+            backup_path(
+                self.backups_dir,
+                self.repo_name,
+                self.version,
+                self.normal_volume,
+            )
+            / "files"
+            / "hello.txt"
+        )
+        self.assertTrue(p.is_file(), f"Expected backed up file at: {p}")

tests/e2e/test_e2e_mariadb_anonymous_preemption.py

@@ -0,0 +1,166 @@
+"""
+Bug-repro for: mariadb-dump fails with `ERROR 1045 Access denied for user
+'<u>'@'localhost' (using password: YES)` when only '<u>'@'%' is granted and a
+preempting ''@'localhost' user is present.
+
+The fix forces TCP loopback in baudolo.backup.db so the dump matches the
+'<u>'@'%' grant instead of the socket->localhost auth row.
+
+This file:
+- builds the exact preconditions that triggered the production failure,
+- as a NEGATIVE control, runs a socket-based mariadb-dump (== the old code path)
+  and asserts that it fails with the literal 1045 / @'localhost' error,
+- as a POSITIVE proof, calls backup_database() (where the fix lives) against
+  the same DB container and asserts the dump file is produced and contains the
+  seed data.
+
+Note: the volume-rsync stage of baudolo is intentionally NOT exercised here.
+That stage needs root on /var/lib/docker/volumes, which is provided by the
+DinD wrapper in `make test-e2e` but not by an on-host invocation. The bug we
+are verifying is in the DB-dump stage, so testing backup_database() directly
+keeps the assertion focused and the test runnable both on-host and in DinD.
+"""
+
+import tempfile
+import unittest
+from pathlib import Path
+
+import pandas as pd
+
+from baudolo.backup import db as db_mod
+from baudolo.generation import DUMP_SUFFIX, SQL_DIR
+
+from .helpers import (
+    MARIADB_DATA_DIR,
+    MARIADB_IMAGE,
+    cleanup_docker,
+    require_docker,
+    run,
+    unique,
+    wait_for_mariadb,
+    wait_for_mariadb_sql,
+)
+
+
+class TestE2EMariaDBAnonymousPreemption(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        cls.prefix = unique("baudolo-e2e-mariadb-anon")
+        cls.db_container = f"{cls.prefix}-mariadb"
+        cls.db_volume = f"{cls.prefix}-mariadb-vol"
+        cls.containers = [cls.db_container]
+        cls.volumes = [cls.db_volume]
+
+        cls.db_name = "appdb"
+        cls.db_user = "tcponly"
+        cls.db_password = "tcponlypw"
+        cls.root_password = "rootpw"
+
+        run(["docker", "volume", "create", cls.db_volume])
+
+        # Boot WITHOUT MARIADB_USER/MARIADB_PASSWORD/MARIADB_DATABASE so the
+        # entrypoint does not auto-create '<u>'@'%'. We provision the user
+        # explicitly below to mirror the SQL path used by svc-db-mariadb.
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.db_container,
+                "-e",
+                f"MARIADB_ROOT_PASSWORD={cls.root_password}",
+                "-v",
+                f"{cls.db_volume}:{MARIADB_DATA_DIR}",
+                MARIADB_IMAGE,
+            ]
+        )
+
+        wait_for_mariadb(
+            cls.db_container, root_password=cls.root_password, timeout_s=120
+        )
+
+        # Provision: '<u>'@'%' (the app/backup grant) + anonymous ''@'localhost'
+        # (the preemption trigger). Mirrors the production state that produced
+        # `ERROR 1045 ... '<u>'@'localhost' (using password: YES)`.
+        bootstrap_sql = (
+            f"CREATE DATABASE {cls.db_name};"
+            f"CREATE USER '{cls.db_user}'@'%' IDENTIFIED BY '{cls.db_password}';"
+            f"GRANT ALL PRIVILEGES ON {cls.db_name}.* TO '{cls.db_user}'@'%';"
+            f"CREATE USER ''@'localhost' IDENTIFIED BY 'anonpw-not-{cls.db_password}';"
+            "FLUSH PRIVILEGES;"
+            f"CREATE TABLE {cls.db_name}.t (id INT PRIMARY KEY, v VARCHAR(50));"
+            f"INSERT INTO {cls.db_name}.t VALUES (1,'ok');"
+        )
+        run(
+            [
+                "docker",
+                "exec",
+                cls.db_container,
+                "sh",
+                "-lc",
+                f'mariadb -uroot --protocol=socket -e "{bootstrap_sql}"',
+            ]
+        )
+
+        # Sanity: '<u>' can log in over TCP (matches '%'). If THIS fails,
+        # the precondition for the fix to even apply is broken.
+        wait_for_mariadb_sql(
+            cls.db_container, user=cls.db_user, password=cls.db_password, timeout_s=60
+        )
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=cls.containers, volumes=cls.volumes)
+
+    def test_negative_control_socket_dump_fails_with_1045(self) -> None:
+        # Reproduces the OLD code path (no -h/--protocol). MUST fail with 1045
+        # under the configured preemption. If this ever starts passing, either
+        # the MariaDB auth semantics changed or the anonymous-user setup did
+        # not take effect — in both cases the positive test below loses its
+        # ability to discriminate "fix works" vs "bug never reproduced".
+        p = run(
+            [
+                "docker",
+                "exec",
+                self.db_container,
+                "sh",
+                "-lc",
+                f"mariadb-dump -u{self.db_user} -p{self.db_password} {self.db_name}",
+            ],
+            capture=True,
+            check=False,
+        )
+        self.assertNotEqual(p.returncode, 0, "socket-based dump unexpectedly succeeded")
+        self.assertIn("1045", (p.stderr or "") + (p.stdout or ""))
+        self.assertIn("@'localhost'", (p.stderr or "") + (p.stdout or ""))
+
+    def test_backup_database_succeeds_with_tcp_fix(self) -> None:
+        # Drives the function where the fix lives. No rsync, no privileged
+        # paths — just the dump that the negative-control proved is failing
+        # under the same preemption setup.
+        with tempfile.TemporaryDirectory() as volume_dir:
+            df = pd.DataFrame(
+                [(self.db_container, self.db_name, self.db_user, self.db_password)],
+                columns=["instance", "database", "username", "password"],
+            )
+            produced = db_mod.backup_database(
+                container=self.db_container,
+                volume_dir=volume_dir,
+                db_type="mariadb",
+                dump_tool="mariadb-dump",
+                databases_df=df,
+                database_containers=[self.db_container],
+            )
+            self.assertTrue(produced, "backup_database did not produce a dump")
+            dump_path = Path(volume_dir) / SQL_DIR / f"{self.db_name}{DUMP_SUFFIX}"
+            self.assertTrue(dump_path.is_file(), f"expected dump at {dump_path}")
+            with dump_path.open(encoding="utf-8", errors="replace") as f:
+                content = f.read()
+            self.assertIn("INSERT INTO", content)
+            self.assertIn("'ok'", content)
+
+
+if __name__ == "__main__":
+    unittest.main(verbosity=2)

tests/e2e/test_e2e_mariadb_full.py

@@ -1,19 +1,20 @@
-# tests/e2e/test_e2e_mariadb_full.py
 import unittest
 
 from .helpers import (
-    backup_run,
+    MARIADB_DATA_DIR,
+    MARIADB_IMAGE,
     backup_path,
+    backup_run,
     cleanup_docker,
     create_minimal_compose_dir,
     ensure_empty_dir,
     latest_version_dir,
     require_docker,
-    unique,
-    write_databases_csv,
     run,
+    unique,
     wait_for_mariadb,
     wait_for_mariadb_sql,
+    write_databases_csv,
 )
 
 
@@ -56,8 +57,8 @@ class TestE2EMariaDBFull(unittest.TestCase):
                 "-e",
                 f"MARIADB_PASSWORD={cls.db_password}",
                 "-v",
-                f"{cls.db_volume}:/var/lib/mysql",
+                f"{cls.db_volume}:{MARIADB_DATA_DIR}",
-                "mariadb:11",
+                MARIADB_IMAGE,
             ]
         )
 
@@ -69,7 +70,6 @@ class TestE2EMariaDBFull(unittest.TestCase):
             cls.db_container, user=cls.db_user, password=cls.db_password, timeout_s=90
         )
 
-        # Create table + data via the dedicated user (TCP)
         run(
             [
                 "docker",
@@ -77,9 +77,11 @@ class TestE2EMariaDBFull(unittest.TestCase):
                 cls.db_container,
                 "sh",
                 "-lc",
-                f"mariadb -h 127.0.0.1 -u{cls.db_user} -p{cls.db_password} "
+                (
-                f'-e "CREATE TABLE {cls.db_name}.t (id INT PRIMARY KEY, v VARCHAR(50)); '
+                    f"mariadb -h 127.0.0.1 -u{cls.db_user} -p{cls.db_password} "
-                f"INSERT INTO {cls.db_name}.t VALUES (1,'ok');\"",
+                    f'-e "CREATE TABLE {cls.db_name}.t (id INT PRIMARY KEY, v VARCHAR(50)); '
+                    f"INSERT INTO {cls.db_name}.t VALUES (1,'ok');\""
+                ),
             ]
         )
 
@@ -97,7 +99,7 @@ class TestE2EMariaDBFull(unittest.TestCase):
             compose_dir=cls.compose_dir,
             databases_csv=cls.databases_csv,
             database_containers=[cls.db_container],
-            images_no_stop_required=["mariadb", "mysql", "alpine", "postgres"],
+            images_no_stop_required=[MARIADB_IMAGE],
         )
 
         cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
@@ -110,8 +112,10 @@ class TestE2EMariaDBFull(unittest.TestCase):
                 cls.db_container,
                 "sh",
                 "-lc",
-                f"mariadb -h 127.0.0.1 -u{cls.db_user} -p{cls.db_password} "
+                (
-                f'-e "DROP TABLE {cls.db_name}.t;"',
+                    f"mariadb -h 127.0.0.1 -u{cls.db_user} -p{cls.db_password} "
+                    f'-e "DROP TABLE {cls.db_name}.t;"'
+                ),
             ]
         )
 
@@ -159,8 +163,10 @@ class TestE2EMariaDBFull(unittest.TestCase):
                 self.db_container,
                 "sh",
                 "-lc",
-                f"mariadb -h 127.0.0.1 -u{self.db_user} -p{self.db_password} "
+                (
-                f'-N -e "SELECT v FROM {self.db_name}.t WHERE id=1;"',
+                    f"mariadb -h 127.0.0.1 -u{self.db_user} -p{self.db_password} "
+                    f'-N -e "SELECT v FROM {self.db_name}.t WHERE id=1;"'
+                ),
             ]
         )
         self.assertEqual((p.stdout or "").strip(), "ok")

tests/e2e/test_e2e_mariadb_no_copy.py

@@ -1,19 +1,20 @@
-# tests/e2e/test_e2e_mariadb_no_copy.py
 import unittest
 
 from .helpers import (
-    backup_run,
+    MARIADB_DATA_DIR,
+    MARIADB_IMAGE,
     backup_path,
+    backup_run,
     cleanup_docker,
     create_minimal_compose_dir,
     ensure_empty_dir,
     latest_version_dir,
     require_docker,
-    unique,
-    write_databases_csv,
     run,
+    unique,
     wait_for_mariadb,
     wait_for_mariadb_sql,
+    write_databases_csv,
 )
 
 
@@ -55,8 +56,8 @@ class TestE2EMariaDBNoCopy(unittest.TestCase):
                 "-e",
                 f"MARIADB_PASSWORD={cls.db_password}",
                 "-v",
-                f"{cls.db_volume}:/var/lib/mysql",
+                f"{cls.db_volume}:{MARIADB_DATA_DIR}",
-                "mariadb:11",
+                MARIADB_IMAGE,
             ]
         )
 
@@ -67,7 +68,6 @@ class TestE2EMariaDBNoCopy(unittest.TestCase):
             cls.db_container, user=cls.db_user, password=cls.db_password, timeout_s=90
         )
 
-        # Create table + data (TCP)
         run(
             [
                 "docker",
@@ -75,9 +75,11 @@ class TestE2EMariaDBNoCopy(unittest.TestCase):
                 cls.db_container,
                 "sh",
                 "-lc",
-                f"mariadb -h 127.0.0.1 -u{cls.db_user} -p{cls.db_password} "
+                (
-                f'-e "CREATE TABLE {cls.db_name}.t (id INT PRIMARY KEY, v VARCHAR(50)); '
+                    f"mariadb -h 127.0.0.1 -u{cls.db_user} -p{cls.db_password} "
-                f"INSERT INTO {cls.db_name}.t VALUES (1,'ok');\"",
+                    f'-e "CREATE TABLE {cls.db_name}.t (id INT PRIMARY KEY, v VARCHAR(50)); '
+                    f"INSERT INTO {cls.db_name}.t VALUES (1,'ok');\""
+                ),
             ]
         )
 
@@ -87,15 +89,15 @@ class TestE2EMariaDBNoCopy(unittest.TestCase):
             [(cls.db_container, cls.db_name, cls.db_user, cls.db_password)],
         )
 
-        # dump-only => no files
+        # only-sql => no files
         backup_run(
             backups_dir=cls.backups_dir,
             repo_name=cls.repo_name,
             compose_dir=cls.compose_dir,
             databases_csv=cls.databases_csv,
             database_containers=[cls.db_container],
-            images_no_stop_required=["mariadb", "mysql", "alpine", "postgres"],
+            images_no_stop_required=[MARIADB_IMAGE],
-            dump_only=True,
+            only_sql=True,
         )
 
         cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
@@ -108,8 +110,10 @@ class TestE2EMariaDBNoCopy(unittest.TestCase):
                 cls.db_container,
                 "sh",
                 "-lc",
-                f"mariadb -h 127.0.0.1 -u{cls.db_user} -p{cls.db_password} "
+                (
-                f'-e "DROP TABLE {cls.db_name}.t;"',
+                    f"mariadb -h 127.0.0.1 -u{cls.db_user} -p{cls.db_password} "
+                    f'-e "DROP TABLE {cls.db_name}.t;"'
+                ),
             ]
         )
 
@@ -156,8 +160,10 @@ class TestE2EMariaDBNoCopy(unittest.TestCase):
                 self.db_container,
                 "sh",
                 "-lc",
-                f"mariadb -h 127.0.0.1 -u{self.db_user} -p{self.db_password} "
+                (
-                f'-N -e "SELECT v FROM {self.db_name}.t WHERE id=1;"',
+                    f"mariadb -h 127.0.0.1 -u{self.db_user} -p{self.db_password} "
+                    f'-N -e "SELECT v FROM {self.db_name}.t WHERE id=1;"'
+                ),
             ]
         )
         self.assertEqual((p.stdout or "").strip(), "ok")

tests/e2e/test_e2e_only_files.py

@@ -0,0 +1,116 @@
+"""--only-files backs a database up as a file tree and asks for no credentials.
+
+The run deliberately passes no --databases-csv at all: a host that only copies
+files has no reason to hold database passwords, and requiring the file would
+make the flag useless there.
+"""
+
+import unittest
+
+from .helpers import (
+    POSTGRES_DATA_DIR,
+    POSTGRES_IMAGE,
+    backup_path,
+    cleanup_docker,
+    create_minimal_compose_dir,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    wait_for_postgres,
+)
+
+MARKER = "only-files-marker"
+
+
+class TestE2EOnlyFiles(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        cls.prefix = unique("baudolo-e2e-only-files")
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+
+        cls.pg_container = f"{cls.prefix}-pg"
+        cls.pg_volume = f"{cls.prefix}-pg-vol"
+        cls.containers = [cls.pg_container]
+        cls.volumes = [cls.pg_volume]
+
+        run(["docker", "volume", "create", cls.pg_volume])
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.pg_container,
+                "-e",
+                "POSTGRES_PASSWORD=pgpw",
+                "-e",
+                "POSTGRES_DB=appdb",
+                "-e",
+                "POSTGRES_USER=postgres",
+                "-v",
+                f"{cls.pg_volume}:{POSTGRES_DATA_DIR}",
+                POSTGRES_IMAGE,
+            ]
+        )
+        wait_for_postgres(cls.pg_container, user="postgres", timeout_s=90)
+        run(
+            [
+                "docker",
+                "exec",
+                cls.pg_container,
+                "sh",
+                "-lc",
+                f"echo '{MARKER}' > {POSTGRES_DATA_DIR}/marker.txt",
+            ]
+        )
+
+        cp = run(
+            [
+                "baudolo",
+                "--compose-dir",
+                cls.compose_dir,
+                "--repo-name",
+                cls.repo_name,
+                "--backups-dir",
+                cls.backups_dir,
+                "--images-no-stop-required",
+                POSTGRES_IMAGE,
+                "--only-files",
+            ],
+            capture=True,
+            check=True,
+        )
+        cls.stdout = cp.stdout or ""
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=cls.containers, volumes=cls.volumes)
+
+    def _volume_dir(self):
+        return backup_path(
+            self.backups_dir, self.repo_name, self.version, self.pg_volume
+        )
+
+    def test_the_database_volume_is_backed_up_as_files(self) -> None:
+        marker = self._volume_dir() / "files" / "marker.txt"
+        self.assertTrue(marker.is_file(), f"expected a file backup at {marker}")
+        self.assertEqual(marker.read_text(encoding="utf-8").strip(), MARKER)
+
+    def test_no_dump_is_written(self) -> None:
+        sql_dir = self._volume_dir() / "sql"
+        dumps = list(sql_dir.glob("*.sql")) if sql_dir.exists() else []
+        self.assertEqual(dumps, [], f"did not expect any dump, found: {dumps}")
+
+    def test_the_missing_databases_csv_is_not_reported(self) -> None:
+        self.assertNotIn("databases.csv", self.stdout)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/e2e/test_e2e_only_sql_fallback_to_files.py

@@ -0,0 +1,204 @@
+import json
+import unittest
+
+from baudolo.generation import FILES_DIR, MANIFEST_FILE, MANIFEST_SCHEMA, SQL_DIR
+
+from .helpers import (
+    POSTGRES_DATA_DIR,
+    POSTGRES_IMAGE,
+    backup_path,
+    cleanup_docker,
+    create_minimal_compose_dir,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    wait_for_postgres,
+    write_databases_csv,
+)
+
+
+class TestE2EOnlySqlFallbackToFiles(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        cls.prefix = unique("baudolo-e2e-only-sql-fallback")
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+
+        cls.pg_container = f"{cls.prefix}-pg"
+        cls.pg_volume = f"{cls.prefix}-pg-vol"
+        cls.restore_volume = f"{cls.prefix}-restore-vol"
+
+        cls.containers = [cls.pg_container]
+        cls.volumes = [cls.pg_volume, cls.restore_volume]
+
+        run(["docker", "volume", "create", cls.pg_volume])
+
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.pg_container,
+                "-e",
+                "POSTGRES_PASSWORD=pgpw",
+                "-e",
+                "POSTGRES_DB=appdb",
+                "-e",
+                "POSTGRES_USER=postgres",
+                "-v",
+                f"{cls.pg_volume}:{POSTGRES_DATA_DIR}",
+                POSTGRES_IMAGE,
+            ]
+        )
+        wait_for_postgres(cls.pg_container, user="postgres", timeout_s=90)
+
+        # Add a deterministic marker file into the volume
+        cls.marker = "only-sql-fallback-marker"
+        run(
+            [
+                "docker",
+                "exec",
+                cls.pg_container,
+                "sh",
+                "-lc",
+                f"echo '{cls.marker}' > {POSTGRES_DATA_DIR}/marker.txt",
+            ]
+        )
+
+        # databases.csv WITHOUT matching entry for this instance -> should skip dump
+        cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
+        write_databases_csv(cls.databases_csv, [])  # empty except header
+
+        # Run baudolo with --only-sql and a DB container present:
+        # Expected: WARNING + FALLBACK to file backup (files/ must exist)
+        cmd = [
+            "baudolo",
+            "--compose-dir",
+            cls.compose_dir,
+            "--hard-restart-projects",
+            "mailu",
+            "--repo-name",
+            cls.repo_name,
+            "--databases-csv",
+            cls.databases_csv,
+            "--backups-dir",
+            cls.backups_dir,
+            "--database-containers",
+            cls.pg_container,
+            "--images-no-stop-required",
+            POSTGRES_IMAGE,
+            "--only-sql",
+        ]
+        cp = run(cmd, capture=True, check=True)
+
+        cls.stdout = cp.stdout or ""
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+
+        # Restore files into a fresh volume to prove file backup happened
+        run(["docker", "volume", "create", cls.restore_volume])
+        run(
+            [
+                "baudolo-restore",
+                "files",
+                cls.restore_volume,
+                cls.hash,
+                cls.version,
+                "--backups-dir",
+                cls.backups_dir,
+                "--repo-name",
+                cls.repo_name,
+                "--source-volume",
+                cls.pg_volume,
+            ]
+        )
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=cls.containers, volumes=cls.volumes)
+
+    def test_warns_about_missing_dump_in_dump_only_mode(self) -> None:
+        self.assertIn(
+            "WARNING: only-sql requested but no DB dump was produced",
+            self.stdout,
+            f"Expected warning in baudolo output. STDOUT:\n{self.stdout}",
+        )
+
+    def test_files_backup_exists_due_to_fallback(self) -> None:
+        p = (
+            backup_path(
+                self.backups_dir,
+                self.repo_name,
+                self.version,
+                self.pg_volume,
+            )
+            / "files"
+        )
+        self.assertTrue(p.is_dir(), f"Expected files backup dir at: {p}")
+
+    def test_sql_dump_not_present(self) -> None:
+        # There should be no sql dumps because databases.csv had no matching entry.
+        sql_dir = (
+            backup_path(
+                self.backups_dir,
+                self.repo_name,
+                self.version,
+                self.pg_volume,
+            )
+            / "sql"
+        )
+        # Could exist (dir created) in some edge cases, but should contain no *.sql dumps.
+        if sql_dir.exists():
+            dumps = list(sql_dir.glob("*.sql"))
+            self.assertEqual(
+                len(dumps),
+                0,
+                f"Did not expect SQL dump files, found: {dumps}",
+            )
+
+    def manifest(self) -> dict:
+        generation = backup_path(
+            self.backups_dir, self.repo_name, self.version, self.pg_volume
+        ).parent
+        return json.loads((generation / MANIFEST_FILE).read_text(encoding="utf-8"))
+
+    def test_the_manifest_records_the_volume_as_a_database_left_undumped(self) -> None:
+        """The fallback is invisible in the tree: files/ looks like any copy."""
+        self.assertEqual(
+            self.manifest()["volumes"][self.pg_volume],
+            {"database": True, "dumped": False, "engine": "postgres"},
+        )
+
+    def test_the_manifest_layout_names_where_the_payload_really_landed(self) -> None:
+        layout = self.manifest()["layout"]
+        volume_dir = backup_path(
+            self.backups_dir, self.repo_name, self.version, self.pg_volume
+        )
+        self.assertTrue((volume_dir / layout["files_dir"]).is_dir())
+        self.assertEqual(layout["files_dir"], FILES_DIR)
+        self.assertEqual(layout["sql_dir"], SQL_DIR)
+
+    def test_the_manifest_states_a_schema_a_reader_can_check(self) -> None:
+        self.assertEqual(self.manifest()["schema"], MANIFEST_SCHEMA)
+
+    def test_restored_files_contain_marker(self) -> None:
+        p = run(
+            [
+                "docker",
+                "run",
+                "--rm",
+                "-v",
+                f"{self.restore_volume}:/data",
+                "alpine:3.20",
+                "sh",
+                "-lc",
+                "cat /data/marker.txt",
+            ]
+        )
+        self.assertEqual((p.stdout or "").strip(), self.marker)

tests/e2e/test_e2e_only_sql_mixed_run.py

@@ -0,0 +1,202 @@
+import json
+import unittest
+
+from baudolo.generation import MANIFEST_FILE
+
+from .helpers import (
+    POSTGRES_DATA_DIR,
+    POSTGRES_IMAGE,
+    backup_path,
+    cleanup_docker,
+    create_minimal_compose_dir,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    wait_for_postgres,
+    write_databases_csv,
+)
+
+
+class TestE2EOnlySqlMixedRun(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        cls.prefix = unique("baudolo-e2e-only-sql-mixed-run")
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+
+        # --- Volumes ---
+        cls.db_volume = f"{cls.prefix}-vol-db"
+        cls.files_volume = f"{cls.prefix}-vol-files"
+
+        # Track for cleanup
+        cls.containers: list[str] = []
+        cls.volumes = [cls.db_volume, cls.files_volume]
+
+        run(["docker", "volume", "create", cls.db_volume])
+        run(["docker", "volume", "create", cls.files_volume])
+
+        # Put a marker into the non-db volume
+        run(
+            [
+                "docker",
+                "run",
+                "--rm",
+                "-v",
+                f"{cls.files_volume}:/data",
+                "alpine:3.20",
+                "sh",
+                "-lc",
+                "echo 'hello-non-db' > /data/hello.txt",
+            ]
+        )
+
+        # --- Start Postgres container using the DB volume ---
+        cls.pg_container = f"{cls.prefix}-pg"
+        cls.containers.append(cls.pg_container)
+
+        cls.pg_password = "postgres"
+        cls.pg_db = "testdb"
+        cls.pg_user = "postgres"
+
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.pg_container,
+                "-e",
+                f"POSTGRES_PASSWORD={cls.pg_password}",
+                "-v",
+                f"{cls.db_volume}:{POSTGRES_DATA_DIR}",
+                POSTGRES_IMAGE,
+            ]
+        )
+        wait_for_postgres(cls.pg_container, user="postgres", timeout_s=90)
+
+        # Create deterministic content in DB so dump is non-empty
+        run(
+            [
+                "docker",
+                "exec",
+                cls.pg_container,
+                "sh",
+                "-lc",
+                f'psql -U postgres -c "CREATE DATABASE {cls.pg_db};" || true',
+            ],
+            check=True,
+        )
+        run(
+            [
+                "docker",
+                "exec",
+                cls.pg_container,
+                "sh",
+                "-lc",
+                (
+                    f"psql -U postgres -d {cls.pg_db} -c "
+                    '"CREATE TABLE IF NOT EXISTS t (id INT PRIMARY KEY, v TEXT);'
+                    "INSERT INTO t(id,v) VALUES (1,'hello-db') "
+                    'ON CONFLICT (id) DO UPDATE SET v=EXCLUDED.v;"'
+                ),
+            ],
+            check=True,
+        )
+
+        # databases.csv with an entry => dump should succeed
+        cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
+        write_databases_csv(
+            cls.databases_csv,
+            [(cls.pg_container, cls.pg_db, cls.pg_user, cls.pg_password)],
+        )
+
+        cmd = [
+            "baudolo",
+            "--compose-dir",
+            cls.compose_dir,
+            "--databases-csv",
+            cls.databases_csv,
+            "--database-containers",
+            cls.pg_container,
+            "--images-no-stop-required",
+            POSTGRES_IMAGE,
+            "--only-sql",
+            "--backups-dir",
+            cls.backups_dir,
+            "--repo-name",
+            cls.repo_name,
+        ]
+        cp = run(cmd, capture=True, check=True)
+        cls.stdout = cp.stdout
+        cls.stderr = cp.stderr
+
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=cls.containers, volumes=cls.volumes)
+
+    def test_db_volume_has_dump_and_no_files_dir(self) -> None:
+        base = backup_path(
+            self.backups_dir, self.repo_name, self.version, self.db_volume
+        )
+
+        dumps = base / "sql"
+        files = base / "files"
+
+        self.assertTrue(dumps.exists(), f"Expected dumps dir for DB volume at: {dumps}")
+        self.assertFalse(
+            files.exists(),
+            f"Did not expect files dir for DB volume when dump succeeded at: {files}",
+        )
+
+        # Optional: at least one dump file exists
+        dump_files = list(dumps.glob("*.sql")) + list(dumps.glob("*.sql.gz"))
+        self.assertTrue(
+            dump_files,
+            f"Expected at least one SQL dump file in {dumps}, found none.",
+        )
+
+    def test_non_db_volume_has_files_dir(self) -> None:
+        base = backup_path(
+            self.backups_dir, self.repo_name, self.version, self.files_volume
+        )
+        files = base / "files"
+        self.assertTrue(
+            files.exists(),
+            f"Expected files dir for non-DB volume at: {files}",
+        )
+
+    def test_only_sql_does_not_disable_non_db_files_backup(self) -> None:
+        # Regression guard: even with --only-sql, non-DB volumes must still be backed up as files
+        base = backup_path(
+            self.backups_dir, self.repo_name, self.version, self.files_volume
+        )
+        self.assertTrue(
+            (base / "files").exists(),
+            f"Expected non-DB volume files backup to exist at: {base / 'files'}",
+        )
+
+    def manifest(self) -> dict:
+        generation = backup_path(
+            self.backups_dir, self.repo_name, self.version, self.db_volume
+        ).parent
+        return json.loads((generation / MANIFEST_FILE).read_text(encoding="utf-8"))
+
+    def test_the_manifest_records_the_dumped_volume_as_dumped(self) -> None:
+        self.assertEqual(
+            self.manifest()["volumes"][self.db_volume],
+            {"database": True, "dumped": True, "engine": "postgres"},
+        )
+
+    def test_the_manifest_records_the_plain_volume_as_no_database(self) -> None:
+        self.assertEqual(
+            self.manifest()["volumes"][self.files_volume],
+            {"database": False, "dumped": False, "engine": None},
+        )

tests/e2e/test_e2e_postgres_cluster_restore.py

@@ -0,0 +1,215 @@
+import unittest
+
+from .helpers import (
+    POSTGRES_DATA_DIR,
+    POSTGRES_IMAGE,
+    backup_path,
+    backup_run,
+    cleanup_docker,
+    create_minimal_compose_dir,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    wait_for_postgres,
+    write_databases_csv,
+)
+
+# One statement per entry: psql wraps a multi-statement -c in a transaction,
+# and CREATE DATABASE is forbidden inside one.
+SEED_SQL = (
+    "CREATE ROLE app LOGIN PASSWORD 'apppw'",
+    "CREATE DATABASE first OWNER app",
+    "CREATE DATABASE second OWNER app",
+)
+SIBLING_SQL = (
+    "CREATE ROLE neighbour LOGIN PASSWORD 'neighbourpw'",
+    "CREATE DATABASE sibling OWNER neighbour",
+)
+SIBLING_PAYLOAD = "CREATE TABLE t (v text); INSERT INTO t VALUES ('sibling-payload');"
+FIRST_SQL = "CREATE TABLE t (v text); INSERT INTO t VALUES ('first-payload');"
+SECOND_SQL = "CREATE TABLE t (v text); INSERT INTO t VALUES ('second-payload');"
+
+
+class TestE2EPostgresClusterRestore(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        cls.prefix = unique("baudolo-e2e-pg-cluster")
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+
+        cls.pg_container = f"{cls.prefix}-pg"
+        cls.pg_volume = f"{cls.prefix}-pg-vol"
+        cls.containers = [cls.pg_container]
+        cls.volumes = [cls.pg_volume]
+
+        run(["docker", "volume", "create", cls.pg_volume])
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.pg_container,
+                "-e",
+                "POSTGRES_PASSWORD=pgpw",
+                "-v",
+                f"{cls.pg_volume}:{POSTGRES_DATA_DIR}",
+                POSTGRES_IMAGE,
+            ]
+        )
+        wait_for_postgres(cls.pg_container, user="postgres")
+
+        for statement in SEED_SQL:
+            cls._psql("postgres", statement)
+        cls._psql("first", FIRST_SQL)
+        cls._psql("second", SECOND_SQL)
+
+        cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
+        write_databases_csv(
+            cls.databases_csv, [(cls.pg_container, "*", "postgres", "pgpw")]
+        )
+
+        backup_run(
+            backups_dir=cls.backups_dir,
+            repo_name=cls.repo_name,
+            compose_dir=cls.compose_dir,
+            databases_csv=cls.databases_csv,
+            database_containers=[cls.pg_container],
+            images_no_stop_required=[POSTGRES_IMAGE],
+        )
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+        cls.dump = (
+            backup_path(cls.backups_dir, cls.repo_name, cls.version, cls.pg_volume)
+            / "sql"
+            / f"{cls.pg_container}.cluster.backup.sql"
+        )
+
+        run(
+            [
+                "baudolo-restore",
+                "cluster",
+                cls.pg_volume,
+                cls.hash,
+                cls.version,
+                "--backups-dir",
+                cls.backups_dir,
+                "--repo-name",
+                cls.repo_name,
+                "--container",
+                cls.pg_container,
+                "--instance",
+                cls.pg_container,
+                "--db-user",
+                "postgres",
+                "--db-password",
+                "pgpw",
+                "--empty",
+            ]
+        )
+
+        for statement in SIBLING_SQL:
+            cls._psql("postgres", statement)
+        cls._psql("sibling", SIBLING_PAYLOAD)
+        cls.refused = run(cls._restore_argv(), check=False)
+
+    @classmethod
+    def _restore_argv(cls) -> list:
+        return [
+            "baudolo-restore",
+            "cluster",
+            cls.pg_volume,
+            cls.hash,
+            cls.version,
+            "--backups-dir",
+            cls.backups_dir,
+            "--repo-name",
+            cls.repo_name,
+            "--container",
+            cls.pg_container,
+            "--instance",
+            cls.pg_container,
+            "--db-user",
+            "postgres",
+            "--db-password",
+            "pgpw",
+            "--empty",
+        ]
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=cls.containers, volumes=cls.volumes)
+
+    @classmethod
+    def _psql(cls, database: str, sql: str) -> str:
+        p = run(
+            [
+                "docker",
+                "exec",
+                cls.pg_container,
+                "sh",
+                "-lc",
+                f'psql -U postgres -d {database} -t -A -c "{sql}"',
+            ]
+        )
+        return (p.stdout or "").strip()
+
+    def test_the_backup_wrote_a_cluster_dump(self) -> None:
+        self.assertTrue(self.dump.is_file(), f"no cluster dump at {self.dump}")
+
+    def test_the_preclean_really_dropped_a_populated_cluster(self) -> None:
+        self.assertEqual(self._psql("first", "SELECT v FROM t"), "first-payload")
+
+    def test_a_second_empty_is_refused_once_a_foreign_database_exists(self) -> None:
+        self.assertNotEqual(self.refused.returncode, 0, self.refused.stdout)
+        self.assertIn("sibling", self.refused.stderr)
+
+    def test_the_refusal_left_the_foreign_database_alone(self) -> None:
+        self.assertEqual(self._psql("sibling", "SELECT v FROM t"), "sibling-payload")
+
+    def test_the_refusal_dropped_nothing_of_its_own(self) -> None:
+        self.assertEqual(self._psql("first", "SELECT v FROM t"), "first-payload")
+
+    def test_both_databases_are_back(self) -> None:
+        listed = self._psql(
+            "postgres",
+            "SELECT datname FROM pg_database WHERE datname IN ('first','second') ORDER BY 1",
+        )
+        self.assertEqual(listed.split(), ["first", "second"])
+
+    def test_each_database_carries_its_own_payload(self) -> None:
+        self.assertEqual(self._psql("first", "SELECT v FROM t"), "first-payload")
+        self.assertEqual(self._psql("second", "SELECT v FROM t"), "second-payload")
+
+    def test_the_superusers_own_create_was_filtered(self) -> None:
+        self.assertEqual(
+            self._psql(
+                "postgres", "SELECT rolsuper FROM pg_roles WHERE rolname = 'postgres'"
+            ),
+            "t",
+        )
+
+    def test_the_owning_role_is_back(self) -> None:
+        self.assertEqual(
+            self._psql(
+                "postgres", "SELECT rolname FROM pg_roles WHERE rolname = 'app'"
+            ),
+            "app",
+        )
+
+    def test_ownership_survived(self) -> None:
+        self.assertEqual(
+            self._psql(
+                "postgres",
+                "SELECT pg_get_userbyid(datdba) FROM pg_database WHERE datname = 'first'",
+            ),
+            "app",
+        )
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/e2e/test_e2e_postgres_empty_drop_hard.py

@@ -0,0 +1,185 @@
+import unittest
+
+from .helpers import (
+    POSTGRES_DATA_DIR,
+    POSTGRES_IMAGE,
+    backup_run,
+    cleanup_docker,
+    create_minimal_compose_dir,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    wait_for_postgres,
+    write_databases_csv,
+)
+
+# The scenario the --empty pre-clean must survive: a non-public user schema
+# plus one object of every class the discovery SELECT enumerates. Restore
+# --empty runs against the still-populated DB (no wipe), so the pre-clean must
+# drop discourse_functions too or the dump's CREATE SCHEMA aborts the replay
+# under ON_ERROR_STOP; the old public-only DROP left it and broke discourse.
+# The f()/f(int) pair reproduces discourse's overload abort ("function name
+# is not unique") and english_stem_nostop reproduces taiga's text search
+# dictionary abort (duplicate pg_ts_dict_dictname_index).
+SCENARIO_SQL = (
+    "CREATE SCHEMA discourse_functions;"
+    "CREATE TABLE discourse_functions.helper (id int);"
+    "INSERT INTO discourse_functions.helper VALUES (1);"
+    "CREATE TABLE public.t (id int primary key, v text);"
+    "INSERT INTO public.t VALUES (1, 'ok');"
+    "CREATE VIEW public.t_view AS SELECT * FROM public.t;"
+    "CREATE SEQUENCE public.s;"
+    "CREATE TYPE public.mood AS ENUM ('ok', 'bad');"
+    "CREATE FUNCTION public.f() RETURNS int LANGUAGE sql AS 'SELECT 1';"
+    "CREATE FUNCTION public.f(i int) RETURNS int LANGUAGE sql AS 'SELECT i';"
+    "CREATE COLLATION public.c (locale = 'C');"
+    "CREATE TEXT SEARCH DICTIONARY public.english_stem_nostop"
+    " (Template = snowball, Language = english);"
+    "CREATE TEXT SEARCH CONFIGURATION public.english_nostop (COPY = english);"
+    "ALTER TEXT SEARCH CONFIGURATION public.english_nostop"
+    " ALTER MAPPING FOR asciiword WITH public.english_stem_nostop;"
+)
+
+
+class TestE2EPostgresEmptyDropHard(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        cls.prefix = unique("baudolo-e2e-postgres-empty-drop-hard")
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+
+        cls.pg_container = f"{cls.prefix}-pg"
+        cls.pg_volume = f"{cls.prefix}-pg-vol"
+        cls.containers = [cls.pg_container]
+        cls.volumes = [cls.pg_volume]
+
+        run(["docker", "volume", "create", cls.pg_volume])
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.pg_container,
+                "-e",
+                "POSTGRES_PASSWORD=pgpw",
+                "-e",
+                "POSTGRES_DB=appdb",
+                "-e",
+                "POSTGRES_USER=postgres",
+                "-v",
+                f"{cls.pg_volume}:{POSTGRES_DATA_DIR}",
+                POSTGRES_IMAGE,
+            ]
+        )
+        wait_for_postgres(cls.pg_container, user="postgres", timeout_s=90)
+
+        run(
+            [
+                "docker",
+                "exec",
+                cls.pg_container,
+                "sh",
+                "-lc",
+                f'psql -U postgres -d appdb -v ON_ERROR_STOP=1 -c "{SCENARIO_SQL}"',
+            ]
+        )
+
+        cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
+        write_databases_csv(
+            cls.databases_csv, [(cls.pg_container, "appdb", "postgres", "pgpw")]
+        )
+        backup_run(
+            backups_dir=cls.backups_dir,
+            repo_name=cls.repo_name,
+            compose_dir=cls.compose_dir,
+            databases_csv=cls.databases_csv,
+            database_containers=[cls.pg_container],
+            images_no_stop_required=[POSTGRES_IMAGE],
+        )
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+
+        # No wipe: restore --empty must pre-clean the fully-populated DB
+        # (incl. the non-public schema) before replaying the dump.
+        run(
+            [
+                "baudolo-restore",
+                "postgres",
+                cls.pg_volume,
+                cls.hash,
+                cls.version,
+                "--backups-dir",
+                cls.backups_dir,
+                "--repo-name",
+                cls.repo_name,
+                "--container",
+                cls.pg_container,
+                "--db-name",
+                "appdb",
+                "--db-user",
+                "postgres",
+                "--db-password",
+                "pgpw",
+                "--empty",
+            ]
+        )
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=cls.containers, volumes=cls.volumes)
+
+    def _scalar(self, sql: str) -> str:
+        p = run(
+            [
+                "docker",
+                "exec",
+                self.pg_container,
+                "sh",
+                "-lc",
+                f'psql -U postgres -d appdb -t -A -c "{sql}"',
+            ]
+        )
+        return (p.stdout or "").strip()
+
+    def test_public_data_restored(self) -> None:
+        self.assertEqual(self._scalar("SELECT v FROM public.t WHERE id=1;"), "ok")
+
+    def test_view_restored(self) -> None:
+        self.assertEqual(self._scalar("SELECT count(*) FROM public.t_view;"), "1")
+
+    def test_non_public_schema_restored(self) -> None:
+        self.assertEqual(
+            self._scalar(
+                "SELECT count(*) FROM pg_namespace WHERE nspname='discourse_functions';"
+            ),
+            "1",
+        )
+
+    def test_overloaded_functions_restored_once_each(self) -> None:
+        self.assertEqual(
+            self._scalar("SELECT count(*) FROM pg_proc WHERE proname='f';"), "2"
+        )
+        self.assertEqual(self._scalar("SELECT public.f(41) + public.f();"), "42")
+
+    def test_text_search_dictionary_restored_once(self) -> None:
+        self.assertEqual(
+            self._scalar(
+                "SELECT count(*) FROM pg_ts_dict WHERE dictname='english_stem_nostop';"
+            ),
+            "1",
+        )
+        self.assertEqual(
+            self._scalar(
+                "SELECT count(*) FROM pg_ts_config WHERE cfgname='english_nostop';"
+            ),
+            "1",
+        )
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/e2e/test_e2e_postgres_full.py

@@ -1,18 +1,19 @@
-# tests/e2e/test_e2e_postgres_full.py
 import unittest
 
 from .helpers import (
-    backup_run,
+    POSTGRES_DATA_DIR,
+    POSTGRES_IMAGE,
     backup_path,
+    backup_run,
     cleanup_docker,
     create_minimal_compose_dir,
     ensure_empty_dir,
     latest_version_dir,
     require_docker,
-    unique,
-    write_databases_csv,
     run,
+    unique,
     wait_for_postgres,
+    write_databases_csv,
 )
 
 
@@ -47,13 +48,12 @@ class TestE2EPostgresFull(unittest.TestCase):
                 "-e",
                 "POSTGRES_USER=postgres",
                 "-v",
-                f"{cls.pg_volume}:/var/lib/postgresql/data",
+                f"{cls.pg_volume}:{POSTGRES_DATA_DIR}",
-                "postgres:16",
+                POSTGRES_IMAGE,
             ]
         )
         wait_for_postgres(cls.pg_container, user="postgres", timeout_s=90)
 
-        # Create a table + data
         run(
             [
                 "docker",
@@ -76,7 +76,7 @@ class TestE2EPostgresFull(unittest.TestCase):
             compose_dir=cls.compose_dir,
             databases_csv=cls.databases_csv,
             database_containers=[cls.pg_container],
-            images_no_stop_required=["postgres", "mariadb", "mysql", "alpine"],
+            images_no_stop_required=[POSTGRES_IMAGE],
         )
 
         cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)

tests/e2e/test_e2e_postgres_no_copy.py

@@ -1,18 +1,19 @@
-# tests/e2e/test_e2e_postgres_no_copy.py
 import unittest
 
 from .helpers import (
-    backup_run,
+    POSTGRES_DATA_DIR,
+    POSTGRES_IMAGE,
     backup_path,
+    backup_run,
     cleanup_docker,
     create_minimal_compose_dir,
     ensure_empty_dir,
     latest_version_dir,
     require_docker,
-    unique,
-    write_databases_csv,
     run,
+    unique,
     wait_for_postgres,
+    write_databases_csv,
 )
 
 
@@ -46,8 +47,8 @@ class TestE2EPostgresNoCopy(unittest.TestCase):
                 "-e",
                 "POSTGRES_USER=postgres",
                 "-v",
-                f"{cls.pg_volume}:/var/lib/postgresql/data",
+                f"{cls.pg_volume}:{POSTGRES_DATA_DIR}",
-                "postgres:16",
+                POSTGRES_IMAGE,
             ]
         )
         wait_for_postgres(cls.pg_container, user="postgres", timeout_s=90)
@@ -74,8 +75,8 @@ class TestE2EPostgresNoCopy(unittest.TestCase):
             compose_dir=cls.compose_dir,
             databases_csv=cls.databases_csv,
             database_containers=[cls.pg_container],
-            images_no_stop_required=["postgres", "mariadb", "mysql", "alpine"],
+            images_no_stop_required=[POSTGRES_IMAGE],
-            dump_only=True,
+            only_sql=True,
         )
 
         cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)

tests/e2e/test_e2e_postgres_password_required.py

@@ -0,0 +1,159 @@
+"""An engine whose loopback auth really demands a password.
+
+Every other Postgres scenario runs stock postgres:alpine, whose generated
+pg_hba grants trust on 127.0.0.1 and ::1 - so `pg_dump -h localhost` never
+needs the password and a dump succeeds whether or not baudolo hands one to the
+container. This module makes the password mandatory, which is what a dedicated
+engine on a real host does.
+"""
+
+import unittest
+from pathlib import Path
+
+from baudolo.generation import CLUSTER_SUFFIX, DUMP_SUFFIX, FILES_DIR, SQL_DIR
+
+from .helpers import (
+    POSTGRES_DATA_DIR,
+    POSTGRES_IMAGE,
+    backup_path,
+    backup_run,
+    cleanup_docker,
+    create_minimal_compose_dir,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    wait_for_postgres,
+    write_databases_csv,
+)
+
+
+class TestE2EPostgresPasswordRequired(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        cls.prefix = unique("baudolo-e2e-pg-password-required")
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+
+        cls.pg_container = f"{cls.prefix}-pg"
+        cls.pg_volume = f"{cls.prefix}-pg-vol"
+        cls.containers = [cls.pg_container]
+        cls.volumes = [cls.pg_volume]
+
+        run(["docker", "volume", "create", cls.pg_volume])
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.pg_container,
+                "-e",
+                "POSTGRES_PASSWORD=pgpw",
+                "-e",
+                "POSTGRES_DB=appdb",
+                "-e",
+                "POSTGRES_USER=postgres",
+                # The entrypoint evals this into its initdb call, so the host
+                # lines of pg_hba demand scram while the local socket stays
+                # trust - the entrypoint's own init and the seeding below keep
+                # working, and only a TCP connection needs the password.
+                "-e",
+                "POSTGRES_INITDB_ARGS=--auth-host=scram-sha-256",
+                "-v",
+                f"{cls.pg_volume}:{POSTGRES_DATA_DIR}",
+                POSTGRES_IMAGE,
+            ]
+        )
+        wait_for_postgres(cls.pg_container, user="postgres", timeout_s=90)
+
+        run(
+            [
+                "docker",
+                "exec",
+                cls.pg_container,
+                "sh",
+                "-lc",
+                (
+                    'psql -U postgres -d appdb -c "CREATE TABLE t (id int primary '
+                    "key, v text); INSERT INTO t VALUES (1,'ok');\""
+                ),
+            ],
+            check=True,
+        )
+
+        cls.unauthenticated = run(
+            [
+                "docker",
+                "exec",
+                cls.pg_container,
+                "sh",
+                "-lc",
+                "pg_dump -U postgres -d appdb -h localhost",
+            ],
+            capture=True,
+            check=False,
+        )
+
+        cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
+        write_databases_csv(
+            cls.databases_csv,
+            [
+                (cls.pg_container, "appdb", "postgres", "pgpw"),
+                (cls.pg_container, "*", "postgres", "pgpw"),
+            ],
+        )
+
+        backup_run(
+            backups_dir=cls.backups_dir,
+            repo_name=cls.repo_name,
+            compose_dir=cls.compose_dir,
+            databases_csv=cls.databases_csv,
+            database_containers=[cls.pg_container],
+            images_no_stop_required=[POSTGRES_IMAGE],
+            only_sql=True,
+        )
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=cls.containers, volumes=cls.volumes)
+
+    def volume_dir(self) -> Path:
+        return backup_path(
+            self.backups_dir, self.repo_name, self.version, self.pg_volume
+        )
+
+    def test_a_dump_without_the_password_is_refused_by_the_server(self) -> None:
+        """Without this the module is vacuous: a pg_hba still saying trust would
+        let a baudolo that forwards nothing pass just as well."""
+        self.assertNotEqual(self.unauthenticated.returncode, 0)
+        self.assertIn("no password supplied", self.unauthenticated.stderr or "")
+
+    def test_the_configured_database_was_dumped(self) -> None:
+        dump = self.volume_dir() / SQL_DIR / f"appdb{DUMP_SUFFIX}"
+        self.assertTrue(dump.is_file(), f"expected a dump at {dump}")
+        self.assertIn("Dumped by pg_dump", dump.read_text(encoding="utf-8"))
+
+    def test_the_dump_carries_the_payload(self) -> None:
+        """pg_dump emits table data as COPY ... FROM stdin, so the row reads as
+        tab-separated values rather than as an INSERT literal."""
+        dump = self.volume_dir() / SQL_DIR / f"appdb{DUMP_SUFFIX}"
+        self.assertIn("COPY public.t (id, v) FROM stdin;", dump.read_text("utf-8"))
+        self.assertIn("1\tok", dump.read_text(encoding="utf-8"))
+
+    def test_the_cluster_row_was_dumped_too(self) -> None:
+        cluster = self.volume_dir() / SQL_DIR / f"{self.pg_container}{CLUSTER_SUFFIX}"
+        self.assertTrue(cluster.is_file(), f"expected a cluster dump at {cluster}")
+        self.assertIn("CREATE DATABASE", cluster.read_text(encoding="utf-8"))
+
+    def test_only_sql_left_no_file_copy_behind(self) -> None:
+        self.assertFalse((self.volume_dir() / FILES_DIR).exists())
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/e2e/test_e2e_postgres_single_transaction_live_writer.py

@@ -0,0 +1,184 @@
+import unittest
+
+from .helpers import (
+    POSTGRES_DATA_DIR,
+    POSTGRES_IMAGE,
+    backup_run,
+    cleanup_docker,
+    create_minimal_compose_dir,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    wait_for_postgres,
+    write_databases_csv,
+)
+
+# The discourse restore-drill race: `restore --empty` replays the dump into a
+# LIVE database while a background writer keeps touching a primary-key row
+# (discourse's mini_scheduler upserts scheduler_stats(id=1)). Without a
+# single-transaction replay, the pre-clean drops the table, the replay recreates
+# it and auto-commits, the writer wins the gap and inserts id=1, and the dump's
+# COPY of the same id then aborts with a duplicate-key violation under
+# ON_ERROR_STOP -> the whole restore fails. The --single-transaction replay keeps
+# the recreated table invisible until commit, so the writer can never insert the
+# racing row and the restore completes. A wide filler table makes the COPY slow
+# enough that the non-transactional variant loses the race deterministically.
+SEED_SQL = (
+    "CREATE TABLE public.scheduler_stats (id int primary key, v text);"
+    "INSERT INTO public.scheduler_stats VALUES (1, 'from-dump');"
+    "CREATE TABLE public.filler (id serial primary key, blob text);"
+    "INSERT INTO public.filler (blob)"
+    " SELECT repeat('x', 512) FROM generate_series(1, 100000);"
+)
+
+WRITER_LOOP = (
+    "while true; do "
+    "psql -h 127.0.0.1 -U postgres -d appdb "
+    "-c \"INSERT INTO public.scheduler_stats(id, v) VALUES (1, 'live') "
+    'ON CONFLICT (id) DO NOTHING;" >/dev/null 2>&1; '
+    "done"
+)
+
+
+class TestE2EPostgresSingleTransactionLiveWriter(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        cls.prefix = unique("baudolo-e2e-pg-single-txn")
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+
+        cls.pg_container = f"{cls.prefix}-pg"
+        cls.pg_volume = f"{cls.prefix}-pg-vol"
+        cls.writer = f"{cls.prefix}-writer"
+        cls.containers = [cls.pg_container, cls.writer]
+        cls.volumes = [cls.pg_volume]
+
+        run(["docker", "volume", "create", cls.pg_volume])
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.pg_container,
+                "-e",
+                "POSTGRES_PASSWORD=pgpw",
+                "-e",
+                "POSTGRES_DB=appdb",
+                "-e",
+                "POSTGRES_USER=postgres",
+                "-v",
+                f"{cls.pg_volume}:{POSTGRES_DATA_DIR}",
+                POSTGRES_IMAGE,
+            ]
+        )
+        wait_for_postgres(cls.pg_container, user="postgres", timeout_s=90)
+
+        run(
+            [
+                "docker",
+                "exec",
+                cls.pg_container,
+                "sh",
+                "-lc",
+                f'psql -U postgres -d appdb -v ON_ERROR_STOP=1 -c "{SEED_SQL}"',
+            ]
+        )
+
+        cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
+        write_databases_csv(
+            cls.databases_csv, [(cls.pg_container, "appdb", "postgres", "pgpw")]
+        )
+        backup_run(
+            backups_dir=cls.backups_dir,
+            repo_name=cls.repo_name,
+            compose_dir=cls.compose_dir,
+            databases_csv=cls.databases_csv,
+            database_containers=[cls.pg_container],
+            images_no_stop_required=[POSTGRES_IMAGE],
+        )
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.writer,
+                "--network",
+                f"container:{cls.pg_container}",
+                "-e",
+                "PGPASSWORD=pgpw",
+                POSTGRES_IMAGE,
+                "sh",
+                "-lc",
+                WRITER_LOOP,
+            ]
+        )
+
+        cls.restore = run(
+            [
+                "baudolo-restore",
+                "postgres",
+                cls.pg_volume,
+                cls.hash,
+                cls.version,
+                "--backups-dir",
+                cls.backups_dir,
+                "--repo-name",
+                cls.repo_name,
+                "--container",
+                cls.pg_container,
+                "--db-name",
+                "appdb",
+                "--db-user",
+                "postgres",
+                "--db-password",
+                "pgpw",
+                "--empty",
+            ],
+            capture=True,
+            check=False,
+        )
+
+        run(["docker", "rm", "-f", cls.writer], capture=True, check=False)
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=cls.containers, volumes=cls.volumes)
+
+    def _scalar(self, sql: str) -> str:
+        p = run(
+            [
+                "docker",
+                "exec",
+                self.pg_container,
+                "sh",
+                "-lc",
+                f'psql -U postgres -d appdb -t -A -c "{sql}"',
+            ]
+        )
+        return (p.stdout or "").strip()
+
+    def test_restore_survived_the_live_writer(self) -> None:
+        self.assertEqual(
+            self.restore.returncode,
+            0,
+            f"restore aborted (duplicate-key race not contained):\n{self.restore.stderr}",
+        )
+
+    def test_primary_key_row_restored(self) -> None:
+        self.assertEqual(
+            self._scalar("SELECT count(*) FROM public.scheduler_stats WHERE id=1;"),
+            "1",
+        )
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/e2e/test_e2e_restore_files_backing_store.py

@@ -0,0 +1,120 @@
+"""Restoring files into a volume that has a backing store of its own.
+
+Docker keeps the same ``/var/lib/docker/volumes/<name>/_data`` path for such a
+volume and mounts the real storage over it only while a container holds it.
+Writing there unmounted lands in the empty directory underneath, is hidden by
+the next mount, and rsync reports success - so the restore has to refuse.
+"""
+
+import unittest
+from pathlib import Path
+
+from .helpers import (
+    backup_path,
+    cleanup_docker,
+    ensure_empty_dir,
+    machine_hash,
+    require_docker,
+    run,
+    unique,
+)
+
+MARKER = "restored-payload"
+VERSION = "20260817000000"
+
+
+def mountpoint_of(volume: str) -> Path:
+    return Path("/var/lib/docker/volumes") / volume / "_data"
+
+
+def contents(directory: Path) -> list[str]:
+    return sorted(p.name for p in directory.iterdir()) if directory.is_dir() else []
+
+
+class TestE2ERestoreFilesBackingStore(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        cls.prefix = unique("baudolo-e2e-backing")
+        cls.repo_name = cls.prefix
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        cls.backing = Path(f"/tmp/{cls.prefix}/backing")
+        ensure_empty_dir(cls.backups_dir)
+        ensure_empty_dir(str(cls.backing))
+
+        cls.bound_volume = f"{cls.prefix}-bound"
+        cls.plain_volume = f"{cls.prefix}-plain"
+        cls.volumes = [cls.bound_volume, cls.plain_volume]
+
+        for volume in cls.volumes:
+            files = (
+                backup_path(cls.backups_dir, cls.repo_name, VERSION, volume) / "files"
+            )
+            files.mkdir(parents=True, exist_ok=True)
+            (files / "marker.txt").write_text(MARKER, encoding="utf-8")
+
+        run(
+            [
+                "docker",
+                "volume",
+                "create",
+                "--driver",
+                "local",
+                "--opt",
+                "type=none",
+                "--opt",
+                "o=bind",
+                "--opt",
+                f"device={cls.backing}",
+                cls.bound_volume,
+            ]
+        )
+        run(["docker", "volume", "create", cls.plain_volume])
+
+        cls.refused = cls.restore(cls.bound_volume)
+        cls.accepted = cls.restore(cls.plain_volume)
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=[], volumes=cls.volumes)
+
+    @classmethod
+    def restore(cls, volume: str):
+        return run(
+            [
+                "baudolo-restore",
+                "files",
+                volume,
+                machine_hash(),
+                VERSION,
+                "--backups-dir",
+                cls.backups_dir,
+                "--repo-name",
+                cls.repo_name,
+            ],
+            check=False,
+        )
+
+    def test_a_volume_with_its_own_backing_store_is_refused(self) -> None:
+        self.assertEqual(self.refused.returncode, 2, self.refused.stdout)
+        self.assertIn("backing store of its own", self.refused.stderr)
+
+    def test_nothing_was_written_into_the_backing_store(self) -> None:
+        self.assertEqual(contents(self.backing), [])
+
+    def test_nothing_was_written_under_the_mount_either(self) -> None:
+        self.assertEqual(
+            contents(mountpoint_of(self.bound_volume)),
+            [],
+            "the copy landed in the directory the next mount hides",
+        )
+
+    def test_a_plain_volume_is_still_restored(self) -> None:
+        self.assertEqual(self.accepted.returncode, 0, self.accepted.stderr)
+        restored = mountpoint_of(self.plain_volume) / "marker.txt"
+        self.assertTrue(restored.is_file(), f"{restored} missing")
+        self.assertEqual(restored.read_text(encoding="utf-8"), MARKER)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/e2e/test_e2e_seed_star_and_db_entries_backup_postgres.py

@@ -0,0 +1,230 @@
+import unittest
+
+from .helpers import (
+    POSTGRES_DATA_DIR,
+    POSTGRES_IMAGE,
+    backup_path,
+    cleanup_docker,
+    create_minimal_compose_dir,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    wait_for_postgres,
+)
+
+
+class TestE2ESeedStarAndDbEntriesBackupPostgres(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+
+        cls.prefix = unique("baudolo-e2e-seed-star-and-db")
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+
+        # --- Volumes ---
+        cls.db_volume = f"{cls.prefix}-vol-db"
+        cls.files_volume = f"{cls.prefix}-vol-files"
+        cls.volumes = [cls.db_volume, cls.files_volume]
+
+        run(["docker", "volume", "create", cls.db_volume])
+        run(["docker", "volume", "create", cls.files_volume])
+
+        # Put a marker into the non-db volume
+        cls.marker = "hello-non-db-seed-star"
+        run(
+            [
+                "docker",
+                "run",
+                "--rm",
+                "-v",
+                f"{cls.files_volume}:/data",
+                "alpine:3.20",
+                "sh",
+                "-lc",
+                f"echo '{cls.marker}' > /data/hello.txt",
+            ]
+        )
+
+        # --- Start Postgres container using the DB volume ---
+        cls.pg_container = f"{cls.prefix}-pg"
+        cls.containers = [cls.pg_container]
+
+        cls.pg_password = "postgres"
+        cls.pg_user = "postgres"
+
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.pg_container,
+                "-e",
+                f"POSTGRES_PASSWORD={cls.pg_password}",
+                "-v",
+                f"{cls.db_volume}:{POSTGRES_DATA_DIR}",
+                POSTGRES_IMAGE,
+            ]
+        )
+        wait_for_postgres(cls.pg_container, user="postgres", timeout_s=90)
+
+        # Create two DBs and deterministic content, so pg_dumpall is meaningful
+        cls.pg_db1 = "testdb1"
+        cls.pg_db2 = "testdb2"
+
+        run(
+            [
+                "docker",
+                "exec",
+                cls.pg_container,
+                "sh",
+                "-lc",
+                (
+                    f'psql -U {cls.pg_user} -c "CREATE DATABASE {cls.pg_db1};" || true; '
+                    f'psql -U {cls.pg_user} -c "CREATE DATABASE {cls.pg_db2};" || true; '
+                ),
+            ],
+            check=True,
+        )
+
+        run(
+            [
+                "docker",
+                "exec",
+                cls.pg_container,
+                "sh",
+                "-lc",
+                (
+                    f"psql -U {cls.pg_user} -d {cls.pg_db1} -c "
+                    '"CREATE TABLE IF NOT EXISTS t (id INT PRIMARY KEY, v TEXT);'
+                    "INSERT INTO t(id,v) VALUES (1,'hello-db1') "
+                    'ON CONFLICT (id) DO UPDATE SET v=EXCLUDED.v;"'
+                ),
+            ],
+            check=True,
+        )
+        run(
+            [
+                "docker",
+                "exec",
+                cls.pg_container,
+                "sh",
+                "-lc",
+                (
+                    f"psql -U {cls.pg_user} -d {cls.pg_db2} -c "
+                    '"CREATE TABLE IF NOT EXISTS t (id INT PRIMARY KEY, v TEXT);'
+                    "INSERT INTO t(id,v) VALUES (1,'hello-db2') "
+                    'ON CONFLICT (id) DO UPDATE SET v=EXCLUDED.v;"'
+                ),
+            ],
+            check=True,
+        )
+
+        # --- Seed databases.csv using CLI (star + concrete db) ---
+        cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
+
+        # IMPORTANT: because we pass --database-containers <container>,
+        # get_instance() will use the container name as instance key.
+        instance = cls.pg_container
+
+        # Seed star entry (pg_dumpall)
+        run(
+            [
+                "baudolo-seed",
+                cls.databases_csv,
+                instance,
+                "*",
+                cls.pg_user,
+                cls.pg_password,
+            ]
+        )
+
+        # Seed concrete DB entry (pg_dump)
+        run(
+            [
+                "baudolo-seed",
+                cls.databases_csv,
+                instance,
+                cls.pg_db1,
+                cls.pg_user,
+                cls.pg_password,
+            ]
+        )
+
+        cmd = [
+            "baudolo",
+            "--compose-dir",
+            cls.compose_dir,
+            "--databases-csv",
+            cls.databases_csv,
+            "--database-containers",
+            cls.pg_container,
+            "--images-no-stop-required",
+            POSTGRES_IMAGE,
+            "--only-sql",
+            "--backups-dir",
+            cls.backups_dir,
+            "--repo-name",
+            cls.repo_name,
+        ]
+        cp = run(cmd, capture=True, check=True)
+        cls.stdout = cp.stdout or ""
+        cls.stderr = cp.stderr or ""
+
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=cls.containers, volumes=cls.volumes)
+
+    def test_db_volume_has_cluster_dump_and_concrete_db_dump_and_no_files(self) -> None:
+        base = backup_path(
+            self.backups_dir, self.repo_name, self.version, self.db_volume
+        )
+        sql_dir = base / "sql"
+        files_dir = base / "files"
+
+        self.assertTrue(sql_dir.exists(), f"Expected sql dir at: {sql_dir}")
+        self.assertFalse(
+            files_dir.exists(),
+            f"Did not expect files dir for DB volume when only-sql succeeded: {files_dir}",
+        )
+
+        # Cluster dump file produced by '*' entry
+        cluster = sql_dir / f"{self.pg_container}.cluster.backup.sql"
+        self.assertTrue(cluster.is_file(), f"Expected cluster dump file at: {cluster}")
+
+        # Concrete DB dump produced by normal entry
+        db1 = sql_dir / f"{self.pg_db1}.backup.sql"
+        self.assertTrue(db1.is_file(), f"Expected db dump file at: {db1}")
+
+        # Basic sanity: cluster dump usually contains CREATE DATABASE statements
+        txt = cluster.read_text(encoding="utf-8", errors="ignore")
+        self.assertIn(
+            "CREATE DATABASE",
+            txt,
+            "Expected cluster dump to contain CREATE DATABASE statements",
+        )
+
+    def test_non_db_volume_still_has_files_backup(self) -> None:
+        base = backup_path(
+            self.backups_dir, self.repo_name, self.version, self.files_volume
+        )
+        files_dir = base / "files"
+
+        self.assertTrue(
+            files_dir.exists(), f"Expected files dir for non-DB volume at: {files_dir}"
+        )
+
+        marker = files_dir / "hello.txt"
+        self.assertTrue(marker.is_file(), f"Expected marker file at: {marker}")
+        self.assertEqual(
+            marker.read_text(encoding="utf-8").strip(),
+            self.marker,
+        )

tests/e2e/test_e2e_snapshot.py

@@ -0,0 +1,169 @@
+"""Snapshot capture against real filesystems.
+
+Loop devices are only available to a privileged container, so each case builds
+its filesystem inside one and drives snapshot_driver.py there. A filesystem
+without snapshot support must fail loudly rather than degrade to a live copy,
+which is the property that makes the mode safe to offer at all.
+"""
+
+from __future__ import annotations
+
+import os
+import shutil
+import unittest
+from pathlib import Path
+
+from .helpers import require_docker, run, unique
+
+REPO_SRC = Path(__file__).resolve().parents[2] / "src"
+DRIVER = Path(__file__).resolve().parent / "snapshot_driver.py"
+FAITHFUL_DRIVER = Path(__file__).resolve().parent / "faithful_driver.py"
+IMAGE = "alpine:3.20"
+PACKAGES = "apk add -q btrfs-progs e2fsprogs zfs python3 util-linux"
+ATTACH = (
+    "LOOP=$(losetup -f | awk '{print $1}') "
+    '&& { [ -b "$LOOP" ] || mknod "$LOOP" b 7 "${LOOP#/dev/loop}"; }; '
+    'losetup "$LOOP" /img'
+)
+LOOP_FS = {
+    "btrfs": "btrfs subvolume create /subject/docker >/dev/null",
+    "ext4": "mkdir -p /subject/docker",
+}
+# A container carries no /lib/modules, so modprobe fails even on a loaded module.
+ZFS_READY = "{ [ -c /dev/zfs ] || modprobe zfs 2>/dev/null; }; [ -c /dev/zfs ]"
+
+
+def mount_script(fstype: str) -> str:
+    """Build a filesystem on a loop device and carve out the snapshot subject."""
+    if fstype == "zfs":
+        return (
+            f"{PACKAGES} && {ZFS_READY} && truncate -s 400M /img "
+            "&& zpool create -m none baudolo /img "
+            "&& zfs create -o mountpoint=/subject/docker baudolo/docker"
+        )
+    return (
+        f"{PACKAGES} && truncate -s 400M /img && mkfs.{fstype} -q /img "
+        f'&& mkdir -p /subject && {ATTACH} && mount -t {fstype} "$LOOP" /subject '
+        f"&& {LOOP_FS[fstype]}"
+    )
+
+
+def zfs_usable() -> bool:
+    """Whether this host's kernel can serve zfs to a privileged container."""
+    proc = run(
+        [
+            "docker",
+            "run",
+            "--rm",
+            "--privileged",
+            IMAGE,
+            "sh",
+            "-lc",
+            f"apk add -q zfs >/dev/null 2>&1 && {ZFS_READY}",
+        ],
+        capture=True,
+        check=False,
+    )
+    return proc.returncode == 0
+
+
+def required(fstype: str) -> bool:
+    """Whether this run must cover ``fstype`` instead of skipping it.
+
+    CI sets E2E_REQUIRE_FILESYSTEMS so a missing kernel module fails the build
+    rather than passing it with a filesystem silently untested.
+    """
+    demanded = os.environ.get("E2E_REQUIRE_FILESYSTEMS", "")
+    return fstype in demanded.replace(",", " ").split()
+
+
+def stage(driver: Path) -> Path:
+    """Copy source and driver under /tmp, the only path the DinD daemon shares."""
+    staged = Path("/tmp") / unique("baudolo-e2e-snapshot")
+    shutil.copytree(REPO_SRC, staged / "src")
+    shutil.copy(driver, staged / "driver.py")
+    return staged
+
+
+def drive(fstype: str, arguments: str, *, driver: Path = DRIVER) -> str:
+    staged = stage(driver)
+    script = f"set -e; {mount_script(fstype)}; python3 /driver.py {arguments}"
+    try:
+        proc = run(
+            [
+                "docker",
+                "run",
+                "--rm",
+                "--privileged",
+                "--name",
+                staged.name,
+                "-v",
+                f"{staged / 'src'}:/src:ro",
+                "-v",
+                f"{staged / 'driver.py'}:/driver.py:ro",
+                IMAGE,
+                "sh",
+                "-lc",
+                script,
+            ],
+            capture=True,
+            check=False,
+        )
+    finally:
+        shutil.rmtree(staged, ignore_errors=True)
+    if proc.returncode != 0:
+        raise AssertionError(
+            f"{fstype} driver failed on {arguments}:\n{proc.stdout}\n{proc.stderr}"
+        )
+    return proc.stdout
+
+
+class TestE2ESnapshot(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+
+    def assert_freezes(self, fstype: str) -> None:
+        output = drive(fstype, f"{fstype} /subject/docker supported")
+        self.assertIn("PASS the snapshot exposes the volume", output)
+        self.assertIn("PASS a later write does not reach the snapshot", output)
+        self.assertIn("PASS the snapshot is removed afterwards", output)
+        self.assertIn("ALL OK", output)
+
+    def test_btrfs_snapshot_freezes_the_volume(self) -> None:
+        self.assert_freezes("btrfs")
+
+    def test_zfs_snapshot_freezes_the_volume(self) -> None:
+        if not zfs_usable():
+            if required("zfs"):
+                self.fail(
+                    "E2E_REQUIRE_FILESYSTEMS demands zfs, but this kernel provides no "
+                    "zfs module; load it before running the suite"
+                )
+            self.skipTest(
+                "this kernel provides no zfs module, so no pool can be created"
+            )
+        self.assert_freezes("zfs")
+
+    def test_ext4_has_no_snapshot_and_says_so(self) -> None:
+        output = drive("ext4", "btrfs /subject/docker unsupported")
+        self.assertIn("PASS refused loudly", output)
+
+    def test_an_unknown_kind_is_refused_before_touching_the_filesystem(self) -> None:
+        output = drive("ext4", "lvm /subject/docker unsupported")
+        self.assertIn("PASS refused loudly", output)
+
+    def test_a_volume_with_its_own_storage_is_copied_live_not_from_the_snapshot(
+        self,
+    ) -> None:
+        output = drive("btrfs", "/subject/docker", driver=FAITHFUL_DRIVER)
+        self.assertIn(
+            "PASS the snapshot shows the other volume as an empty directory", output
+        )
+        self.assertIn("PASS the other volume degrades to live", output)
+        self.assertIn("PASS the plain volume is read from the snapshot", output)
+        self.assertIn("ALL OK", output)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/e2e/test_e2e_snapshot_db.py

@@ -0,0 +1,112 @@
+"""A live database survives being captured from a snapshot.
+
+The database is written to while the snapshot is taken and keeps writing
+afterwards, so the copy can only be a point in time - never a clean shutdown.
+A second server is then started on that copy: it must recover on its own and
+still hold every row committed before the snapshot.
+"""
+
+from __future__ import annotations
+
+import shutil
+import unittest
+from pathlib import Path
+
+from .helpers import require_docker, run, unique
+
+REPO_SRC = Path(__file__).resolve().parents[2] / "src"
+DRIVER = Path(__file__).resolve().parent / "snapshot_db_driver.py"
+IMAGE = "alpine:3.20"
+SOCKET = "/tmp/live.sock"
+RESTORED_SOCKET = "/tmp/restored.sock"
+DATADIR = "/subject/docker/volumes/mariadb_data/_data"
+RESTORED = "/restored"
+
+SCRIPT = f"""set -e
+apk add -q btrfs-progs util-linux python3 mariadb mariadb-client rsync
+truncate -s 900M /img
+mkfs.btrfs -q /img
+mkdir -p /subject
+LOOP=$(losetup -f | awk '{{print $1}}')
+{{ [ -b "$LOOP" ] || mknod "$LOOP" b 7 "${{LOOP#/dev/loop}}"; }}
+losetup "$LOOP" /img
+mount -t btrfs "$LOOP" /subject
+btrfs subvolume create /subject/docker >/dev/null
+mkdir -p {DATADIR}
+
+mariadb-install-db --user=root --datadir={DATADIR} >/dev/null 2>&1
+mariadbd --user=root --datadir={DATADIR} --socket={SOCKET} --skip-networking &
+for i in $(seq 1 60); do mariadb-admin --socket={SOCKET} ping >/dev/null 2>&1 && break; sleep 1; done
+
+mariadb --socket={SOCKET} -e "CREATE DATABASE demo;
+  CREATE TABLE demo.t (id INT PRIMARY KEY, v VARCHAR(32)) ENGINE=InnoDB;
+  INSERT INTO demo.t VALUES (1,'committed'),(2,'committed');"
+
+mariadb --socket={SOCKET} -e "INSERT INTO demo.t VALUES (3,'committed');"
+python3 /driver.py
+mariadb --socket={SOCKET} -e "INSERT INTO demo.t VALUES (4,'after-snapshot');"
+
+mkdir -p {RESTORED}
+rsync -a /backups/20260731/mariadb_data/files/ {RESTORED}/
+mariadbd --user=root --datadir={RESTORED} --socket={RESTORED_SOCKET} --skip-networking &
+for i in $(seq 1 60); do mariadb-admin --socket={RESTORED_SOCKET} ping >/dev/null 2>&1 && break; sleep 1; done
+
+echo "RESTORED_ROWS=$(mariadb --socket={RESTORED_SOCKET} -N -B -e 'SELECT COUNT(*) FROM demo.t;')"
+echo "RESTORED_AFTER=$(mariadb --socket={RESTORED_SOCKET} -N -B -e \\
+  "SELECT COUNT(*) FROM demo.t WHERE v='after-snapshot';")"
+echo DB_OK
+"""
+
+
+class TestE2ESnapshotDatabase(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        staged = Path("/tmp") / unique("baudolo-e2e-snapshot-db")
+        shutil.copytree(REPO_SRC, staged / "src")
+        shutil.copy(DRIVER, staged / "driver.py")
+        try:
+            proc = run(
+                [
+                    "docker",
+                    "run",
+                    "--rm",
+                    "--privileged",
+                    "--name",
+                    staged.name,
+                    "-v",
+                    f"{staged / 'src'}:/src:ro",
+                    "-v",
+                    f"{staged / 'driver.py'}:/driver.py:ro",
+                    IMAGE,
+                    "sh",
+                    "-lc",
+                    SCRIPT,
+                ],
+                capture=True,
+                check=False,
+            )
+        finally:
+            shutil.rmtree(staged, ignore_errors=True)
+        cls.output = proc.stdout + proc.stderr
+        cls.returncode = proc.returncode
+
+    def test_the_run_completed(self) -> None:
+        self.assertEqual(self.returncode, 0, self.output)
+        self.assertIn("DB_OK", self.output)
+
+    def test_the_backup_came_from_the_snapshot(self) -> None:
+        self.assertIn("SNAPSHOT COPY DONE", self.output)
+
+    def test_the_restored_server_recovered_on_its_own(self) -> None:
+        self.assertIn("RESTORED_ROWS=3", self.output)
+
+    def test_writes_after_the_snapshot_are_absent(self) -> None:
+        self.assertIn("RESTORED_AFTER=0", self.output)
+
+    def test_the_copy_was_an_unclean_state_the_engine_had_to_repair(self) -> None:
+        self.assertRegex(self.output, r"(?i)crash recovery|rolling back|log sequence")
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/e2e/test_e2e_swarm_task_skip.py

@@ -0,0 +1,194 @@
+# Reproduces the swarm flake fixed on this branch: baudolo used to stop a
+# swarm task container around the volume file backup because its image was
+# not whitelisted; the orchestrator immediately replaced the stopped task and
+# the later `docker start` failed on the detached overlay network, killing
+# the backup run. With the fix the task container is skipped (backed up hot):
+# the backup succeeds, the very same container instance keeps running, and
+# the service never has to replace a task.
+import time
+import unittest
+
+from .helpers import (
+    backup_path,
+    backup_run,
+    create_minimal_compose_dir,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    write_databases_csv,
+)
+
+
+def _swarm_state() -> str:
+    return run(
+        ["docker", "info", "--format", "{{.Swarm.LocalNodeState}}"]
+    ).stdout.strip()
+
+
+def _task_container_id(service: str, timeout_s: int = 60) -> str:
+    deadline = time.time() + timeout_s
+    while time.time() < deadline:
+        out = run(
+            [
+                "docker",
+                "ps",
+                "--filter",
+                f"label=com.docker.swarm.service.name={service}",
+                "--format",
+                "{{.ID}}",
+            ]
+        ).stdout.strip()
+        if out:
+            return out.splitlines()[0]
+        time.sleep(2)
+    raise RuntimeError(f"No running task container for service {service}")
+
+
+def _started_at(container_id: str) -> str:
+    return run(
+        ["docker", "inspect", "--format", "{{.State.StartedAt}}", container_id]
+    ).stdout.strip()
+
+
+class TestE2ESwarmTaskSkip(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        cls.prefix = unique("baudolo-e2e-swarm-skip")
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+
+        cls.swarm_initted = False
+        if _swarm_state() != "active":
+            run(["docker", "swarm", "init", "--advertise-addr", "127.0.0.1"])
+            cls.swarm_initted = True
+
+        cls.volume = f"{cls.prefix}-vol"
+        cls.service = f"{cls.prefix}-svc"
+        cls.volumes = [cls.volume]
+
+        run(["docker", "volume", "create", cls.volume])
+        run(
+            [
+                "docker",
+                "run",
+                "--rm",
+                "-v",
+                f"{cls.volume}:/data",
+                "alpine:3.20",
+                "sh",
+                "-lc",
+                "echo 'swarm-payload' > /data/payload.txt",
+            ]
+        )
+
+        run(
+            [
+                "docker",
+                "service",
+                "create",
+                "--name",
+                cls.service,
+                "--replicas",
+                "1",
+                "--mount",
+                f"type=volume,source={cls.volume},target=/data",
+                "alpine:3.20",
+                "sleep",
+                "3600",
+            ]
+        )
+        cls.task_cid = _task_container_id(cls.service)
+        cls.task_started_at = _started_at(cls.task_cid)
+
+        cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
+        write_databases_csv(cls.databases_csv, [])
+
+        # Whitelist that matches nothing: on main this forces a stop of every
+        # container at the volume, i.e. exactly the flake; on this branch the
+        # swarm task must be skipped instead. (An empty list would leave the
+        # --images-no-stop-required flag without arguments and argparse-fail.)
+        backup_run(
+            backups_dir=cls.backups_dir,
+            repo_name=cls.repo_name,
+            compose_dir=cls.compose_dir,
+            databases_csv=cls.databases_csv,
+            database_containers=["dummy-db"],
+            images_no_stop_required=["image-that-matches-nothing"],
+        )
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        run(["docker", "service", "rm", cls.service], check=False)
+        deadline = time.time() + 30
+        while time.time() < deadline:
+            out = run(
+                [
+                    "docker",
+                    "ps",
+                    "-aq",
+                    "--filter",
+                    f"label=com.docker.swarm.service.name={cls.service}",
+                ],
+                check=False,
+            ).stdout.strip()
+            if not out:
+                break
+            time.sleep(2)
+        for v in cls.volumes:
+            run(["docker", "volume", "rm", "-f", v], check=False)
+        if cls.swarm_initted:
+            run(["docker", "swarm", "leave", "--force"], check=False)
+
+    def test_volume_backed_up_hot(self) -> None:
+        p = (
+            backup_path(
+                self.backups_dir,
+                self.repo_name,
+                self.version,
+                self.volume,
+            )
+            / "files"
+            / "payload.txt"
+        )
+        self.assertTrue(p.is_file(), f"Expected backed up file at: {p}")
+
+    def test_task_container_never_stopped(self) -> None:
+        out = run(
+            ["docker", "ps", "-q", "--no-trunc", "--filter", f"id={self.task_cid}"]
+        ).stdout.strip()
+        self.assertTrue(
+            out.startswith(self.task_cid) or self.task_cid.startswith(out.strip()[:12]),
+            f"Task container {self.task_cid} is no longer running",
+        )
+        self.assertEqual(
+            self.task_started_at,
+            _started_at(self.task_cid),
+            "Task container was restarted during the backup",
+        )
+
+    def test_service_never_replaced_the_task(self) -> None:
+        states = run(
+            [
+                "docker",
+                "service",
+                "ps",
+                self.service,
+                "--format",
+                "{{.DesiredState}} {{.CurrentState}}",
+            ]
+        ).stdout.strip()
+        lines = [line for line in states.splitlines() if line.strip()]
+        self.assertEqual(
+            len(lines), 1, f"Service task history shows replacements:\n{states}"
+        )
+        self.assertIn("Running", lines[0])
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/e2e/test_e2e_version_gate.py

@@ -0,0 +1,283 @@
+"""A dump from a newer engine must be refused before --empty destroys anything.
+
+The pre-clean and the replay are two separate sessions with no rollback across
+them, so a dump the engine cannot parse leaves an emptied database behind. The
+decisive assertion here is not the non-zero exit - it is that the payload is
+still readable afterwards.
+"""
+
+import re
+import unittest
+from pathlib import Path
+
+from .helpers import (
+    MARIADB_DATA_DIR,
+    MARIADB_IMAGE,
+    POSTGRES_DATA_DIR,
+    POSTGRES_IMAGE,
+    backup_path,
+    backup_run,
+    cleanup_docker,
+    create_minimal_compose_dir,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    wait_for_mariadb,
+    wait_for_mariadb_sql,
+    wait_for_postgres,
+    write_databases_csv,
+)
+
+PAYLOAD = "gate-payload"
+FUTURE = "99.0"
+
+
+def rewrite_version(dump: Path, pattern: str, version: str) -> str:
+    """Make the dump claim ``version``; return what it claimed before."""
+    text = dump.read_text(encoding="utf-8", errors="replace")
+    found = re.search(pattern, text)
+    if not found:
+        raise AssertionError(f"{dump} carries no version header matching {pattern}")
+    claimed = found.group(1)
+    dump.write_text(
+        text.replace(found.group(0), found.group(0).replace(claimed, version), 1),
+        encoding="utf-8",
+    )
+    return claimed
+
+
+class GateCase:
+    """Drive one engine through refusal, escape hatch and truthful replay."""
+
+    engine = ""
+    pattern = ""
+
+    @classmethod
+    def restore(cls, *extra: str):
+        return run(
+            [
+                "baudolo-restore",
+                cls.engine,
+                cls.volume,
+                cls.hash,
+                cls.version,
+                "--backups-dir",
+                cls.backups_dir,
+                "--repo-name",
+                cls.repo_name,
+                "--container",
+                cls.container,
+                "--db-name",
+                cls.db_name,
+                "--db-user",
+                cls.db_user,
+                "--db-password",
+                cls.db_password,
+                "--empty",
+                *extra,
+            ],
+            check=False,
+        )
+
+    @classmethod
+    def prepare(cls) -> None:
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+        cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
+        write_databases_csv(
+            cls.databases_csv,
+            [(cls.container, cls.db_name, cls.db_user, cls.db_password)],
+        )
+        backup_run(
+            backups_dir=cls.backups_dir,
+            repo_name=cls.repo_name,
+            compose_dir=cls.compose_dir,
+            databases_csv=cls.databases_csv,
+            database_containers=[cls.container],
+            images_no_stop_required=[cls.image],
+        )
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+        cls.dump = (
+            backup_path(cls.backups_dir, cls.repo_name, cls.version, cls.volume)
+            / "sql"
+            / f"{cls.db_name}.backup.sql"
+        )
+
+        cls.truthful_version = rewrite_version(cls.dump, cls.pattern, FUTURE)
+        cls.refused = cls.restore()
+        cls.payload_after_refusal = cls.read_payload()
+
+        cls.forced = cls.restore("--no-version-check")
+        cls.payload_after_force = cls.read_payload()
+
+        rewrite_version(cls.dump, cls.pattern, cls.truthful_version)
+        cls.replayed = cls.restore()
+        cls.payload_after_replay = cls.read_payload()
+
+    def test_the_dump_states_the_engine_it_came_from(self) -> None:
+        self.assertRegex(self.truthful_version, r"^\d+")
+
+    def test_a_newer_dump_is_refused(self) -> None:
+        self.assertNotEqual(self.refused.returncode, 0, self.refused.stdout)
+
+    def test_the_refusal_names_the_version_it_refused(self) -> None:
+        self.assertIn(FUTURE, self.refused.stderr)
+        self.assertIn("older engine", self.refused.stderr)
+
+    def test_the_refusal_left_the_data_untouched(self) -> None:
+        self.assertEqual(
+            self.payload_after_refusal,
+            PAYLOAD,
+            "--empty pre-cleaned before the version was checked",
+        )
+
+    def test_the_escape_hatch_replays_anyway(self) -> None:
+        self.assertEqual(self.forced.returncode, 0, self.forced.stderr)
+        self.assertEqual(self.payload_after_force, PAYLOAD)
+
+    def test_a_truthful_dump_replays(self) -> None:
+        self.assertEqual(self.replayed.returncode, 0, self.replayed.stderr)
+        self.assertEqual(self.payload_after_replay, PAYLOAD)
+
+
+class TestE2EPostgresVersionGate(GateCase, unittest.TestCase):
+    engine = "postgres"
+    pattern = r"-- Dumped from database version (\S+)"
+
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        cls.prefix = unique("baudolo-e2e-pg-gate")
+        cls.container = f"{cls.prefix}-pg"
+        cls.volume = f"{cls.prefix}-pg-vol"
+        cls.image = POSTGRES_IMAGE
+        cls.db_name = "appdb"
+        cls.db_user = "postgres"
+        cls.db_password = "pgpw"
+
+        run(["docker", "volume", "create", cls.volume])
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.container,
+                "-e",
+                f"POSTGRES_PASSWORD={cls.db_password}",
+                "-v",
+                f"{cls.volume}:{POSTGRES_DATA_DIR}",
+                POSTGRES_IMAGE,
+            ]
+        )
+        wait_for_postgres(cls.container, user=cls.db_user)
+        cls.sql("postgres", f"CREATE DATABASE {cls.db_name}")
+        cls.sql(
+            cls.db_name,
+            f"CREATE TABLE t (v text); INSERT INTO t VALUES ('{PAYLOAD}');",
+        )
+        cls.prepare()
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=[cls.container], volumes=[cls.volume])
+
+    @classmethod
+    def sql(cls, database: str, statement: str) -> str:
+        p = run(
+            [
+                "docker",
+                "exec",
+                cls.container,
+                "sh",
+                "-lc",
+                f'psql -U {cls.db_user} -d {database} -t -A -c "{statement}"',
+            ],
+            check=False,
+        )
+        return (p.stdout or "").strip()
+
+    @classmethod
+    def read_payload(cls) -> str:
+        return cls.sql(cls.db_name, "SELECT v FROM t")
+
+
+class TestE2EMariadbVersionGate(GateCase, unittest.TestCase):
+    engine = "mariadb"
+    pattern = r"-- Server version\s+(\S+)"
+
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        cls.prefix = unique("baudolo-e2e-mdb-gate")
+        cls.container = f"{cls.prefix}-mdb"
+        cls.volume = f"{cls.prefix}-mdb-vol"
+        cls.image = MARIADB_IMAGE
+        cls.db_name = "appdb"
+        cls.db_user = "test"
+        cls.db_password = "testpw"
+
+        run(["docker", "volume", "create", cls.volume])
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.container,
+                "-e",
+                "MARIADB_ROOT_PASSWORD=rootpw",
+                "-e",
+                f"MARIADB_DATABASE={cls.db_name}",
+                "-e",
+                f"MARIADB_USER={cls.db_user}",
+                "-e",
+                f"MARIADB_PASSWORD={cls.db_password}",
+                "-v",
+                f"{cls.volume}:{MARIADB_DATA_DIR}",
+                MARIADB_IMAGE,
+            ]
+        )
+        wait_for_mariadb(cls.container, root_password="rootpw", timeout_s=90)
+        wait_for_mariadb_sql(
+            cls.container, user=cls.db_user, password=cls.db_password, timeout_s=90
+        )
+        cls.sql(
+            f"CREATE TABLE {cls.db_name}.t (v VARCHAR(50)); "
+            f"INSERT INTO {cls.db_name}.t VALUES ('{PAYLOAD}');"
+        )
+        cls.prepare()
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=[cls.container], volumes=[cls.volume])
+
+    @classmethod
+    def sql(cls, statement: str) -> str:
+        p = run(
+            [
+                "docker",
+                "exec",
+                cls.container,
+                "sh",
+                "-lc",
+                (
+                    f"mariadb -h 127.0.0.1 -u{cls.db_user} -p{cls.db_password} "
+                    f'-N -B -e "{statement}"'
+                ),
+            ],
+            check=False,
+        )
+        return (p.stdout or "").strip()
+
+    @classmethod
+    def read_payload(cls) -> str:
+        return cls.sql(f"SELECT v FROM {cls.db_name}.t")
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/e2e/test_e2e_volumes_no_backup_required_early_skip.py

@@ -0,0 +1,125 @@
+import unittest
+
+from .helpers import (
+    backup_path,
+    cleanup_docker,
+    create_minimal_compose_dir,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    write_databases_csv,
+)
+
+
+class TestE2EVolumesNoBackupRequiredEarlySkip(unittest.TestCase):
+    """Both volumes hang off the same container, so an image-level exclusion
+    could only drop both. Only the named one may disappear."""
+
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+
+        cls.prefix = unique("baudolo-e2e-early-skip-no-backup-volume")
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+
+        cls.container = f"{cls.prefix}-app"
+        cls.excluded_volume = f"{cls.prefix}-derived-vol"
+        cls.kept_volume = f"{cls.prefix}-state-vol"
+
+        cls.containers = [cls.container]
+        cls.volumes = [cls.excluded_volume, cls.kept_volume]
+
+        run(["docker", "volume", "create", cls.excluded_volume])
+        run(["docker", "volume", "create", cls.kept_volume])
+
+        run(
+            [
+                "docker",
+                "run",
+                "--rm",
+                "-v",
+                f"{cls.excluded_volume}:/derived",
+                "-v",
+                f"{cls.kept_volume}:/state",
+                "alpine:3.20",
+                "sh",
+                "-lc",
+                "echo derived > /derived/derived.txt && echo state > /state/state.txt",
+            ]
+        )
+
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.container,
+                "-v",
+                f"{cls.excluded_volume}:/derived",
+                "-v",
+                f"{cls.kept_volume}:/state",
+                "alpine:3.20",
+                "sleep",
+                "600",
+            ]
+        )
+
+        cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
+        write_databases_csv(cls.databases_csv, [])
+
+        cmd = [
+            "baudolo",
+            "--compose-dir",
+            cls.compose_dir,
+            "--repo-name",
+            cls.repo_name,
+            "--databases-csv",
+            cls.databases_csv,
+            "--backups-dir",
+            cls.backups_dir,
+            "--images-no-stop-required",
+            "alpine:3.20",
+            "--volumes-no-backup-required",
+            cls.excluded_volume,
+        ]
+        cp = run(cmd, capture=True, check=True)
+        cls.stdout = cp.stdout or ""
+        cls.stderr = cp.stderr or ""
+
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=cls.containers, volumes=cls.volumes)
+
+    def test_excluded_volume_has_no_backup_directory_at_all(self) -> None:
+        p = backup_path(
+            self.backups_dir,
+            self.repo_name,
+            self.version,
+            self.excluded_volume,
+        )
+        self.assertFalse(
+            p.exists(),
+            f"Expected NO backup directory for the excluded volume, but found: {p}",
+        )
+
+    def test_sibling_volume_of_the_same_container_is_still_backed_up(self) -> None:
+        p = (
+            backup_path(
+                self.backups_dir,
+                self.repo_name,
+                self.version,
+                self.kept_volume,
+            )
+            / "files"
+            / "state.txt"
+        )
+        self.assertTrue(p.is_file(), f"Expected backed up file at: {p}")

tests/integration/test_seed_integration.py

@@ -7,9 +7,47 @@ from pathlib import Path
 
 
 def run_seed(
-    csv_path: Path, instance: str, database: str, username: str, password: str = ""
+    csv_path: Path, instance: str, database: str, username: str, password: str
 ) -> subprocess.CompletedProcess:
-    # Run the real CLI module (integration-style).
+    """
+    Run the real CLI module (E2E-style) using subprocess.
+
+    Seed contract (current):
+    - database must be "*" or a valid name (non-empty, matches allowed charset)
+    - password is required
+    - entry is keyed by (instance, database); username/password get updated
+    """
+    cp = subprocess.run(
+        [
+            sys.executable,
+            "-m",
+            "baudolo.seed",
+            str(csv_path),
+            instance,
+            database,
+            username,
+            password,
+        ],
+        text=True,
+        capture_output=True,
+        check=False,
+    )
+    if cp.returncode != 0:
+        raise AssertionError(
+            "seed command failed unexpectedly.\n"
+            f"returncode: {cp.returncode}\n"
+            f"stdout:\n{cp.stdout}\n"
+            f"stderr:\n{cp.stderr}\n"
+        )
+    return cp
+
+
+def run_seed_expect_fail(
+    csv_path: Path, instance: str, database: str, username: str, password: str
+) -> subprocess.CompletedProcess:
+    """
+    Same as run_seed, but expects non-zero exit. Returns CompletedProcess for inspection.
+    """
     return subprocess.run(
         [
             sys.executable,
@@ -23,7 +61,7 @@ def run_seed(
         ],
         text=True,
         capture_output=True,
-        check=True,
+        check=False,
     )
 
 
@@ -33,6 +71,10 @@ def read_csv_semicolon(path: Path) -> list[dict]:
         return list(reader)
 
 
+def read_text(path: Path) -> str:
+    return path.read_text(encoding="utf-8")
+
+
 class TestSeedIntegration(unittest.TestCase):
     def test_creates_file_and_adds_entry_when_missing(self) -> None:
         with tempfile.TemporaryDirectory() as td:
@@ -41,7 +83,7 @@ class TestSeedIntegration(unittest.TestCase):
 
             cp = run_seed(p, "docker.test", "appdb", "alice", "secret")
 
-            self.assertEqual(cp.returncode, 0, cp.stderr)
+            self.assertEqual(cp.returncode, 0)
             self.assertTrue(p.exists())
 
             rows = read_csv_semicolon(p)
@@ -51,40 +93,121 @@ class TestSeedIntegration(unittest.TestCase):
             self.assertEqual(rows[0]["username"], "alice")
             self.assertEqual(rows[0]["password"], "secret")
 
-    def test_replaces_existing_entry_same_keys(self) -> None:
+    def test_replaces_existing_entry_same_instance_and_database_updates_username_and_password(
+        self,
+    ) -> None:
+        """
+        Replacement semantics:
+        - Key is (instance, database)
+        - username/password are updated in-place
+        """
         with tempfile.TemporaryDirectory() as td:
             p = Path(td) / "databases.csv"
 
-            # First add
             run_seed(p, "docker.test", "appdb", "alice", "oldpw")
             rows = read_csv_semicolon(p)
             self.assertEqual(len(rows), 1)
+            self.assertEqual(rows[0]["username"], "alice")
             self.assertEqual(rows[0]["password"], "oldpw")
 
-            # Replace (same instance+database+username)
+            run_seed(p, "docker.test", "appdb", "bob", "newpw")
-            run_seed(p, "docker.test", "appdb", "alice", "newpw")
             rows = read_csv_semicolon(p)
 
             self.assertEqual(len(rows), 1, "Expected replacement, not a duplicate row")
             self.assertEqual(rows[0]["instance"], "docker.test")
             self.assertEqual(rows[0]["database"], "appdb")
-            self.assertEqual(rows[0]["username"], "alice")
+            self.assertEqual(rows[0]["username"], "bob")
             self.assertEqual(rows[0]["password"], "newpw")
 
-    def test_database_empty_string_matches_existing_empty_database(self) -> None:
+    def test_allows_star_database_for_dump_all(self) -> None:
         with tempfile.TemporaryDirectory() as td:
             p = Path(td) / "databases.csv"
 
-            # Add with empty database
+            cp = run_seed(p, "bigbluebutton", "*", "postgres", "pw")
-            run_seed(p, "docker.test", "", "alice", "pw1")
+            self.assertEqual(cp.returncode, 0)
+
             rows = read_csv_semicolon(p)
             self.assertEqual(len(rows), 1)
-            self.assertEqual(rows[0]["database"], "")
+            self.assertEqual(rows[0]["instance"], "bigbluebutton")
+            self.assertEqual(rows[0]["database"], "*")
+            self.assertEqual(rows[0]["username"], "postgres")
+            self.assertEqual(rows[0]["password"], "pw")
+
+    def test_replaces_existing_star_entry(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            p = Path(td) / "databases.csv"
+
+            run_seed(p, "bigbluebutton", "*", "postgres", "pw1")
+            run_seed(p, "bigbluebutton", "*", "postgres", "pw2")
 
-            # Replace with empty database again
-            run_seed(p, "docker.test", "", "alice", "pw2")
             rows = read_csv_semicolon(p)
-
             self.assertEqual(len(rows), 1)
-            self.assertEqual(rows[0]["database"], "")
+            self.assertEqual(rows[0]["database"], "*")
             self.assertEqual(rows[0]["password"], "pw2")
+
+    def test_rejects_empty_database_value(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            p = Path(td) / "databases.csv"
+
+            cp = run_seed_expect_fail(p, "docker.test", "", "alice", "pw")
+            self.assertNotEqual(cp.returncode, 0)
+
+            combined = ((cp.stdout or "") + "\n" + (cp.stderr or "")).lower()
+            self.assertIn("error:", combined)
+            self.assertIn("database", combined)
+            self.assertIn("not empty", combined)
+
+            self.assertFalse(p.exists(), "Should not create file on invalid input")
+
+    def test_rejects_invalid_database_name_characters(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            p = Path(td) / "databases.csv"
+
+            cp = run_seed_expect_fail(p, "docker.test", "app db", "alice", "pw")
+            self.assertNotEqual(cp.returncode, 0)
+
+            combined = ((cp.stdout or "") + "\n" + (cp.stderr or "")).lower()
+            self.assertIn("error:", combined)
+            self.assertIn("invalid database name", combined)
+
+            self.assertFalse(p.exists(), "Should not create file on invalid input")
+
+    def test_rejects_nan_database_name(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            p = Path(td) / "databases.csv"
+
+            cp = run_seed_expect_fail(p, "docker.test", "nan", "alice", "pw")
+            self.assertNotEqual(cp.returncode, 0)
+
+            combined = ((cp.stdout or "") + "\n" + (cp.stderr or "")).lower()
+            self.assertIn("error:", combined)
+            self.assertIn("must not be 'nan'", combined)
+
+            self.assertFalse(p.exists(), "Should not create file on invalid input")
+
+    def test_accepts_hyphen_and_underscore_database_names(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            p = Path(td) / "databases.csv"
+
+            run_seed(p, "docker.test", "my_db-1", "alice", "pw")
+
+            rows = read_csv_semicolon(p)
+            self.assertEqual(len(rows), 1)
+            self.assertEqual(rows[0]["database"], "my_db-1")
+
+    def test_file_is_semicolon_delimited_and_has_header(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            p = Path(td) / "databases.csv"
+
+            run_seed(p, "docker.test", "appdb", "alice", "pw")
+
+            txt = read_text(p)
+            self.assertTrue(
+                txt.startswith("instance;database;username;password"),
+                f"Unexpected header / delimiter in file:\n{txt}",
+            )
+            self.assertIn(";", txt)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/__init__.py

@@ -0,0 +1,10 @@
+"""The smallest argv the backup CLI accepts, shared by every test that drives it."""
+
+REQUIRED_PAIRS = [
+    ("--compose-dir", "/compose"),
+    ("--backups-dir", "/backups"),
+    ("--repo-name", "stack"),
+    ("--databases-csv", "/etc/baudolo/databases.csv"),
+]
+REQUIRED = [arg for pair in REQUIRED_PAIRS for arg in pair]
+BASE_ARGV = ["baudolo", *REQUIRED]

tests/unit/backup/compose_fixture.py

@@ -0,0 +1,47 @@
+"""Builds the on-disk compose directories the compose tests discover."""
+
+from __future__ import annotations
+
+import shutil
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+    from pathlib import Path
+
+
+def touch(p: Path) -> None:
+    p.parent.mkdir(parents=True, exist_ok=True)
+
+    # ".env/env" leaves ".env" behind as a directory, which blocks a later ".env" file.
+    if p.exists() and p.is_dir():
+        shutil.rmtree(p)
+
+    p.write_text("x", encoding="utf-8")
+
+
+def setup_compose_dir(
+    tmp_path: Path,
+    name: str = "mailu",
+    *,
+    compose_name: str = "docker-compose.yml",
+    with_override: bool = False,
+    with_ca_override: bool = False,
+    env_layout: str | None = None,  # None | ".env" | ".env/env"
+) -> Path:
+    d = tmp_path / name
+    d.mkdir(parents=True, exist_ok=True)
+
+    touch(d / compose_name)
+
+    if with_override:
+        touch(d / "docker-compose.override.yml")
+
+    if with_ca_override:
+        touch(d / "docker-compose.ca.override.yml")
+
+    if env_layout == ".env":
+        touch(d / ".env")
+    elif env_layout == ".env/env":
+        touch(d / ".env" / "env")
+
+    return d

tests/unit/backup/test_app_databases_csv.py

@@ -0,0 +1,75 @@
+import io
+import tempfile
+import unittest
+from contextlib import redirect_stderr
+from pathlib import Path
+
+import pandas as pd
+
+# Adjust if your package name/import path differs.
+from baudolo.backup.dumps import load_databases_df
+
+EXPECTED_COLUMNS = ["instance", "database", "username", "password"]
+
+
+class TestLoadDatabasesDf(unittest.TestCase):
+    def test_missing_csv_is_handled_with_warning_and_empty_df(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            missing_path = str(Path(td) / "does-not-exist.csv")
+
+            buf = io.StringIO()
+            with redirect_stderr(buf):
+                df = load_databases_df(missing_path)
+
+            stderr = buf.getvalue()
+            self.assertIn("WARNING:", stderr)
+            self.assertIn("databases.csv not found", stderr)
+
+            self.assertIsInstance(df, pd.DataFrame)
+            self.assertListEqual(list(df.columns), EXPECTED_COLUMNS)
+            self.assertTrue(df.empty)
+
+    def test_empty_csv_is_handled_with_warning_and_empty_df(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            empty_path = Path(td) / "databases.csv"
+            with empty_path.open("w", encoding="utf-8") as f:
+                f.write("")
+
+            buf = io.StringIO()
+            with redirect_stderr(buf):
+                df = load_databases_df(empty_path)
+
+            stderr = buf.getvalue()
+            self.assertIn("WARNING:", stderr)
+            self.assertIn("exists but is empty", stderr)
+
+            self.assertIsInstance(df, pd.DataFrame)
+            self.assertListEqual(list(df.columns), EXPECTED_COLUMNS)
+            self.assertTrue(df.empty)
+
+    def test_valid_csv_loads_without_warning(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            csv_path = Path(td) / "databases.csv"
+
+            content = "instance;database;username;password\nmyapp;*;dbuser;secret\n"
+            with csv_path.open("w", encoding="utf-8") as f:
+                f.write(content)
+
+            buf = io.StringIO()
+            with redirect_stderr(buf):
+                df = load_databases_df(csv_path)
+
+            stderr = buf.getvalue()
+            self.assertEqual(stderr, "")  # no warning expected
+
+            self.assertIsInstance(df, pd.DataFrame)
+            self.assertListEqual(list(df.columns), EXPECTED_COLUMNS)
+            self.assertEqual(len(df), 1)
+            self.assertEqual(df.loc[0, "instance"], "myapp")
+            self.assertEqual(df.loc[0, "database"], "*")
+            self.assertEqual(df.loc[0, "username"], "dbuser")
+            self.assertEqual(df.loc[0, "password"], "secret")
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_app_manifest.py

@@ -0,0 +1,84 @@
+"""What main() records in the manifest for each volume it touched."""
+
+from __future__ import annotations
+
+import unittest
+from unittest import mock
+
+from baudolo.backup import app
+from baudolo.backup.dumps import VolumeOutcome
+from baudolo.backup.volume import Backing
+
+from . import REQUIRED_PAIRS
+
+ARGV = ["baudolo", *[arg for pair in REQUIRED_PAIRS for arg in pair]]
+
+
+def drive(argv: list[str], dump_result: VolumeOutcome) -> dict:
+    """Run main() over one volume and return the manifest's volume section.
+
+    Args:
+        argv: the command line under test.
+        dump_result: what backup_dumps_for_volume reports.
+    """
+    with (
+        mock.patch("sys.argv", argv),
+        mock.patch.object(app, "get_machine_id", return_value="machine"),
+        mock.patch.object(app, "create_version_directory", return_value="/gen"),
+        mock.patch.object(app, "create_volume_directory", return_value="/gen/vol"),
+        mock.patch.object(app, "load_databases_df", return_value=None),
+        mock.patch.object(app, "docker_volume_names", return_value=["pgdata"]),
+        mock.patch.object(app, "containers_using_volume", return_value=["db"]),
+        mock.patch.object(app, "volume_is_fully_ignored", return_value=False),
+        mock.patch.object(app, "backup_dumps_for_volume", return_value=dump_result),
+        mock.patch.object(app, "inspect_backing", return_value=Backing("/data")),
+        mock.patch.object(app, "write_manifest") as manifest,
+        mock.patch.object(app, "stamp_directory"),
+        mock.patch.object(app, "handle_docker_compose_services"),
+        mock.patch("os.path.isdir", return_value=True),
+        mock.patch.object(app, "backup_volume"),
+        mock.patch.object(app, "filter_stoppable", return_value=[]),
+        mock.patch.object(app, "requires_stop", return_value=False),
+        mock.patch.object(app, "change_containers_status"),
+    ):
+        app.main()
+    return manifest.call_args.args[1]
+
+
+class TestManifest(unittest.TestCase):
+    def test_a_database_volume_without_a_dump_is_recorded_as_undumped(self) -> None:
+        volumes = drive(
+            [*ARGV, "--only-sql"],
+            VolumeOutcome(database=True, dumped=False, engine="postgres"),
+        )
+        self.assertEqual(
+            volumes["pgdata"],
+            VolumeOutcome(database=True, dumped=False, engine="postgres"),
+        )
+
+    def test_a_dumped_database_volume_is_recorded_as_dumped(self) -> None:
+        volumes = drive(
+            [*ARGV, "--only-sql"],
+            VolumeOutcome(database=True, dumped=True, engine="mariadb"),
+        )
+        self.assertEqual(volumes["pgdata"].dumped, True)
+        self.assertEqual(volumes["pgdata"].engine, "mariadb")
+
+    def test_a_plain_volume_is_recorded_as_no_database(self) -> None:
+        volumes = drive(ARGV, VolumeOutcome(database=False, dumped=False))
+        self.assertEqual(volumes["pgdata"].database, False)
+        self.assertIsNone(volumes["pgdata"].engine)
+
+    def test_the_dumped_volume_is_recorded_even_though_the_copy_is_skipped(
+        self,
+    ) -> None:
+        """--only-sql returns to the loop head on success, before the copy."""
+        volumes = drive(
+            [*ARGV, "--only-sql"],
+            VolumeOutcome(database=True, dumped=True, engine="postgres"),
+        )
+        self.assertIn("pgdata", volumes)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_app_only_files.py

@@ -0,0 +1,66 @@
+"""Contract of --only-files: no dump is attempted, every volume is copied."""
+
+from __future__ import annotations
+
+import unittest
+from unittest import mock
+
+from baudolo.backup import app
+from baudolo.backup.volume import Backing
+
+from . import REQUIRED_PAIRS
+
+ARGV_WITHOUT_CSV = [
+    "baudolo",
+    *[arg for pair in REQUIRED_PAIRS if pair[0] != "--databases-csv" for arg in pair],
+    "--only-files",
+]
+
+
+def drive(argv: list[str]) -> tuple[list[str], list, list]:
+    backed_up: list[str] = []
+
+    def record_backup(versions_dir, volume_name, volume_dir, *, authoritative, source):
+        backed_up.append(volume_name)
+
+    with (
+        mock.patch("sys.argv", argv),
+        mock.patch.object(app, "get_machine_id", return_value="machine"),
+        mock.patch.object(app, "create_version_directory", return_value="/gen"),
+        mock.patch.object(app, "create_volume_directory", return_value="/gen/vol"),
+        mock.patch.object(app, "load_databases_df") as load_csv,
+        mock.patch.object(app, "docker_volume_names", return_value=["pgdata"]),
+        mock.patch.object(app, "containers_using_volume", return_value=["db"]),
+        mock.patch.object(app, "volume_is_fully_ignored", return_value=False),
+        mock.patch.object(app, "backup_dumps_for_volume") as dumps,
+        mock.patch.object(app, "inspect_backing", return_value=Backing("/data")),
+        mock.patch.object(app, "write_manifest"),
+        mock.patch.object(app, "stamp_directory"),
+        mock.patch.object(app, "handle_docker_compose_services"),
+        mock.patch("os.path.isdir", return_value=True),
+        mock.patch.object(app, "backup_volume", side_effect=record_backup),
+        mock.patch.object(app, "filter_stoppable", return_value=[]),
+        mock.patch.object(app, "requires_stop", return_value=False),
+        mock.patch.object(app, "change_containers_status"),
+    ):
+        app.main()
+    return backed_up, dumps.mock_calls, load_csv.mock_calls
+
+
+class TestOnlyFiles(unittest.TestCase):
+    def test_no_dump_is_attempted(self) -> None:
+        _backed_up, dumps, _load_csv = drive(ARGV_WITHOUT_CSV)
+        self.assertEqual(dumps, [])
+
+    def test_the_databases_csv_is_never_read(self) -> None:
+        """It may legitimately be absent, so reading it would abort the run."""
+        _backed_up, _dumps, load_csv = drive(ARGV_WITHOUT_CSV)
+        self.assertEqual(load_csv, [])
+
+    def test_the_volume_is_still_copied(self) -> None:
+        backed_up, _dumps, _load_csv = drive(ARGV_WITHOUT_CSV)
+        self.assertEqual(backed_up, ["pgdata"])
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_app_snapshot.py

@@ -0,0 +1,92 @@
+"""Contract of app.main's snapshot branch - the caller that runs in production."""
+
+from __future__ import annotations
+
+import unittest
+from unittest import mock
+
+from baudolo.backup import app
+from baudolo.backup import snapshot as snapshot_mod
+from baudolo.backup.snapshot import volume_snapshot
+from baudolo.backup.volume import Backing
+
+from . import BASE_ARGV
+
+
+def stubbed_snapshot(kind: str, subject: str, tag: str):
+    return volume_snapshot(kind, subject, tag, run=lambda command: [])
+
+
+ARGV = [
+    *BASE_ARGV,
+    "--snapshot",
+    "btrfs",
+    "--snapshot-subject",
+    "/var/lib/docker",
+]
+
+
+def drive(*, present: bool = True, reason: str | None = None) -> list[dict]:
+    calls: list[dict] = []
+
+    def record(versions_dir, volume_name, volume_dir, *, authoritative, source):
+        calls.append(
+            {"volume": volume_name, "authoritative": authoritative, "source": source}
+        )
+
+    with (
+        mock.patch("sys.argv", ARGV),
+        mock.patch.object(app, "get_machine_id", return_value="machine"),
+        mock.patch.object(app, "create_version_directory", return_value="/gen"),
+        mock.patch.object(app, "create_volume_directory", return_value="/gen/vol"),
+        mock.patch.object(app, "load_databases_df", return_value=None),
+        mock.patch.object(app, "docker_volume_names", return_value=["vol"]),
+        mock.patch.object(app, "containers_using_volume", return_value=[]),
+        mock.patch.object(app, "volume_is_fully_ignored", return_value=False),
+        mock.patch.object(app, "backup_dumps_for_volume", return_value=(False, False)),
+        mock.patch.object(
+            app,
+            "inspect_backing",
+            return_value=Backing("/var/lib/docker/volumes/vol/_data"),
+        ),
+        mock.patch.object(snapshot_mod, "unsnapshotted", return_value=reason),
+        mock.patch.object(app, "write_manifest"),
+        mock.patch.object(app, "stamp_directory"),
+        mock.patch.object(app, "handle_docker_compose_services"),
+        mock.patch("os.path.isdir", return_value=present),
+        mock.patch.object(app, "backup_volume", side_effect=record),
+        mock.patch.object(app, "volume_snapshot", stubbed_snapshot),
+    ):
+        app.main()
+    return calls
+
+
+class TestSnapshotBranch(unittest.TestCase):
+    def test_it_passes_a_path_ending_in_a_separator(self) -> None:
+        source = drive(present=True)[0]["source"]
+        self.assertTrue(source.endswith("/volumes/vol/_data/"), source)
+        self.assertNotIn("/var/lib/docker/volumes", source)
+
+    def test_it_reads_from_the_snapshot_and_not_from_the_live_tree(self) -> None:
+        source = drive(present=True)[0]["source"]
+        self.assertTrue(source.startswith("/var/lib/docker/.baudolo-"), source)
+
+    def test_it_compares_by_content_against_the_previous_generation(self) -> None:
+        self.assertTrue(drive(present=True)[0]["authoritative"])
+
+    def test_a_volume_missing_from_the_snapshot_is_copied_live(self) -> None:
+        call = drive(present=False)[0]
+        self.assertEqual(call["source"], "/var/lib/docker/volumes/vol/_data/")
+        self.assertFalse(call["authoritative"])
+
+    def test_a_volume_with_its_own_backing_store_is_copied_live(self) -> None:
+        call = drive(reason="it declares its own backing store")[0]
+        self.assertEqual(call["source"], "/var/lib/docker/volumes/vol/_data/")
+        self.assertFalse(
+            call["authoritative"],
+            "the snapshot holds an empty directory for it, not its data",
+        )
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_app_volumes_no_backup_required.py

@@ -0,0 +1,82 @@
+"""Contract of --volumes-no-backup-required: exclusion is per volume name,
+independent of which containers use it."""
+
+from __future__ import annotations
+
+import unittest
+from unittest import mock
+
+from baudolo.backup import app
+from baudolo.backup.volume import Backing
+
+from . import BASE_ARGV
+
+ARGV = [
+    *BASE_ARGV,
+    "--volumes-no-backup-required",
+    "derived",
+]
+
+
+def drive() -> tuple[list[str], list[str], list[str]]:
+    backed_up: list[str] = []
+    created: list[str] = []
+    inspected: list[str] = []
+
+    def record_backup(versions_dir, volume_name, volume_dir, *, authoritative, source):
+        backed_up.append(volume_name)
+
+    with (
+        mock.patch("sys.argv", ARGV),
+        mock.patch.object(app, "get_machine_id", return_value="machine"),
+        mock.patch.object(app, "create_version_directory", return_value="/gen"),
+        mock.patch.object(
+            app,
+            "create_volume_directory",
+            side_effect=lambda _version_dir, name: created.append(name) or "/gen/vol",
+        ),
+        mock.patch.object(app, "load_databases_df", return_value=None),
+        mock.patch.object(
+            app, "docker_volume_names", return_value=["derived", "state"]
+        ),
+        mock.patch.object(
+            app,
+            "containers_using_volume",
+            side_effect=lambda name: inspected.append(name) or ["app"],
+        ),
+        mock.patch.object(app, "volume_is_fully_ignored", return_value=False),
+        mock.patch.object(app, "backup_dumps_for_volume", return_value=(False, False)),
+        mock.patch.object(app, "inspect_backing", return_value=Backing("/data")),
+        mock.patch.object(app, "write_manifest"),
+        mock.patch.object(app, "stamp_directory"),
+        mock.patch.object(app, "handle_docker_compose_services"),
+        mock.patch("os.path.isdir", return_value=True),
+        mock.patch.object(app, "backup_volume", side_effect=record_backup),
+        mock.patch.object(app, "filter_stoppable", return_value=[]),
+        mock.patch.object(app, "requires_stop", return_value=False),
+        mock.patch.object(app, "change_containers_status"),
+    ):
+        app.main()
+    return backed_up, created, inspected
+
+
+class TestVolumesNoBackupRequired(unittest.TestCase):
+    def test_the_named_volume_is_never_backed_up(self) -> None:
+        backed_up, _created, _inspected = drive()
+        self.assertNotIn("derived", backed_up)
+
+    def test_a_sibling_volume_of_the_same_container_survives(self) -> None:
+        backed_up, _created, _inspected = drive()
+        self.assertEqual(backed_up, ["state"])
+
+    def test_no_generation_directory_is_created_for_it(self) -> None:
+        _backed_up, created, _inspected = drive()
+        self.assertEqual(created, ["state"])
+
+    def test_the_skip_precedes_the_container_inspection(self) -> None:
+        _backed_up, _created, inspected = drive()
+        self.assertEqual(inspected, ["state"])
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_cli.py

@@ -0,0 +1,110 @@
+"""Contract of the backup CLI, in particular the snapshot flag pairing."""
+
+from __future__ import annotations
+
+import unittest
+from unittest import mock
+
+from baudolo.backup.cli import parse_args
+
+from . import REQUIRED, REQUIRED_PAIRS
+
+
+def parse(*extra: str):
+    with mock.patch("sys.argv", ["baudolo", *REQUIRED, *extra]):
+        return parse_args()
+
+
+class TestSnapshotFlags(unittest.TestCase):
+    def test_no_snapshot_by_default(self) -> None:
+        args = parse()
+        self.assertIsNone(args.snapshot)
+        self.assertIsNone(args.snapshot_subject)
+
+    def test_both_flags_together_are_accepted(self) -> None:
+        args = parse("--snapshot", "btrfs", "--snapshot-subject", "/var/lib/docker")
+        self.assertEqual(args.snapshot, "btrfs")
+        self.assertEqual(args.snapshot_subject, "/var/lib/docker")
+
+    def test_the_kind_alone_is_rejected(self) -> None:
+        with self.assertRaises(SystemExit):
+            parse("--snapshot", "btrfs")
+
+    def test_the_subject_alone_is_rejected(self) -> None:
+        with self.assertRaises(SystemExit):
+            parse("--snapshot-subject", "/var/lib/docker")
+
+    def test_an_unsupported_kind_is_rejected(self) -> None:
+        with self.assertRaises(SystemExit):
+            parse("--snapshot", "ext4", "--snapshot-subject", "/var/lib/docker")
+
+    def test_zfs_is_accepted(self) -> None:
+        self.assertEqual(
+            parse("--snapshot", "zfs", "--snapshot-subject", "/d").snapshot, "zfs"
+        )
+
+    def test_shutdown_is_rejected_because_nothing_is_stopped(self) -> None:
+        with self.assertRaises(SystemExit):
+            parse("--snapshot", "btrfs", "--snapshot-subject", "/d", "--shutdown")
+
+    def test_shutdown_stays_available_without_a_snapshot(self) -> None:
+        self.assertTrue(parse("--shutdown").shutdown)
+
+    def test_hard_restart_is_rejected_because_nothing_is_stopped(self) -> None:
+        with self.assertRaises(SystemExit):
+            parse(
+                "--snapshot",
+                "btrfs",
+                "--snapshot-subject",
+                "/d",
+                "--hard-restart-projects",
+                "mailu",
+            )
+
+    def test_hard_restart_stays_available_without_a_snapshot(self) -> None:
+        self.assertEqual(
+            parse("--hard-restart-projects", "mailu").hard_restart_projects, ["mailu"]
+        )
+
+
+class TestRequiredFlags(unittest.TestCase):
+    def test_no_flag_falls_back_to_a_default(self) -> None:
+        for omitted, _ in REQUIRED_PAIRS:
+            argv = [a for pair in REQUIRED_PAIRS if pair[0] != omitted for a in pair]
+            with (
+                self.subTest(omitted=omitted),
+                mock.patch("sys.argv", ["baudolo", *argv]),
+                self.assertRaises(SystemExit),
+            ):
+                parse_args()
+
+
+class TestBackupScope(unittest.TestCase):
+    """--only-sql and --only-files name the two halves a generation can hold."""
+
+    def test_both_halves_by_default(self) -> None:
+        args = parse()
+        self.assertFalse(args.only_sql)
+        self.assertFalse(args.only_files)
+
+    def test_either_half_alone_is_accepted(self) -> None:
+        self.assertTrue(parse("--only-sql").only_sql)
+        self.assertTrue(parse("--only-files").only_files)
+
+    def test_asking_for_both_halves_alone_is_rejected(self) -> None:
+        with self.assertRaises(SystemExit):
+            parse("--only-sql", "--only-files")
+
+    def test_only_files_needs_no_databases_csv(self) -> None:
+        argv = [
+            arg
+            for pair in REQUIRED_PAIRS
+            if pair[0] != "--databases-csv"
+            for arg in pair
+        ]
+        with mock.patch("sys.argv", ["baudolo", *argv, "--only-files"]):
+            self.assertIsNone(parse_args().databases_csv)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_compose.py

@@ -0,0 +1,212 @@
+from __future__ import annotations
+
+import tempfile
+import unittest
+from pathlib import Path
+from unittest.mock import patch
+
+from .compose_fixture import setup_compose_dir as _setup_compose_dir
+
+
+class TestCompose(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        from baudolo.backup import compose as mod
+
+        cls.compose_mod = mod
+
+    def test_find_compose_file_supports_all_valid_names_case_insensitive(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            tmp_path = Path(td)
+
+            variants = [
+                "compose.yml",
+                "compose.yaml",
+                "docker-compose.yml",
+                "docker-compose.yaml",
+                "docker-compose.yAml",
+            ]
+
+            for i, name in enumerate(variants):
+                d = _setup_compose_dir(
+                    tmp_path,
+                    name=f"project{i}",
+                    compose_name=name,
+                )
+                found = self.compose_mod._find_compose_file(str(d))
+                self.assertIsNotNone(found)
+                self.assertEqual(found.name, name)
+
+    def test_find_compose_file_returns_none_when_missing(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            tmp_path = Path(td)
+            d = tmp_path / "empty"
+            d.mkdir(parents=True, exist_ok=True)
+
+            found = self.compose_mod._find_compose_file(str(d))
+            self.assertIsNone(found)
+
+    def test_build_cmd_uses_wrapper_when_present(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            tmp_path = Path(td)
+            d = _setup_compose_dir(
+                tmp_path,
+                with_override=True,
+                with_ca_override=True,
+                env_layout=".env",
+            )
+
+            def fake_which(name: str):
+                if name == "compose":
+                    return "/usr/local/bin/compose"
+                return None
+
+            with patch.object(self.compose_mod.shutil, "which", fake_which):
+                cmd = self.compose_mod._build_compose_cmd(str(d), ["up", "-d"])
+
+            self.assertEqual(
+                cmd,
+                [
+                    "/usr/local/bin/compose",
+                    "--chdir",
+                    str(d.resolve()),
+                    "--",
+                    "up",
+                    "-d",
+                ],
+            )
+
+    def test_build_cmd_fallback_uses_plain_docker_compose_chdir(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            tmp_path = Path(td)
+            d = _setup_compose_dir(
+                tmp_path,
+                with_override=True,
+                with_ca_override=True,
+                env_layout=".env",
+            )
+
+            def fake_which(name: str):
+                if name == "compose":
+                    return None
+                if name == "docker":
+                    return "/usr/bin/docker"
+                return None
+
+            with patch.object(self.compose_mod.shutil, "which", fake_which):
+                cmd = self.compose_mod._build_compose_cmd(
+                    str(d), ["up", "-d", "--force-recreate"]
+                )
+
+            expected: list[str] = [
+                "/usr/bin/docker",
+                "compose",
+                "--chdir",
+                str(d.resolve()),
+                "up",
+                "-d",
+                "--force-recreate",
+            ]
+            self.assertEqual(cmd, expected)
+
+    def test_hard_restart_calls_run_twice_with_correct_cmds_wrapper(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            tmp_path = Path(td)
+            d = _setup_compose_dir(tmp_path, name="mailu", env_layout=".env")
+
+            def fake_which(name: str):
+                if name == "compose":
+                    return "/usr/local/bin/compose"
+                return None
+
+            with patch.object(self.compose_mod.shutil, "which", fake_which):
+                calls = []
+
+                def fake_run(cmd, check: bool):
+                    calls.append((cmd, check))
+                    return 0
+
+                with patch.object(self.compose_mod.subprocess, "run", fake_run):
+                    self.compose_mod.hard_restart_docker_services(str(d))
+
+            self.assertEqual(
+                calls,
+                [
+                    (
+                        [
+                            "/usr/local/bin/compose",
+                            "--chdir",
+                            str(d.resolve()),
+                            "--",
+                            "down",
+                        ],
+                        True,
+                    ),
+                    (
+                        [
+                            "/usr/local/bin/compose",
+                            "--chdir",
+                            str(d.resolve()),
+                            "--",
+                            "up",
+                            "-d",
+                        ],
+                        True,
+                    ),
+                ],
+            )
+
+    def test_hard_restart_calls_run_twice_with_correct_cmds_fallback(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            tmp_path = Path(td)
+            d = _setup_compose_dir(
+                tmp_path,
+                name="mailu",
+                with_override=True,
+                with_ca_override=True,
+                env_layout=".env/env",
+            )
+
+            def fake_which(name: str):
+                if name == "compose":
+                    return None
+                if name == "docker":
+                    return "/usr/bin/docker"
+                return None
+
+            with patch.object(self.compose_mod.shutil, "which", fake_which):
+                calls = []
+
+                def fake_run(cmd, check: bool):
+                    calls.append((cmd, check))
+                    return 0
+
+                with patch.object(self.compose_mod.subprocess, "run", fake_run):
+                    self.compose_mod.hard_restart_docker_services(str(d))
+
+            self.assertEqual(
+                calls,
+                [
+                    (
+                        [
+                            "/usr/bin/docker",
+                            "compose",
+                            "--chdir",
+                            str(d.resolve()),
+                            "down",
+                        ],
+                        True,
+                    ),
+                    (
+                        [
+                            "/usr/bin/docker",
+                            "compose",
+                            "--chdir",
+                            str(d.resolve()),
+                            "up",
+                            "-d",
+                        ],
+                        True,
+                    ),
+                ],
+            )

tests/unit/backup/test_compose_hard_restart.py

@@ -0,0 +1,45 @@
+from __future__ import annotations
+
+import unittest
+from unittest.mock import patch
+
+from . import BASE_ARGV
+
+
+class HardRestartArgTests(unittest.TestCase):
+    """The hard-restart list defaults to empty (no compose down/up); callers
+    opt in per dir, e.g. compose hosts pass 'mailu' while swarm hosts, where
+    the dir is a stack whose overlay network collides with compose up, pass
+    nothing."""
+
+    def _parse(self, extra: list[str]):
+        import sys
+
+        from baudolo.backup import cli
+
+        argv = [
+            *BASE_ARGV,
+            "--database-containers",
+            "postgres",
+            "--images-no-stop-required",
+            "redis",
+            *extra,
+        ]
+        with patch.object(sys, "argv", argv):
+            return cli.parse_args()
+
+    def test_default_is_empty(self) -> None:
+        args = self._parse([])
+        self.assertEqual(args.hard_restart_projects, [])
+
+    def test_empty_flag_stays_empty(self) -> None:
+        args = self._parse(["--hard-restart-projects"])
+        self.assertEqual(args.hard_restart_projects, [])
+
+    def test_explicit_names_preserved(self) -> None:
+        args = self._parse(["--hard-restart-projects", "mailu", "foo"])
+        self.assertEqual(args.hard_restart_projects, ["mailu", "foo"])
+
+
+if __name__ == "__main__":
+    unittest.main(verbosity=2)

tests/unit/backup/test_db_mariadb_dump.py

@@ -0,0 +1,103 @@
+import tempfile
+import unittest
+from unittest.mock import patch
+
+import pandas as pd
+
+from baudolo.backup import db as db_mod
+
+
+def _df(rows):
+    return pd.DataFrame(rows, columns=["instance", "database", "username", "password"])
+
+
+def _capture_dumps(*, db_type, rows, container, dump_tool="mariadb-dump"):
+    """Every (argv, env) the dump path would have run."""
+    captured = []
+
+    def _capture(command, out_file, *, env=None):
+        captured.append((list(command), env))
+
+    with (
+        tempfile.TemporaryDirectory() as td,
+        patch.object(db_mod, "execute_to_file", side_effect=_capture),
+    ):
+        db_mod.backup_database(
+            container=container,
+            volume_dir=td,
+            db_type=db_type,
+            dump_tool=dump_tool,
+            databases_df=_df(rows),
+            database_containers=[container],
+        )
+    return captured
+
+
+class TestMariaDBDumpUsesTCP(unittest.TestCase):
+    # Regression guard for 'Access denied for user <user>@localhost' when only
+    # '<user>'@'%' is granted: the in-container mariadb-dump MUST force TCP so
+    # the connection is auth-matched against '%' instead of socket->localhost.
+
+    def test_mariadb_dump_forces_tcp_loopback(self):
+        captured = _capture_dumps(
+            db_type="mariadb",
+            rows=[("mariadb", "appdb", "appuser", "s3cret")],
+            container="mariadb",
+        )
+        self.assertEqual(len(captured), 1, f"expected one dump, got: {captured}")
+
+        argv, env = captured[0]
+        self.assertEqual(argv[:3], ["docker", "exec", "mariadb"])
+        self.assertIn("--protocol=tcp", argv)
+        self.assertEqual(argv[argv.index("-h") + 1], "127.0.0.1")
+        self.assertEqual(argv[argv.index("-u") + 1], "appuser")
+        self.assertIn("-ps3cret", argv)
+        self.assertEqual(argv[-1], "appdb")
+        self.assertIsNone(env)
+
+    def test_the_probed_client_is_the_one_invoked(self):
+        captured = _capture_dumps(
+            db_type="mariadb",
+            rows=[("mariadb", "appdb", "appuser", "s3cret")],
+            container="mariadb",
+            dump_tool="mysqldump",
+        )
+        argv, _env = captured[0]
+        self.assertIn("mysqldump", argv)
+        self.assertNotIn("mariadb-dump", argv)
+
+    def test_postgres_dump_unaffected(self):
+        captured = _capture_dumps(
+            db_type="postgres",
+            rows=[("pg", "appdb", "appuser", "s3cret")],
+            container="pg",
+        )
+        argv, _env = captured[0]
+        self.assertIn("pg_dump", argv)
+        self.assertNotIn("--protocol=tcp", argv)
+
+    def test_the_password_travels_in_the_environment_not_the_argv(self):
+        """A process listing shows argv; PGPASSWORD must not be in it."""
+        captured = _capture_dumps(
+            db_type="postgres",
+            rows=[("pg", "appdb", "appuser", "s3cret")],
+            container="pg",
+        )
+        argv, env = captured[0]
+        self.assertEqual(env, {"PGPASSWORD": "s3cret"})
+        self.assertNotIn("s3cret", argv)
+
+
+class TestNoShellReachesTheDump(unittest.TestCase):
+    def test_a_hostile_database_name_never_reaches_a_command(self):
+        """validate_database refuses it, so no argv is built at all."""
+        with self.assertRaises(ValueError):
+            _capture_dumps(
+                db_type="postgres",
+                rows=[("pg", "app;rm -rf /", "appuser", "s3cret")],
+                container="pg",
+            )
+
+
+if __name__ == "__main__":
+    unittest.main(verbosity=2)

tests/unit/backup/test_docker_exec_env.py

@@ -0,0 +1,61 @@
+"""How a secret reaches the command running inside the container."""
+
+from __future__ import annotations
+
+import unittest
+
+from baudolo.backup.db import fallback_pg_dumpall
+from baudolo.backup.docker import docker_exec_argv
+
+
+class TestForwardEnv(unittest.TestCase):
+    def test_nothing_is_added_when_no_variable_is_named(self) -> None:
+        self.assertEqual(
+            docker_exec_argv("c1", ["true"]),
+            ["docker", "exec", "c1", "true"],
+        )
+
+    def test_a_named_variable_is_forwarded_without_its_value(self) -> None:
+        """-e NAME=value would publish the secret in the host's process list."""
+        argv = docker_exec_argv("c1", ["true"], forward_env=["PGPASSWORD"])
+        self.assertEqual(argv, ["docker", "exec", "-e", "PGPASSWORD", "c1", "true"])
+
+    def test_the_flag_precedes_the_container(self) -> None:
+        """docker reads options before the container name, arguments after it."""
+        argv = docker_exec_argv(
+            "c1", ["pg_dump", "-U", "u"], interactive=True, forward_env=["PGPASSWORD"]
+        )
+        self.assertLess(argv.index("-e"), argv.index("c1"))
+        self.assertLess(argv.index("-i"), argv.index("c1"))
+        self.assertGreater(argv.index("pg_dump"), argv.index("c1"))
+
+    def test_several_variables_each_get_their_own_flag(self) -> None:
+        argv = docker_exec_argv("c1", ["true"], forward_env=["A", "B"])
+        self.assertEqual(argv[:6], ["docker", "exec", "-e", "A", "-e", "B"])
+
+
+class TestPostgresDumpCarriesThePassword(unittest.TestCase):
+    def test_the_cluster_dump_forwards_pgpassword(self) -> None:
+        seen: dict = {}
+
+        def fake(command, out_file, *, env=None):
+            seen["command"] = command
+            seen["env"] = env
+
+        import baudolo.backup.db as db
+
+        original = db.execute_to_file
+        db.execute_to_file = fake
+        try:
+            fallback_pg_dumpall("pg", "user", "secret", "/tmp/out.sql")
+        finally:
+            db.execute_to_file = original
+
+        self.assertIn("-e", seen["command"])
+        self.assertEqual(seen["command"][seen["command"].index("-e") + 1], "PGPASSWORD")
+        self.assertEqual(seen["env"], {"PGPASSWORD": "secret"})
+        self.assertNotIn("secret", seen["command"])
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_docker_swarm.py

@@ -0,0 +1,53 @@
+import unittest
+from unittest.mock import patch
+
+from baudolo.backup import docker as docker_mod
+from baudolo.backup.shell import BackupError
+
+
+class TestIsSwarmTask(unittest.TestCase):
+    @patch.object(docker_mod, "execute_shell_command", return_value=["task-id-123"])
+    def test_true_when_task_label_present(self, _mock) -> None:
+        self.assertTrue(docker_mod.is_swarm_task("c1"))
+
+    @patch.object(docker_mod, "execute_shell_command", return_value=[""])
+    def test_false_when_label_empty(self, _mock) -> None:
+        self.assertFalse(docker_mod.is_swarm_task("c1"))
+
+    @patch.object(docker_mod, "execute_shell_command", return_value=[])
+    def test_false_when_no_output(self, _mock) -> None:
+        self.assertFalse(docker_mod.is_swarm_task("c1"))
+
+    @patch.object(
+        docker_mod,
+        "execute_shell_command",
+        side_effect=[BackupError("gone"), []],
+    )
+    def test_vanished_container_counts_as_not_stoppable(self, _mock) -> None:
+        # A container removed between listing and inspect must not abort the
+        # whole backup run; treating it as a swarm task keeps it out of every
+        # stop/start and image-inspect path.
+        self.assertTrue(docker_mod.is_swarm_task("gone-container"))
+
+    @patch.object(
+        docker_mod,
+        "execute_shell_command",
+        side_effect=[BackupError("daemon hiccup"), ["still-here"]],
+    )
+    def test_inspect_failure_on_existing_container_still_fails(self, _mock) -> None:
+        # If the container still exists, an inspect failure must keep failing
+        # the run: silently skipping the stop would back up a hot volume and
+        # report green without the stop guarantee.
+        with self.assertRaises(BackupError):
+            docker_mod.is_swarm_task("still-here")
+
+
+class TestFilterStoppable(unittest.TestCase):
+    @patch.object(docker_mod, "is_swarm_task", side_effect=[False, True, False])
+    def test_mixed_list_keeps_order_and_drops_tasks(self, _mock) -> None:
+        result = docker_mod.filter_stoppable(["plain-1", "swarm-task", "plain-2"])
+        self.assertEqual(result, ["plain-1", "plain-2"])
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_docker_tool_probe.py

@@ -0,0 +1,44 @@
+import unittest
+from unittest.mock import patch
+
+from baudolo.backup import docker as docker_mod
+from baudolo.backup.shell import BackupError
+
+
+class TestImageId(unittest.TestCase):
+    def test_the_id_is_returned_without_surrounding_whitespace(self) -> None:
+        with patch.object(
+            docker_mod, "execute_shell_command", return_value=["sha256:abc \n"]
+        ):
+            self.assertEqual(docker_mod.image_id("c1"), "sha256:abc")
+
+
+class TestHasTool(unittest.TestCase):
+    def test_a_tool_that_runs_is_present(self) -> None:
+        with patch.object(docker_mod, "execute_shell_command", return_value=[]):
+            self.assertTrue(docker_mod.has_tool("c1", "pg_dumpall"))
+
+    def test_a_tool_that_exits_non_zero_is_absent(self) -> None:
+        with patch.object(
+            docker_mod, "execute_shell_command", side_effect=BackupError("127")
+        ):
+            self.assertFalse(docker_mod.has_tool("c1", "mariadb-dump"))
+
+    def test_the_probe_needs_no_shell_in_the_image(self) -> None:
+        """A distroless database ships no shell; `sh -c` would deny every tool."""
+        captured = []
+
+        def _capture(cmd):
+            captured.append(cmd)
+            return []
+
+        with patch.object(docker_mod, "execute_shell_command", side_effect=_capture):
+            docker_mod.has_tool("c1", "pg_dumpall")
+
+        self.assertEqual(
+            captured, [["docker", "exec", "c1", "pg_dumpall", "--version"]]
+        )
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_dumps_engine_detection.py

@@ -0,0 +1,137 @@
+import unittest
+from unittest.mock import patch
+
+import pandas as pd
+
+from baudolo.backup import dumps as dumps_mod
+
+
+def _df(rows):
+    return pd.DataFrame(rows, columns=["instance", "database", "username", "password"])
+
+
+class _Probe:
+    def __init__(self, available, image="sha256:aaa"):
+        self.available = set(available)
+        self.image = image
+        self.calls = []
+
+    def has_tool(self, container, tool):
+        self.calls.append((container, tool))
+        return tool in self.available
+
+    def image_id(self, container):
+        return self.image if isinstance(self.image, str) else self.image[container]
+
+
+def _detect(probe, container="c1"):
+    dumps_mod._ENGINE_BY_IMAGE.clear()
+    with (
+        patch.object(dumps_mod, "has_tool", probe.has_tool),
+        patch.object(dumps_mod, "image_id", probe.image_id),
+    ):
+        return dumps_mod.container_engine(container)
+
+
+class TestContainerEngine(unittest.TestCase):
+    def test_a_postgres_is_found_by_its_dump_tool(self):
+        self.assertEqual(_detect(_Probe(["pg_dumpall"])), ("postgres", "pg_dumpall"))
+
+    def test_a_mariadb_is_found_by_its_dump_tool(self):
+        self.assertEqual(_detect(_Probe(["mariadb-dump"])), ("mariadb", "mariadb-dump"))
+
+    def test_an_image_with_only_mysqldump_is_dumped_with_mysqldump(self):
+        self.assertEqual(_detect(_Probe(["mysqldump"])), ("mariadb", "mysqldump"))
+
+    def test_a_container_without_either_tool_is_no_database(self):
+        self.assertIsNone(_detect(_Probe([])))
+
+    def test_the_probe_stops_at_the_first_tool_it_finds(self):
+        probe = _Probe(["pg_dumpall", "mariadb-dump"])
+        _detect(probe)
+        self.assertEqual(probe.calls, [("c1", "pg_dumpall")])
+
+    def test_the_image_name_does_not_decide_the_engine(self):
+        """The trap the old substring test fell into, from both directions."""
+        probe = _Probe(["pg_dumpall"], image="svc-db-mariadb-mgr-01:5000/pg_custom")
+        self.assertEqual(_detect(probe), ("postgres", "pg_dumpall"))
+
+        probe = _Probe(["mariadb-dump"], image="discourse-database:17")
+        self.assertEqual(_detect(probe), ("mariadb", "mariadb-dump"))
+
+
+class TestProbeCache(unittest.TestCase):
+    def test_replicas_of_one_image_are_probed_once(self):
+        probe = _Probe(["pg_dumpall"])
+        dumps_mod._ENGINE_BY_IMAGE.clear()
+        with (
+            patch.object(dumps_mod, "has_tool", probe.has_tool),
+            patch.object(dumps_mod, "image_id", probe.image_id),
+        ):
+            first = dumps_mod.container_engine("replica-1")
+            second = dumps_mod.container_engine("replica-2")
+        self.assertEqual(first, second)
+        self.assertEqual(len(probe.calls), 1)
+
+    def test_a_second_image_is_probed_separately(self):
+        probe = _Probe(["pg_dumpall"], image={"pg": "sha256:aaa", "app": "sha256:bbb"})
+        dumps_mod._ENGINE_BY_IMAGE.clear()
+        with (
+            patch.object(dumps_mod, "has_tool", probe.has_tool),
+            patch.object(dumps_mod, "image_id", probe.image_id),
+        ):
+            self.assertEqual(
+                dumps_mod.container_engine("pg"), ("postgres", "pg_dumpall")
+            )
+            probe.available = set()
+            self.assertIsNone(dumps_mod.container_engine("app"))
+
+
+class TestBackupDispatch(unittest.TestCase):
+    def test_the_probed_tool_reaches_the_dump(self):
+        probe = _Probe(["mysqldump"])
+        seen = {}
+
+        def _fake_backup_database(**kwargs):
+            seen.update(kwargs)
+            return True
+
+        dumps_mod._ENGINE_BY_IMAGE.clear()
+        with (
+            patch.object(dumps_mod, "has_tool", probe.has_tool),
+            patch.object(dumps_mod, "image_id", probe.image_id),
+            patch.object(dumps_mod, "backup_database", _fake_backup_database),
+        ):
+            outcome = dumps_mod.backup_mariadb_or_postgres(
+                container="c1",
+                volume_dir="/tmp",
+                databases_df=_df([("c1", "appdb", "u", "p")]),
+                database_containers=["c1"],
+            )
+
+        self.assertTrue(outcome.database)
+        self.assertTrue(outcome.dumped)
+        self.assertEqual(outcome.engine, "mariadb")
+        self.assertEqual(seen["db_type"], "mariadb")
+        self.assertEqual(seen["dump_tool"], "mysqldump")
+
+    def test_a_non_database_container_is_left_to_the_file_backup(self):
+        probe = _Probe([])
+        dumps_mod._ENGINE_BY_IMAGE.clear()
+        with (
+            patch.object(dumps_mod, "has_tool", probe.has_tool),
+            patch.object(dumps_mod, "image_id", probe.image_id),
+        ):
+            self.assertEqual(
+                dumps_mod.backup_mariadb_or_postgres(
+                    container="c1",
+                    volume_dir="/tmp",
+                    databases_df=_df([]),
+                    database_containers=[],
+                ),
+                dumps_mod.VolumeOutcome(database=False, dumped=False, engine=None),
+            )
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_get_instance.py

@@ -0,0 +1,80 @@
+"""Which databases.csv instance a container name resolves to.
+
+The cases are the container names real deployments produce, in both compose
+and swarm, so a change to the regex has to state which shape it gives up.
+"""
+
+from __future__ import annotations
+
+import unittest
+
+from baudolo.backup.db import get_instance
+
+
+class TestDeclaredContainers(unittest.TestCase):
+    def test_a_declared_container_is_its_own_instance(self) -> None:
+        self.assertEqual(
+            get_instance("postgres-central", ["postgres-central"]), "postgres-central"
+        )
+
+    def test_a_declaration_beats_the_regex(self) -> None:
+        """A declared name is taken whole even when it carries a token the
+        fallback would otherwise strip."""
+        self.assertEqual(
+            get_instance("shop-database", ["shop-database"]), "shop-database"
+        )
+
+    def test_a_qualified_central_name_still_has_to_be_declared(self) -> None:
+        self.assertIsNone(get_instance("postgres-central", []))
+
+
+class TestContainersNamedAfterTheirEngine(unittest.TestCase):
+    def test_a_bare_engine_name_is_its_own_instance(self) -> None:
+        for name in ("postgres", "mariadb", "mysql", "db", "database"):
+            with self.subTest(container=name):
+                self.assertEqual(get_instance(name, []), name)
+
+    def test_a_swarm_task_of_such_a_container_keeps_the_instance(self) -> None:
+        self.assertEqual(get_instance("postgres_postgres.1.k3f9x2", []), "postgres")
+        self.assertEqual(get_instance("mariadb_mariadb.1.k3f9x2", []), "mariadb")
+
+
+class TestDedicatedEngines(unittest.TestCase):
+    def test_compose_names_the_container_with_a_hyphen(self) -> None:
+        self.assertEqual(get_instance("discourse-database", []), "discourse")
+
+    def test_swarm_names_the_task_with_an_underscore_and_a_slot(self) -> None:
+        """Swarm suppresses container_name and names the task
+        <stack>_<service>.<slot>.<id>, which must land on the same instance as
+        the compose name so one databases.csv serves both modes."""
+        self.assertEqual(get_instance("discourse_database.1.k3f9x2", []), "discourse")
+
+    def test_an_explicitly_named_engine_keeps_its_entity(self) -> None:
+        self.assertEqual(get_instance("bigbluebutton-postgres-1", []), "bigbluebutton")
+
+    def test_the_short_token_is_stripped_too(self) -> None:
+        self.assertEqual(get_instance("matomo-db", []), "matomo")
+
+    def test_mariadb_uses_the_same_suffix(self) -> None:
+        self.assertEqual(get_instance("matomo-database", []), "matomo")
+
+    def test_an_engine_named_suffix_is_stripped_too(self) -> None:
+        self.assertEqual(get_instance("shop-mariadb", []), "shop")
+        self.assertEqual(get_instance("shop-mysql", []), "shop")
+
+
+class TestApplicationContainers(unittest.TestCase):
+    def test_a_bare_application_name_is_not_a_database(self) -> None:
+        """Returning the name unchanged here would offer the application as a
+        second engine for its own dedicated database's instance."""
+        self.assertIsNone(get_instance("discourse", []))
+
+    def test_a_swarm_application_task_is_not_a_database(self) -> None:
+        self.assertIsNone(get_instance("discourse_discourse.1.k3f9x2", []))
+
+    def test_an_application_that_merely_starts_with_a_token_is_not_split(self) -> None:
+        self.assertIsNone(get_instance("dbeaver", []))
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_layout.py

@@ -0,0 +1,53 @@
+"""Contract of where a backup run puts its directories."""
+
+from __future__ import annotations
+
+import tempfile
+import unittest
+from pathlib import Path
+from unittest import mock
+
+from baudolo.backup import layout as mod
+from baudolo.backup.shell import BackupError
+
+
+class TestVersionDirectory(unittest.TestCase):
+    def test_it_creates_the_generation_directory(self) -> None:
+        with tempfile.TemporaryDirectory() as tmp:
+            created = mod.create_version_directory(tmp, "20260731020304")
+            self.assertTrue(Path(created).is_dir())
+            self.assertEqual(Path(created).name, "20260731020304")
+
+    def test_it_refuses_a_generation_another_run_already_claimed(self) -> None:
+        with tempfile.TemporaryDirectory() as tmp:
+            mod.create_version_directory(tmp, "20260731")
+            with self.assertRaises(BackupError) as caught:
+                mod.create_version_directory(tmp, "20260731")
+            self.assertIn("20260731", str(caught.exception))
+
+    def test_it_creates_missing_parents(self) -> None:
+        with tempfile.TemporaryDirectory() as tmp:
+            nested = str(Path(tmp) / "machine" / "repo")
+            created = mod.create_version_directory(nested, "20260731")
+            self.assertTrue(Path(created).is_dir())
+
+
+class TestVolumeDirectory(unittest.TestCase):
+    def test_it_nests_the_volume_under_the_generation(self) -> None:
+        with tempfile.TemporaryDirectory() as tmp:
+            created = mod.create_volume_directory(tmp, "postgres_data")
+            self.assertEqual(Path(created).parent, Path(tmp))
+            self.assertTrue(Path(created).is_dir())
+
+
+class TestMachineId(unittest.TestCase):
+    def test_it_takes_the_hash_without_the_filename(self) -> None:
+        digest = "a" * 64
+        with mock.patch.object(
+            mod, "execute_shell_command", return_value=[f"{digest}  /etc/machine-id"]
+        ):
+            self.assertEqual(mod.get_machine_id(), digest)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_policy.py

@@ -0,0 +1,68 @@
+"""Contract of the rules deciding what is backed up and what must stop."""
+
+from __future__ import annotations
+
+import unittest
+from unittest import mock
+
+from baudolo.backup import policy as mod
+
+
+class TestIsImageIgnored(unittest.TestCase):
+    def test_an_empty_whitelist_ignores_nothing(self) -> None:
+        self.assertFalse(mod.is_image_ignored("c1", []))
+
+    def test_a_listed_image_is_ignored(self) -> None:
+        with mock.patch.object(mod, "get_image_info", return_value="alpine:3.20"):
+            self.assertTrue(mod.is_image_ignored("c1", ["alpine:3.20"]))
+
+    def test_matching_is_exact(self) -> None:
+        with mock.patch.object(mod, "get_image_info", return_value="alpine:3.21"):
+            self.assertFalse(mod.is_image_ignored("c1", ["alpine:3.20"]))
+
+
+class TestVolumeIsFullyIgnored(unittest.TestCase):
+    def test_a_volume_without_containers_is_kept(self) -> None:
+        self.assertFalse(mod.volume_is_fully_ignored([], ["alpine:3.20"]))
+
+    def test_it_needs_every_container_to_be_ignored(self) -> None:
+        with mock.patch.object(mod, "get_image_info", side_effect=["a", "b"]):
+            self.assertFalse(mod.volume_is_fully_ignored(["c1", "c2"], ["a"]))
+
+    def test_all_ignored_skips_the_volume(self) -> None:
+        with mock.patch.object(mod, "get_image_info", side_effect=["a", "a"]):
+            self.assertTrue(mod.volume_is_fully_ignored(["c1", "c2"], ["a"]))
+
+
+class TestRequiresStop(unittest.TestCase):
+    def test_no_containers_means_no_stop(self) -> None:
+        self.assertFalse(mod.requires_stop([], []))
+
+    def test_a_swarm_task_never_forces_a_stop(self) -> None:
+        with mock.patch.object(mod, "is_swarm_task", return_value=True):
+            self.assertFalse(mod.requires_stop(["c1"], []))
+
+    def test_a_whitelisted_image_does_not_force_a_stop(self) -> None:
+        with (
+            mock.patch.object(mod, "is_swarm_task", return_value=False),
+            mock.patch.object(mod, "get_image_info", return_value="alpine:3.20"),
+        ):
+            self.assertFalse(mod.requires_stop(["c1"], ["alpine:3.20"]))
+
+    def test_an_unlisted_image_forces_a_stop(self) -> None:
+        with (
+            mock.patch.object(mod, "is_swarm_task", return_value=False),
+            mock.patch.object(mod, "get_image_info", return_value="postgres:17"),
+        ):
+            self.assertTrue(mod.requires_stop(["c1"], ["alpine:3.20"]))
+
+    def test_one_unlisted_container_is_enough(self) -> None:
+        with (
+            mock.patch.object(mod, "is_swarm_task", return_value=False),
+            mock.patch.object(mod, "get_image_info", side_effect=["alpine:3.20", "x"]),
+        ):
+            self.assertTrue(mod.requires_stop(["c1", "c2"], ["alpine:3.20"]))
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_snapshot.py

@@ -0,0 +1,164 @@
+"""Contract of the filesystem snapshot used to capture volumes atomically."""
+
+from __future__ import annotations
+
+import unittest
+
+from baudolo.backup.shell import BackupError
+from baudolo.backup.snapshot import SnapshotError, volume_snapshot
+
+
+class Runner:
+    def __init__(self, replies: dict[str, list[str]] | None = None) -> None:
+        self.calls: list[list[str]] = []
+        self.replies = replies or {}
+
+    def __call__(self, command: list[str]) -> list[str]:
+        self.calls.append(list(command))
+        for prefix, reply in self.replies.items():
+            if " ".join(command).startswith(prefix):
+                return reply
+        return []
+
+
+class TestBtrfs(unittest.TestCase):
+    def test_it_creates_a_read_only_snapshot_inside_the_subject(self) -> None:
+        run = Runner()
+        with volume_snapshot("btrfs", "/var/lib/docker", "20260731", run=run):
+            pass
+        self.assertEqual(
+            run.calls[0],
+            [
+                "btrfs",
+                "subvolume",
+                "snapshot",
+                "-r",
+                "/var/lib/docker",
+                "/var/lib/docker/.baudolo-20260731",
+            ],
+        )
+
+    def test_it_removes_the_snapshot_afterwards(self) -> None:
+        run = Runner()
+        with volume_snapshot("btrfs", "/var/lib/docker", "20260731", run=run):
+            pass
+        self.assertEqual(
+            run.calls[-1],
+            ["btrfs", "subvolume", "delete", "/var/lib/docker/.baudolo-20260731"],
+        )
+
+    def test_it_maps_a_volume_path_into_the_snapshot(self) -> None:
+        run = Runner()
+        with volume_snapshot(
+            "btrfs", "/var/lib/docker", "20260731", run=run
+        ) as resolve:
+            self.assertEqual(
+                resolve("/var/lib/docker/volumes/postgres_data/_data"),
+                "/var/lib/docker/.baudolo-20260731/volumes/postgres_data/_data",
+            )
+
+    def test_it_keeps_the_trailing_slash_rsync_reads_as_contents(self) -> None:
+        run = Runner()
+        with volume_snapshot(
+            "btrfs", "/var/lib/docker", "20260731", run=run
+        ) as resolve:
+            self.assertEqual(
+                resolve("/var/lib/docker/volumes/postgres_data/_data/"),
+                "/var/lib/docker/.baudolo-20260731/volumes/postgres_data/_data/",
+            )
+
+    def test_it_removes_the_snapshot_even_when_the_body_raises(self) -> None:
+        run = Runner()
+        with (
+            self.assertRaises(ZeroDivisionError),
+            volume_snapshot("btrfs", "/var/lib/docker", "20260731", run=run),
+        ):
+            raise ZeroDivisionError
+        self.assertEqual(run.calls[-1][:3], ["btrfs", "subvolume", "delete"])
+
+
+class TestZfs(unittest.TestCase):
+    def _run(self) -> Runner:
+        return Runner({"zfs list": ["tank/docker"]})
+
+    def test_it_snapshots_the_dataset_mounted_at_the_subject(self) -> None:
+        run = self._run()
+        with volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run):
+            pass
+        self.assertIn(["zfs", "snapshot", "tank/docker@baudolo-20260731"], run.calls)
+
+    def test_it_destroys_the_snapshot_afterwards(self) -> None:
+        run = self._run()
+        with volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run):
+            pass
+        self.assertEqual(
+            run.calls[-1], ["zfs", "destroy", "tank/docker@baudolo-20260731"]
+        )
+
+    def test_it_maps_a_volume_path_through_the_dot_zfs_directory(self) -> None:
+        run = self._run()
+        with volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run) as resolve:
+            self.assertEqual(
+                resolve("/var/lib/docker/volumes/postgres_data/_data"),
+                "/var/lib/docker/.zfs/snapshot/baudolo-20260731/volumes/postgres_data/_data",
+            )
+
+    def test_an_unmounted_dataset_is_an_error(self) -> None:
+        run = Runner({"zfs list": [""]})
+        with (
+            self.assertRaises(SnapshotError),
+            volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run),
+        ):
+            pass
+
+
+class TestRejections(unittest.TestCase):
+    def test_an_unknown_kind_is_rejected(self) -> None:
+        run = Runner()
+        with (
+            self.assertRaises(SnapshotError),
+            volume_snapshot("ext4", "/var/lib/docker", "20260731", run=run),
+        ):
+            pass
+        self.assertEqual(run.calls, [])
+
+    def test_a_path_outside_the_subject_is_rejected(self) -> None:
+        run = Runner()
+        with (
+            volume_snapshot("btrfs", "/var/lib/docker", "20260731", run=run) as resolve,
+            self.assertRaises(SnapshotError),
+        ):
+            resolve("/etc/passwd")
+
+    def test_the_subject_itself_resolves_to_the_snapshot_root(self) -> None:
+        run = Runner()
+        with volume_snapshot(
+            "btrfs", "/var/lib/docker", "20260731", run=run
+        ) as resolve:
+            self.assertEqual(
+                resolve("/var/lib/docker"), "/var/lib/docker/.baudolo-20260731"
+            )
+
+
+class Busy(Runner):
+    def __call__(self, command: list[str]) -> list[str]:
+        if command[:3] == ["btrfs", "subvolume", "delete"]:
+            raise BackupError("target is busy")
+        return super().__call__(command)
+
+
+class TestRemovalFailure(unittest.TestCase):
+    def test_a_failed_removal_does_not_fail_a_completed_run(self) -> None:
+        with volume_snapshot("btrfs", "/var/lib/docker", "20260731", run=Busy()):
+            pass
+
+    def test_a_failed_removal_does_not_mask_the_body(self) -> None:
+        with (
+            self.assertRaises(ZeroDivisionError),
+            volume_snapshot("btrfs", "/var/lib/docker", "20260731", run=Busy()),
+        ):
+            raise ZeroDivisionError
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_snapshot_faithfulness.py

@@ -0,0 +1,127 @@
+"""Which volumes a snapshot of the subject actually contains.
+
+The failure this guards against is silent: a volume with a backing store of
+its own is present inside the snapshot as an empty directory, so rsync
+succeeds, the generation is stamped complete, and the volume is empty in it.
+"""
+
+from __future__ import annotations
+
+import os
+import tempfile
+import unittest
+from pathlib import Path
+from unittest import mock
+
+from baudolo.backup.snapshot import SnapshotError, snapshot_source, unsnapshotted
+from baudolo.backup.volume import Backing
+
+
+class TestUnsnapshotted(unittest.TestCase):
+    def setUp(self) -> None:
+        self.subject = tempfile.mkdtemp()
+        self.mountpoint = str(Path(self.subject) / "volumes" / "app" / "_data")
+        Path(self.mountpoint).mkdir(parents=True)
+
+    def backing(self, **kwargs) -> Backing:
+        return Backing(kwargs.pop("mountpoint", self.mountpoint), **kwargs)
+
+    def test_a_plain_local_volume_is_captured(self) -> None:
+        self.assertIsNone(unsnapshotted(self.backing(), self.subject))
+
+    def test_a_foreign_driver_is_not(self) -> None:
+        reason = unsnapshotted(self.backing(driver="rexray"), self.subject)
+        self.assertIn("rexray", reason)
+
+    def test_declared_driver_options_are_not(self) -> None:
+        reason = unsnapshotted(
+            self.backing(options={"type": "nfs", "device": ":/exports/app"}),
+            self.subject,
+        )
+        self.assertIn("backing store", reason)
+
+    def test_the_declaration_decides_not_the_mount_table(self) -> None:
+        """Docker unmounts an NFS volume when its last container stops."""
+        with mock.patch.object(os.path, "ismount", return_value=False):
+            reason = unsnapshotted(self.backing(options={"type": "nfs"}), self.subject)
+        self.assertIsNotNone(reason)
+
+    def test_a_volume_without_a_mountpoint_is_not(self) -> None:
+        reason = unsnapshotted(Backing(""), self.subject)
+        self.assertIn("no mountpoint", reason)
+
+    def test_an_own_mount_is_not(self) -> None:
+        with mock.patch.object(os.path, "ismount", return_value=True):
+            reason = unsnapshotted(self.backing(), self.subject)
+        self.assertIn("own mount", reason)
+
+    def test_a_filesystem_boundary_is_not(self) -> None:
+        real = os.stat
+
+        def crossing(path, *args, **kwargs):
+            info = real(path, *args, **kwargs)
+            if os.path.realpath(path) == os.path.realpath(self.mountpoint):
+                return os.stat_result(
+                    (info.st_mode, info.st_ino, info.st_dev + 1, *tuple(info)[3:])
+                )
+            return info
+
+        with mock.patch.object(os, "stat", side_effect=crossing):
+            reason = unsnapshotted(self.backing(), self.subject)
+        self.assertIn("filesystem boundary", reason)
+
+    def test_an_unreadable_mountpoint_is_not(self) -> None:
+        reason = unsnapshotted(
+            self.backing(mountpoint=str(Path(self.subject) / "gone")), self.subject
+        )
+        self.assertIn("could not be read", reason)
+
+
+class TestSnapshotSource(unittest.TestCase):
+    def setUp(self) -> None:
+        self.subject = tempfile.mkdtemp()
+        self.mountpoint = str(Path(self.subject) / "volumes" / "app" / "_data")
+        Path(self.mountpoint).mkdir(parents=True)
+        self.snapshot = str(
+            Path(self.subject) / ".baudolo-tag" / "volumes" / "app" / "_data"
+        )
+        Path(self.snapshot).mkdir(parents=True)
+        self.backing = Backing(self.mountpoint)
+
+    def test_a_captured_volume_reads_from_the_snapshot(self) -> None:
+        source, reason = snapshot_source(
+            lambda path: self.snapshot + "/", self.backing, self.subject
+        )
+        self.assertEqual(source, self.snapshot + "/")
+        self.assertEqual(reason, "")
+
+    def test_an_uncaptured_volume_is_refused_before_the_resolver_runs(self) -> None:
+        def resolve(path):
+            raise AssertionError("must not resolve a volume the snapshot misses")
+
+        source, reason = snapshot_source(
+            resolve, Backing(self.mountpoint, options={"type": "nfs"}), self.subject
+        )
+        self.assertIsNone(source)
+        self.assertIn("backing store", reason)
+
+    def test_a_volume_outside_the_subject_degrades_instead_of_raising(self) -> None:
+        def resolve(path):
+            raise SnapshotError(f"{path} lies outside the snapshot subject")
+
+        source, reason = snapshot_source(resolve, self.backing, self.subject)
+        self.assertIsNone(source)
+        self.assertIn("lies outside", reason)
+
+    def test_a_volume_created_after_the_snapshot_degrades(self) -> None:
+        source, reason = snapshot_source(
+            lambda path: str(Path(self.subject) / "absent") + "/",
+            self.backing,
+            self.subject,
+        )
+        self.assertIsNone(source)
+        self.assertIn("created after", reason)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_volume.py

@@ -0,0 +1,95 @@
+"""Contract of the rsync invocation that copies a volume."""
+
+from __future__ import annotations
+
+import tempfile
+import unittest
+from pathlib import Path
+from unittest import mock
+
+from baudolo.backup import volume as mod
+
+
+class TestBackupVolume(unittest.TestCase):
+    def copy(self, **kwargs) -> str:
+        with tempfile.TemporaryDirectory() as tmp:
+            defaults = {
+                "versions_dir": tmp,
+                "volume_name": "demo",
+                "volume_dir": str(Path(tmp) / "gen" / "demo"),
+                "authoritative": False,
+                "source": "/var/lib/docker/volumes/demo/_data/",
+            }
+            defaults.update(kwargs)
+            with mock.patch.object(mod, "execute_shell_command") as run:
+                mod.backup_volume(
+                    defaults.pop("versions_dir"),
+                    defaults.pop("volume_name"),
+                    defaults.pop("volume_dir"),
+                    **defaults,
+                )
+            return run.call_args[0][0]
+
+    def test_the_quick_check_pass_carries_no_checksum(self) -> None:
+        self.assertNotIn("--checksum", self.copy(authoritative=False))
+
+    def test_the_authoritative_pass_compares_by_content(self) -> None:
+        self.assertIn("--checksum", self.copy(authoritative=True))
+
+    def test_it_reads_from_the_given_source(self) -> None:
+        command = self.copy(source="/snapshot/volumes/demo/_data/")
+        self.assertIn("/snapshot/volumes/demo/_data/", command)
+
+    def test_it_always_deletes_what_the_source_no_longer_has(self) -> None:
+        self.assertIn("--delete", self.copy())
+
+    def test_it_carries_no_kernel_objects_into_a_generation(self) -> None:
+        self.assertIn("--no-D", self.copy())
+
+    def test_it_keeps_no_twin_of_what_the_second_pass_replaces(self) -> None:
+        command = self.copy(authoritative=True)
+        self.assertEqual(command[:2], ["rsync", "-aP"])
+        self.assertNotIn("--backup", command)
+
+    def test_it_creates_the_destination(self) -> None:
+        with tempfile.TemporaryDirectory() as tmp:
+            dest = Path(tmp) / "gen" / "demo"
+            with mock.patch.object(mod, "execute_shell_command"):
+                mod.backup_volume(
+                    tmp, "demo", str(dest), authoritative=False, source="/src/"
+                )
+            self.assertTrue((dest / "files").is_dir())
+
+    def test_source_is_required(self) -> None:
+        with self.assertRaises(TypeError):
+            mod.backup_volume("/v", "demo", "/d", authoritative=False)
+
+    def test_authoritative_is_required(self) -> None:
+        with self.assertRaises(TypeError):
+            mod.backup_volume("/v", "demo", "/d", source="/src/")
+
+
+class TestLastBackupDir(unittest.TestCase):
+    def test_it_ignores_the_generation_being_written(self) -> None:
+        with tempfile.TemporaryDirectory() as tmp:
+            current = Path(tmp) / "20260101" / "demo" / "files"
+            current.mkdir(parents=True)
+            found = mod.get_last_backup_dir(tmp, "demo", str(current) + "/")
+            self.assertIsNone(found)
+
+    def test_it_finds_the_previous_generation(self) -> None:
+        with tempfile.TemporaryDirectory() as tmp:
+            older = Path(tmp) / "20260101" / "demo" / "files"
+            older.mkdir(parents=True)
+            current = Path(tmp) / "20260102" / "demo" / "files"
+            current.mkdir(parents=True)
+            found = mod.get_last_backup_dir(tmp, "demo", str(current) + "/")
+            self.assertEqual(found, str(older) + "/")
+
+    def test_a_first_run_has_no_predecessor(self) -> None:
+        with tempfile.TemporaryDirectory() as tmp:
+            self.assertIsNone(mod.get_last_backup_dir(tmp, "demo", f"{tmp}/x/"))
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/restore/test_cli_version_flag.py

@@ -0,0 +1,54 @@
+import unittest
+from unittest.mock import patch
+
+from baudolo.restore import __main__ as cli
+
+ENGINES = {
+    "postgres": ("restore_postgres_sql", ["--db-name", "app"]),
+    "mariadb": ("restore_mariadb_sql", ["--db-name", "app"]),
+    "cluster": ("restore_cluster_sql", ["--instance", "central", "--db-user", "root"]),
+}
+
+
+class TestVersionFlagReachesEveryEngine(unittest.TestCase):
+    def call(self, engine: str, extra: list) -> dict:
+        target, required = ENGINES[engine]
+        argv = [
+            engine,
+            "app_vol",
+            "hash",
+            "20260817000000",
+            "--repo-name",
+            "repo",
+            "--container",
+            "db",
+            "--db-password",
+            "pw",
+            *required,
+            *extra,
+        ]
+        with patch.object(cli, target) as restore:
+            self.assertEqual(cli.main(argv), 0)
+        return restore.call_args.kwargs
+
+    def test_the_gate_is_on_by_default(self) -> None:
+        for engine in ENGINES:
+            with self.subTest(engine=engine):
+                self.assertTrue(self.call(engine, [])["check_version"])
+
+    def test_the_flag_turns_it_off(self) -> None:
+        for engine in ENGINES:
+            with self.subTest(engine=engine):
+                kwargs = self.call(engine, ["--no-version-check"])
+                self.assertFalse(kwargs["check_version"])
+
+    def test_empty_stays_independent_of_the_gate(self) -> None:
+        for engine in ENGINES:
+            with self.subTest(engine=engine):
+                kwargs = self.call(engine, ["--empty"])
+                self.assertTrue(kwargs["empty"])
+                self.assertTrue(kwargs["check_version"])
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/restore/test_cluster_preclean_scope.py

@@ -0,0 +1,126 @@
+"""The pre-clean may only drop what the dump can bring back.
+
+A shared instance carries databases and roles from other applications, and a
+database created after the backup is in no dump at all. Dropping those would
+destroy data this restore cannot restore.
+"""
+
+import tempfile
+import unittest
+from pathlib import Path
+
+from baudolo.restore.db import cluster as cluster_mod
+
+DUMP = """--
+-- PostgreSQL database cluster dump
+--
+
+CREATE ROLE app;
+ALTER ROLE app WITH LOGIN;
+CREATE ROLE reporting;
+
+CREATE DATABASE appdb OWNER app;
+
+\\connect appdb
+
+CREATE TABLE t (v text);
+
+\\connect template1
+"""
+
+
+def dump_file(text: str) -> str:
+    path = Path(tempfile.mkdtemp()) / "central.cluster.backup.sql"
+    path.write_text(text, encoding="utf-8")
+    return str(path)
+
+
+class TestDumpInventory(unittest.TestCase):
+    def test_it_reads_databases_and_roles_the_dump_recreates(self) -> None:
+        databases, roles = cluster_mod.dump_inventory(dump_file(DUMP))
+        self.assertEqual(databases, ["appdb", "template1"])
+        self.assertEqual(roles, ["app", "reporting"])
+
+    def test_a_quoted_name_keeps_its_spaces(self) -> None:
+        databases, roles = cluster_mod.dump_inventory(
+            dump_file('\\connect "odd name"\nCREATE ROLE "odd role";\n')
+        )
+        self.assertEqual(databases, ["odd name"])
+        self.assertEqual(roles, ["odd role"])
+
+    def test_psql_options_are_not_mistaken_for_the_database(self) -> None:
+        databases, _roles = cluster_mod.dump_inventory(
+            dump_file("\\connect -reuse-previous=on dbname=appdb\n")
+        )
+        self.assertEqual(databases, ["appdb"])
+
+    def test_create_database_options_are_not_part_of_the_name(self) -> None:
+        databases, _roles = cluster_mod.dump_inventory(
+            dump_file("CREATE DATABASE appdb WITH TEMPLATE = template0 OWNER = app;\n")
+        )
+        self.assertEqual(databases, ["appdb"])
+
+    def test_a_name_is_listed_once(self) -> None:
+        databases, _roles = cluster_mod.dump_inventory(
+            dump_file("\\connect a\n\\connect a\n")
+        )
+        self.assertEqual(databases, ["a"])
+
+
+class TestInstanceRefusal(unittest.TestCase):
+    """--empty wipes the whole instance, so it may only run on one this dump
+    can rebuild. Scoping the sweep instead wedges the restore: a surviving
+    database that grants to a dumped role pins it, DROP ROLE fails, and the
+    pre-clean aborts after the dump's own databases are already gone."""
+
+    def check(self, present: str, dump: str = DUMP):
+        from unittest import mock
+
+        with mock.patch.object(
+            cluster_mod, "instance_databases", return_value=present.split()
+        ):
+            cluster_mod.assert_instance_matches_dump(
+                "db", "postgres", dump_file(dump), {}
+            )
+
+    def test_an_instance_the_dump_covers_passes(self) -> None:
+        self.check("appdb")
+
+    def test_an_empty_instance_passes(self) -> None:
+        self.check("")
+
+    def test_a_database_the_dump_lacks_is_refused(self) -> None:
+        with self.assertRaises(RuntimeError) as raised:
+            self.check("appdb sibling")
+        self.assertIn("sibling", str(raised.exception))
+
+    def test_the_refusal_names_every_foreign_database(self) -> None:
+        with self.assertRaises(RuntimeError) as raised:
+            self.check("one two")
+        message = str(raised.exception)
+        self.assertIn("one", message)
+        self.assertIn("two", message)
+
+    def test_the_refusal_happens_before_anything_is_dropped(self) -> None:
+        from unittest import mock
+
+        with (
+            mock.patch.object(
+                cluster_mod, "instance_databases", return_value=["foreign"]
+            ),
+            mock.patch.object(cluster_mod, "docker_exec") as touched,
+            self.assertRaises(RuntimeError),
+        ):
+            cluster_mod.restore_cluster_sql(
+                container="db",
+                user="postgres",
+                password="pw",
+                sql_path=dump_file(DUMP),
+                empty=True,
+                check_version=False,
+            )
+        touched.assert_not_called()
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/restore/test_cluster_replay.py

@@ -0,0 +1,125 @@
+import tempfile
+import unittest
+from unittest.mock import MagicMock, patch
+
+from baudolo.restore.db import cluster as cluster_mod
+from baudolo.restore.paths import BackupPaths
+
+
+class TestClusterReplay(unittest.TestCase):
+    def _replay(self, *, empty: bool):
+        calls = []
+
+        def _capture(container, argv, **kwargs):
+            if "-tAc" in argv:
+                return MagicMock(stdout=b"")
+            calls.append((argv, kwargs.get("stdin")))
+            return MagicMock()
+
+        with tempfile.NamedTemporaryFile(suffix=".sql") as sql:
+            sql.write(b"CREATE ROLE app;\nCREATE DATABASE app OWNER app;\n")
+            sql.flush()
+            with patch.object(cluster_mod, "docker_exec", side_effect=_capture):
+                cluster_mod.restore_cluster_sql(
+                    container="db",
+                    user="postgres",
+                    password="pw",
+                    sql_path=sql.name,
+                    empty=empty,
+                    check_version=False,
+                )
+        return calls
+
+    def test_the_replay_is_not_wrapped_in_a_transaction(self) -> None:
+        argv, _ = self._replay(empty=False)[0]
+        self.assertNotIn(
+            "--single-transaction",
+            argv,
+            "CREATE DATABASE cannot run inside a transaction block, so unlike the "
+            "single-database replay this stream must not be wrapped in one",
+        )
+        self.assertIn("ON_ERROR_STOP=1", argv)
+
+    def test_the_replay_targets_the_control_database(self) -> None:
+        argv, _ = self._replay(empty=False)[0]
+        self.assertEqual(argv[argv.index("-d") + 1], cluster_mod.CONTROL_DB)
+        self.assertEqual(argv[argv.index("-U") + 1], "postgres")
+
+    def test_without_empty_nothing_is_dropped_first(self) -> None:
+        self.assertEqual(len(self._replay(empty=False)), 1)
+
+    def test_empty_drops_databases_before_their_owners(self) -> None:
+        calls = self._replay(empty=True)
+        self.assertEqual(len(calls), 2, f"expected pre-clean + replay: {calls}")
+        preclean = calls[0][1].decode()
+        self.assertLess(
+            preclean.index("DROP DATABASE"),
+            preclean.index("DROP ROLE"),
+            "a role cannot be dropped while it still owns a database",
+        )
+        self.assertIn("DROP OWNED BY", preclean)
+        self.assertIn("ORDER BY phase", preclean)
+
+    def test_the_preclean_spares_what_no_dump_recreates(self) -> None:
+        preclean = self._replay(empty=True)[0][1].decode()
+        self.assertIn("NOT datistemplate", preclean)
+        self.assertIn("datname <> current_database()", preclean)
+        self.assertIn("starts_with(rolname, 'pg_')", preclean)
+        self.assertIn("rolname <> current_user", preclean)
+
+    def test_only_the_connecting_role_loses_its_create(self) -> None:
+        # Captured from pg_dumpall 17.
+        dump = [
+            b"CREATE ROLE app;\n",
+            b"ALTER ROLE app WITH NOSUPERUSER INHERIT LOGIN PASSWORD 'SCRAM-SHA-256$...';\n",
+            b"CREATE ROLE postgres;\n",
+            b"ALTER ROLE postgres WITH SUPERUSER INHERIT LOGIN PASSWORD 'SCRAM-SHA-256$...';\n",
+            b'CREATE ROLE "odd-name";\n',
+        ]
+        kept = list(cluster_mod.filter_own_role_creation(dump, "postgres"))
+        self.assertNotIn(b"CREATE ROLE postgres;\n", kept)
+        self.assertIn(b"CREATE ROLE app;\n", kept)
+        self.assertIn(b'CREATE ROLE "odd-name";\n', kept)
+        self.assertEqual(
+            sum(1 for line in kept if line.startswith(b"ALTER ROLE postgres")),
+            1,
+            "the ALTER re-applies the superuser's attributes and password",
+        )
+
+    def test_a_quoted_connecting_role_is_matched_too(self) -> None:
+        kept = list(
+            cluster_mod.filter_own_role_creation(
+                [b'CREATE ROLE "odd-name";\n'], "odd-name"
+            )
+        )
+        self.assertEqual(kept, [])
+
+    def test_a_role_whose_name_merely_starts_the_same_is_kept(self) -> None:
+        kept = list(
+            cluster_mod.filter_own_role_creation(
+                [b"CREATE ROLE postgresql;\n"], "postgres"
+            )
+        )
+        self.assertEqual(kept, [b"CREATE ROLE postgresql;\n"])
+
+    def test_a_missing_dump_is_reported_as_such(self) -> None:
+        with self.assertRaises(FileNotFoundError):
+            cluster_mod.restore_cluster_sql(
+                container="db",
+                user="postgres",
+                password="pw",
+                sql_path="/nonexistent/x.cluster.backup.sql",
+                empty=False,
+                check_version=False,
+            )
+
+    def test_the_path_helper_names_the_dumpall_file(self) -> None:
+        paths = BackupPaths("vol", "hash", "v1", repo_name="repo", backups_dir="/B")
+        self.assertEqual(
+            paths.cluster_file("bigbluebutton"),
+            "/B/hash/repo/v1/vol/sql/bigbluebutton.cluster.backup.sql",
+        )
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/restore/test_files_backing_store.py

@@ -0,0 +1,62 @@
+import tempfile
+import unittest
+from unittest.mock import MagicMock, patch
+
+from baudolo.restore import files as files_mod
+
+
+class TestBackingStoreGuard(unittest.TestCase):
+    def restore(self, inspect: str, mounted: bool) -> tuple[int, list]:
+        calls = []
+
+        def _run(cmd, **kwargs):
+            calls.append(cmd)
+            return MagicMock(stdout=inspect.encode())
+
+        with (
+            patch.object(files_mod, "docker_volume_exists", return_value=True),
+            patch.object(files_mod.os.path, "ismount", return_value=mounted),
+            patch.object(files_mod, "run", side_effect=_run),
+        ):
+            code = files_mod.restore_volume_files("app_data", tempfile.mkdtemp())
+        return code, calls
+
+    def rsynced(self, calls: list) -> bool:
+        return any(cmd[0] == "rsync" for cmd in calls)
+
+    def test_plain_local_volume_is_restored_unmounted(self) -> None:
+        code, calls = self.restore("/var/lib/docker/volumes/a/_data|local|plain", False)
+        self.assertEqual(code, 0)
+        self.assertTrue(self.rsynced(calls))
+
+    def test_volume_with_driver_options_is_refused_while_unmounted(self) -> None:
+        code, calls = self.restore("/var/lib/docker/volumes/a/_data|local|opts", False)
+        self.assertEqual(code, 2)
+        self.assertFalse(
+            self.rsynced(calls),
+            "an NFS or bind volume writes under the mount and reports success",
+        )
+
+    def test_volume_with_driver_options_is_restored_once_mounted(self) -> None:
+        code, calls = self.restore("/var/lib/docker/volumes/a/_data|local|opts", True)
+        self.assertEqual(code, 0)
+        self.assertTrue(self.rsynced(calls))
+
+    def test_foreign_driver_is_refused_while_unmounted(self) -> None:
+        code, calls = self.restore("/mnt/gluster/a|glusterfs|plain", False)
+        self.assertEqual(code, 2)
+        self.assertFalse(self.rsynced(calls))
+
+    def test_an_unresolvable_mountpoint_still_fails_first(self) -> None:
+        code, calls = self.restore("|local|plain", False)
+        self.assertEqual(code, 2)
+        self.assertFalse(self.rsynced(calls))
+
+    def test_a_format_without_the_new_fields_is_treated_as_plain(self) -> None:
+        code, calls = self.restore("/var/lib/docker/volumes/a/_data", False)
+        self.assertEqual(code, 0)
+        self.assertTrue(self.rsynced(calls))
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/restore/test_mariadb_empty_drop.py

@@ -0,0 +1,45 @@
+import tempfile
+import unittest
+from unittest.mock import MagicMock, patch
+
+from baudolo.restore.db import mariadb as mariadb_mod
+
+
+class TestMariadbEmptyDrop(unittest.TestCase):
+    def test_drops_run_in_one_session_with_fk_checks_off(self) -> None:
+        calls = []
+
+        def _capture(container, argv, **kwargs):
+            calls.append(argv)
+            result = MagicMock()
+            result.stdout = b"users\nfetches\n"
+            return result
+
+        with tempfile.NamedTemporaryFile(suffix=".sql") as sql:
+            sql.write(b"CREATE TABLE t (id int);\n")
+            sql.flush()
+            with (
+                patch.object(mariadb_mod, "docker_exec", side_effect=_capture),
+                patch.object(mariadb_mod, "_pick_client", return_value="mariadb"),
+            ):
+                mariadb_mod.restore_mariadb_sql(
+                    container="db",
+                    db_name="mailu",
+                    user="mailu",
+                    password="pw",
+                    sql_path=sql.name,
+                    empty=True,
+                    check_version=False,
+                )
+
+        drop_calls = [argv for argv in calls if any("DROP TABLE" in a for a in argv)]
+        self.assertEqual(len(drop_calls), 1, f"expected ONE drop session: {calls}")
+        drop_sql = drop_calls[0][-1]
+        self.assertTrue(drop_sql.startswith("SET FOREIGN_KEY_CHECKS=0; "))
+        self.assertIn("DROP TABLE IF EXISTS `mailu`.`users`;", drop_sql)
+        self.assertIn("DROP TABLE IF EXISTS `mailu`.`fetches`;", drop_sql)
+        self.assertTrue(drop_sql.endswith("SET FOREIGN_KEY_CHECKS=1;"))
+
+
+if __name__ == "__main__":
+    unittest.main()