The --empty pre-clean is a catalog-wide sweep: it drops every non-template database and every non-pg_ role of the instance. On a dedicated instance that is exactly right, because the dump recreates all of it. On a shared one it destroys databases the dump does not carry, with nothing to restore them from - and no test ever executed that sweep, because the e2e dropped the cluster by hand first and left the pre-clean with zero rows to generate.
Scoping the sweep to the dump's own inventory looks like the fix and is worse. A surviving database that owns or merely grants to one of the dump's roles pins that role in pg_shdepend; DROP OWNED BY only reaches the control database the pre-clean is connected to, so DROP ROLE fails - after phase 1 has already dropped the dump's databases. ON_ERROR_STOP aborts, the replay never starts, and the instance is left half emptied.
So the instance is checked instead. --empty now refuses when the instance holds a database the dump does not carry, names it, and touches nothing. The sweep stays as it was, safe behind that refusal. Reading the dump's inventory needs a real identifier parser: a quoted name may hold spaces, and psql options precede the target of a \\connect line.
The e2e no longer drops the cluster itself, so --empty has to do it and the replay has to put it back; a second pass then adds a foreign database and requires the refusal to leave both it and the restored data alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ruff was never wired into this repository: no target, no CI step, no pin.
It reported 45 findings across sources and tests, so nothing enforced
what the codebase already mostly followed.
Adds `make ruff` (check + format --check), `make ruff-fix`, and `make
lint` as its alias, and makes `make test` run lint as a fourth parallel
spur. The CI workflow calls `make test`, so it is covered there too. The
linter is pinned in a `lint` extra: a ruff minor bump changes which rules
fire, and with the suite gating on a clean run an unpinned linter would
fail it on an unrelated day.
The 45 findings are fixed rather than configured away. Three needed a
decision instead of the mechanical fix:
- The generation timestamp keeps its local wall clock (DTZ005 waived).
Generations sort by that name, and UTC would order new ones before the
existing ones wherever the offset is positive - "newest generation" is
what every restore path selects on.
- The per-volume `copy` closure now binds volume_name and vol_dir as
default arguments (B023). It only worked because it is called inside
the same iteration.
- The two CLI top-level handlers keep their blind except (BLE001
waived): turning any failure into exit 1 is what a CLI boundary is
for. The two in run.py did not need it and were narrowed to what they
actually catch.
Also drops the comments that restate the code: the section banners in
restore/__main__.py, the filename repeated as line 1 of nine test files,
step narration above the statement it narrates, and a block in app.py
documenting parameters that had moved to another module. What names a
trip-wire stays - the snapshot destination rule, the mysql-binary
absence in MariaDB 11 images, the session-scoped FOREIGN_KEY_CHECKS, the
spooled temp file for multi-GB dumps, and the negative control that
loses its discriminating power if it ever passes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A databases.csv row asking for every database of an instance
(database = '*') makes the backup side write <instance>.cluster.backup.sql
via pg_dumpall, and nothing could read it back: the restore CLI knew
files, postgres and mariadb. That dump was stored and unrestorable - a
format whose producer had no consumer.
Adds `baudolo-restore cluster`. Three properties of a cluster stream
shape it, and each one bit during development:
- It recreates databases, and CREATE DATABASE cannot run inside a
transaction block. So unlike the single-database replay this one must
NOT be wrapped in --single-transaction. The unit tests now pin both
contracts against each other.
- It recreates every role including the one the replay connects as, and
the pre-clean cannot drop the role holding its own session. That
single CREATE ROLE is filtered out of the stream while its ALTER ROLE
is kept, because that is what carries the attributes and the password.
Found by running it: the first replay died on `role "postgres"
already exists`.
- --empty means more than for one database: the cluster's databases go
first, then DROP OWNED BY releases what a role still holds in the
control database, then the roles themselves. The order is pinned by a
phase column because \gexec would otherwise emit them interleaved, and
a role cannot be dropped while it still owns a database.
Without --empty the replay stops at the first object that already
exists. Recreating a cluster over a populated one is a decision, not a
default.
The e2e test drills the real thing: two databases and their owning role
are dropped outright and have to come back with their payload and their
ownership intact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
