Two defects kept dedicated Postgres databases out of the backup.
docker exec never forwarded PGPASSWORD. execute_to_file set it on baudolo's
own process, but nothing carried it across the container boundary, so an
engine whose pg_hba demands a password on TCP loopback refused the dump.
forward_env passes a bare `-e NAME`, letting docker copy the value out of
this process's environment instead of spelling it into argv, where the
host's process list would publish it.
get_instance returned the container name unchanged when that name carried no
database token, claiming an instance it had never derived. An application
container therefore answered the same databases.csv row as its own dedicated
engine, and application images often ship the engine's client tools, so the
dump command started and wrote a file that looked like a backup and held
none of the data. Discourse is the live case: its launcher names the
container `discourse`, and the image ships pg_dumpall.
The regex stays a normaliser - `<app>-database` from compose and
`<app>_database.1.<task>` from swarm still resolve to the same instance.
Only the fallthrough changes.
BREAKING CHANGE: a database container whose name carries no `database`, `db`
or `postgres` token must now be named in --database-containers. Without that
declaration its rows no longer match and no dump is written.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The package carried no ruff configuration at all, so it ran on the defaults (E4+E7+E9+F) while infinito-nexus-core, its only consumer, holds itself to a far wider selection. Measured against that selection this tree had 224 findings. It now has none.
The selector list is core's verbatim so both repositories answer to one bar. target-version stays py39 rather than core's py311, because requires-python still declares >=3.9 and pyupgrade would otherwise propose syntax the declared minimum cannot run. Every ignore carries its reason: S603/S607 in particular, since running docker and dump binaries from PATH in list form is this tool's whole job and is already injection-safe.
Two conversions are judgement rather than mechanics. os.path.join(dir, '') was the rsync idiom for a trailing separator, which Path drops, so it becomes an explicit os.sep. os.path.abspath stays where Path.resolve() would follow symlinks and let a symlinked volume test as inside the snapshot subject.
BREAKING CHANGE: VersionMismatch is renamed VersionMismatchError.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A finished generation cannot show whether a volume held a database, nor whether a dump was produced for it: under --only-sql a failed dump falls back to a file copy, and the resulting files/ tree looks like any other copy. The run knows both and threw the knowledge away as a printed warning, leaving every reader to guess from file names.
Each generation now carries a manifest.json stating its layout and, per volume, database / dumped / engine. baudolo.generation is the single place those names are spelled; restore/paths.py, backup/db.py and backup/volume.py stop repeating them. It is deliberately import-free so a consumer can read the manifest with nothing but json, on hosts where this package is not installed.
BREAKING CHANGE: BackupException is renamed BackupError. The rename is atomic across the ten modules that define or import it, three of which also carry the manifest change, so it lands in this commit rather than a separate one that could not import.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every test target required 'clean', which is 'git clean -fdX .' - so running the unit tests deleted every git-ignored file the operator had, venv and caches included. It was compensating for a missing .dockerignore: the Dockerfile's COPY . . otherwise drags __pycache__, egg-info and build output into the image context.
The ignore file fixes that where it belongs, so the prerequisite can go. 'clean' remains available as its own target.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The backup built command strings and handed them to shell=True, so four interpolated values per dump - user, password, container, database - were each a way out of the command. validate_database covered one of them since the previous commit; now there is nothing to cover: every command is an argv list, and a value can only ever be an argument.
execute_to_file absorbs the atomic dump write. The shell redirect into <file>.tmp and the separate mv process become a Python file handle and os.replace, and a failing dump deletes its partial file instead of leaving it. PGPASSWORD moves out of the command string into the child's environment, where a process listing does not show it.
docker exec is built in one place, docker_exec_argv; db.py's three hand-built copies and the probe use it. The dead docker_volume_exists goes - never called, and the restore side owns the living twin. The rsync quoting in --link-dest falls away: inside an argv it would have become part of the path.
The snapshot module's injected runner changes type with it, which the three e2e drivers implement - the first conversion missed them, btrfs ran with no arguments, and the e2e caught it. Marked breaking for that contract: any external runner injected into volume_snapshot must now accept a list.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The schema lived three times in this repo alone: the seed and the backup each spelled out the column list and the semicolon, and _validate_database_value existed twice under one name with different strictness - the seed checked the character set, the backup only checked for empty. A hand-edited file therefore bypassed the only real validation on its way into a dump command.
baudolo.databases now holds the columns, the delimiter, the cluster marker, the validator and a read_rows() for consumers that do not want pandas. Values come back verbatim: a password may begin or end with a space, so stripping belongs to the caller that compares, never to the reader. DatabasesCsvError subclasses ValueError, so callers that predate the module keep catching what they caught.
The backup's call sites switch over in the next commit, which rewrites them anyway.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two defaults could not be right. --repo-name fell back to the literal 'backup-docker-to-local' while its help promised the git repo folder name, which nothing ever derived. --databases-csv pointed inside the installed package directory, where credentials must not live; when it applied, load_databases_df read a missing file as empty and the run finished without a single dump and without an error. Both are required now, --repo-name in the restore CLI too. The file itself may still be absent - babadcb's tolerance is untouched, only the path must be named.
--everything is withdrawn. Its one effect was to ignore --images-no-stop-required, which is what leaving that list empty already does, and its branch was the default path minus the requires_stop check. No caller, no test, and help and README described it differently.
--dump-only-sql becomes --only-sql, and --only-files joins it as the opposite half: no dumps at all, every volume as files. They form a mutually exclusive group. A host that only copies files has no business holding database passwords, so --databases-csv is not required there and is never read.
The smallest valid argv turned out to be written four times across the test tree; it now lives once. Withdrawn flags are listed in one place and proven to exit 2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
36b2336 matched postgres and mariadb against the image's repository path. That name is not a property of the software: a dedicated Postgres inside an app's own stack is tagged <app>-database or postgis/postgis and carries no engine token at all, so it was never recognised, never dumped, and its data directory was copied as files without a single warning - the fallback notice hangs on found_db, which stayed false.
The container is now asked what it can run. pg_dumpall, mariadb-dump and mysqldump are probed by executing them, not by asking a shell for them, because a distroless image has no shell and would deny every tool it ships. The verdict is cached per image ID rather than per container, so replicas of one image cost a single probe.
Because the probe names the tool it found, the dump uses it instead of the hardcoded /usr/bin/mariadb-dump, which makes an image that ships only mysqldump dumpable rather than silently file-copied.
image_name and has_image are gone with their registry-host and tag stripping; the trap they worked around cannot occur when nothing reads the name. get_image_info stays, since the --images-* lists match exact references.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The --empty pre-clean is a catalog-wide sweep: it drops every non-template database and every non-pg_ role of the instance. On a dedicated instance that is exactly right, because the dump recreates all of it. On a shared one it destroys databases the dump does not carry, with nothing to restore them from - and no test ever executed that sweep, because the e2e dropped the cluster by hand first and left the pre-clean with zero rows to generate.
Scoping the sweep to the dump's own inventory looks like the fix and is worse. A surviving database that owns or merely grants to one of the dump's roles pins that role in pg_shdepend; DROP OWNED BY only reaches the control database the pre-clean is connected to, so DROP ROLE fails - after phase 1 has already dropped the dump's databases. ON_ERROR_STOP aborts, the replay never starts, and the instance is left half emptied.
So the instance is checked instead. --empty now refuses when the instance holds a database the dump does not carry, names it, and touches nothing. The sweep stays as it was, safe behind that refusal. Reading the dump's inventory needs a real identifier parser: a quoted name may hold spaces, and psql options precede the target of a \\connect line.
The e2e no longer drops the cluster itself, so --empty has to do it and the replay has to put it back; a second pass then adds a foreign database and requires the refusal to leave both it and the restored data alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A volume with a backing store of its own is not in a snapshot of the docker data root: it appears there as an existing empty directory, so the copy succeeds, the generation is stamped complete, and the volume is empty in it. The existing check only asked whether the path was inside the snapshot, which that empty directory answers with yes.
The driver, its options and the filesystem the mountpoint sits on now decide, per volume. An uncaptured volume is copied live - correct data without the point in time - while every other volume of the same run keeps its snapshot. One NFS volume no longer costs the whole host its consistent backup.
A volume resolving outside the subject degrades the same way instead of aborting the run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Writing into a volume's mountpoint only restores it when the mountpoint is the storage. A volume with driver options - NFS, a bind device, tmpfs - keeps the same /var/lib/docker/volumes/<name>/_data path, but docker mounts the real backing store over it on demand and unmounts it again when the last consumer stops. Restoring while nothing holds it lands in the empty directory underneath, is hidden by the next mount, and rsync reports success.
The declaration decides, not the mount table: the driver and its options are true at every moment, where the mount table is only true while a container happens to hold the volume.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A restore with --empty destroys before it replays: the pre-clean drops the schema in one session and the dump goes in the next, with no rollback across the two. A dump the engine cannot parse therefore does not fail harmlessly, it leaves an emptied database behind. The version each side is on decides that up front, so the refusal lands before the first session opens.
Both engines state their origin in the dump's own header and spell it differently. Postgres names the source server; MariaDB opens with mariadb-dump's own version and names the server further down, so matching the first number would read the tool on one engine and the server on the other. A pg_dumpall stream carries no version line of its own at all - the first belongs to the first database's embedded pg_dump output, arbitrarily far down - hence the deep scan.
Restoring forward across a major version stays allowed; that is the upgrade path. Only backward is refused, with --no-version-check as the way out.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ruff was never wired into this repository: no target, no CI step, no pin.
It reported 45 findings across sources and tests, so nothing enforced
what the codebase already mostly followed.
Adds `make ruff` (check + format --check), `make ruff-fix`, and `make
lint` as its alias, and makes `make test` run lint as a fourth parallel
spur. The CI workflow calls `make test`, so it is covered there too. The
linter is pinned in a `lint` extra: a ruff minor bump changes which rules
fire, and with the suite gating on a clean run an unpinned linter would
fail it on an unrelated day.
The 45 findings are fixed rather than configured away. Three needed a
decision instead of the mechanical fix:
- The generation timestamp keeps its local wall clock (DTZ005 waived).
Generations sort by that name, and UTC would order new ones before the
existing ones wherever the offset is positive - "newest generation" is
what every restore path selects on.
- The per-volume `copy` closure now binds volume_name and vol_dir as
default arguments (B023). It only worked because it is called inside
the same iteration.
- The two CLI top-level handlers keep their blind except (BLE001
waived): turning any failure into exit 1 is what a CLI boundary is
for. The two in run.py did not need it and were narrowed to what they
actually catch.
Also drops the comments that restate the code: the section banners in
restore/__main__.py, the filename repeated as line 1 of nine test files,
step narration above the statement it narrates, and a block in app.py
documenting parameters that had moved to another module. What names a
trip-wire stays - the snapshot destination rule, the mysql-binary
absence in MariaDB 11 images, the session-scoped FOREIGN_KEY_CHECKS, the
spooled temp file for multi-GB dumps, and the negative control that
loses its discriminating power if it ever passes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A databases.csv row asking for every database of an instance
(database = '*') makes the backup side write <instance>.cluster.backup.sql
via pg_dumpall, and nothing could read it back: the restore CLI knew
files, postgres and mariadb. That dump was stored and unrestorable - a
format whose producer had no consumer.
Adds `baudolo-restore cluster`. Three properties of a cluster stream
shape it, and each one bit during development:
- It recreates databases, and CREATE DATABASE cannot run inside a
transaction block. So unlike the single-database replay this one must
NOT be wrapped in --single-transaction. The unit tests now pin both
contracts against each other.
- It recreates every role including the one the replay connects as, and
the pre-clean cannot drop the role holding its own session. That
single CREATE ROLE is filtered out of the stream while its ALTER ROLE
is kept, because that is what carries the attributes and the password.
Found by running it: the first replay died on `role "postgres"
already exists`.
- --empty means more than for one database: the cluster's databases go
first, then DROP OWNED BY releases what a role still holds in the
control database, then the roles themselves. The order is pinned by a
phase column because \gexec would otherwise emit them interleaved, and
a role cannot be dropped while it still owns a database.
Without --empty the replay stops at the first object that already
exists. Recreating a cluster over a populated one is a decision, not a
default.
The e2e test drills the real thing: two databases and their owning role
are dropped outright and have to come back with their payload and their
ownership intact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A run starting in the same wall-clock second as its predecessor reused that
predecessor's generation directory: mkdir carried exist_ok=True, so rsync
--delete overwrote a finished generation before create_stamp_file refused the
already-stamped directory and exited 2. The guard fired after the damage.
Claim the directory exclusively instead. create_version_directory is the first
filesystem action of a run, so the abort now happens with zero writes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
has_image tested the pattern against the raw .Config.Image, so anything in the reference could decide which dump tool runs -- including the registry host and the tag. A swarm node that hosts the local registry prefixes every pull with its own name, and that node is named after the app under test, so a Postgres container reads as svc-db-mariadb-swarm-mgr-01:5000/postgres_custom:17-3.5. dumps.py tries mariadb before postgres, matched on the hostname, and dumped Postgres with mariadb-dump: exit 127, the image does not ship it. The BackupException took the backup unit down with it.
image_name strips digest, tag and registry host and matches on the repository path, so the decision rests on the image alone. Same intent as the exact --images-* matching from f9776ac, applied to the one place that commit did not reach. Tags stop deciding too: xwiki_custom:lts-postgres-tomcat no longer reads as Postgres.
The e2e reproduces the shape without a registry -- a docker tag is enough for .Config.Image to carry the reference verbatim -- and asserts a real pg_dump lands. Under the old code mariadb-dump aborts and no dump file exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
infinito-nexus-core gates `git commit` behind an explicit per-invocation
confirmation. This repository did not, so an agent working here could commit
without being asked. Mirror the rule.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each volume is copied twice into the same destination: once hot with the
container running, once cold after it is stopped (backup/app.py:127-131).
rsync ran with -b, so --delete did not remove a file the source had dropped
between the passes - it renamed it. Stopping a container is exactly what
makes the source drop files: a graceful shutdown flushes, and the format
rolls its commit point.
For an opaque payload the twin is stale bytes nobody reads. For a format
that enumerates its own directory it is corruption. Lucene resolves the
current commit by parsing every file starting with segments as a radix-36
generation, so a restored segments_3~ raises NumberFormatException, the
shard store cannot be read, and the primary is left NO_VALID_SHARD_COPY.
With .security-7 unallocatable the reserved elastic user has no password
hash, every probe gets HTTP 401, and the container never turns healthy.
Seen in infinito-nexus-core CI run 30963648828: the restore drill waited
1200s on elasticsearch while all 29 other containers came back healthy;
the generation held segments_3~ next to segments_4 in all three index
directories. A run two days earlier passed the same drill because that
stack was idle and nothing rolled between the passes - which is why this
reads as flaky rather than broken.
Reproduced with real rsync in all three shapes: two passes into the same
destination with -b leave segments_3~ beside segments_4, without -b only
segments_4 survives, and a single pass keeps segments_3. Measured on the
same fixtures, dropping -b leaves the predecessor generation byte-identical
and keeps the --link-dest hardlinks intact, so the incremental scheme is
unaffected; generations get smaller, never larger.
--link-dest already provides the cheap incrementals. -b contributed nothing
on top of it but the twins, and no caller reads them: the restore path is an
unfiltered rsync -avv --delete into the live volume (restore/files.py:35).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
-a implies -D, so a generation was written with --devices --specials and rsync recreated every unix socket and fifo it found in a volume. On the swarm manager that generation lives on an nfs-ganesha export, and ganesha accepts the socket on write but cannot serve it back: the remote pull's sender then fails with readdir/readlink_stat 'Invalid argument (22)' and exits 23, deterministically, for all twelve retries - 58 minutes per run.
Postfix's queue directory is the case that surfaced it, where public/ and private/ hold roughly fifty AF_UNIX sockets and nothing else. The class is wider: a discourse /shared with its in-container postgres socket, a checkmk OMD site with tmp/run/nagios.cmd, a container whose /tmp is a persisted volume. --no-D is type-based and closes all of them without anyone having to know which image binds a socket where.
Nothing restorable is lost. A socket inode is meaningless after a restore; postfix's master, checkmk's omd start and discourse's supervisor recreate theirs. The whole postfix queue survives - incoming, active, deferred, hold, maildrop - so accepted-but-undelivered mail stays in the backup, which excluding the volume outright would have dropped. Device nodes go too, and the only volume that could hold them is a nested docker data root, already carrying backup: false.
This is the writer side, whose source is a local docker volume. On the reader the same flag provably does nothing: rsync still stats the entry before -D decides, and getdents64 on the containing directory is outside its reach entirely.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
volume_is_fully_ignored can only skip a volume when every container using it is ignored, so a container holding a derived tree next to state that must be kept cannot express the exclusion at all. The matrix docker-in-docker runner is exactly that: matrix_mdad_docker, matrix_mdad_matrix and matrix_mdad_state all hang off one container, and the derived one is an inner overlay2 store that no rsync in the chain can restore faithfully (none carries -X, so trusted.overlay.* is stripped in both directions).
--volumes-no-backup-required names volumes directly. The check runs before containers_using_volume, so an excluded volume costs no docker call and the decision no longer depends on which containers happen to exist at backup time.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The kernel refuses a snapshot whose destination sits on another
filesystem. Placing it at <parent>/.baudolo-<tag> hit that on the most
sensible layout there is: a dedicated disk mounted straight onto
/var/lib/docker, where the parent directory belongs to a different
filesystem and every run failed with EXDEV.
Placing it at <subject>/.baudolo-<tag> makes source and destination the
same filesystem by construction, so the failure cannot occur on any
layout. It also aligns the two backends: the zfs path already resolves
its snapshot inside the subject, at <subject>/.zfs/snapshot/<tag>.
btrfs does not include nested subvolumes in a snapshot, so a leftover
from an interrupted run appears in the next snapshot as an empty
directory rather than recursing, and the copy only ever reads the volume
tree below it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four defects in the snapshot mode 3.2.0 introduced.
The resolver dropped the trailing separator get_storage_path puts on a
volume path, because os.path.abspath strips it. rsync reads "dir" as
"copy the directory" where "dir/" means "copy its contents", so every
snapshot generation landed at <volume>/files/_data/... while the live
path lands at <volume>/files/... . Restores read the live layout, and
--link-dest found nothing to match against the previous generation. The
e2e never caught it because its driver appended the separator by hand.
Snapshot teardown was fatal and masking. A busy `btrfs subvolume delete`
raised out of the finally, which skipped the generation stamp and the
compose handling on a run whose data was already complete, and replaced
whatever the body had raised. The leftover is reported instead; removing
it is a cleanup problem, not a reason to discard a good generation.
A volume created after the snapshot was taken aborted the whole run: the
volume list is enumerated inside the snapshot context, and nothing is
stopped in snapshot mode, so the host keeps creating volumes for the
duration of the copy. Such a volume is now copied live with a warning,
which is exactly what the pre-snapshot code did for it.
The snapshot pass compares by content again. 3.2.0 dropped --checksum
because a snapshot source cannot move, which is true, but the comparison
that matters is against --link-dest: a file that changed while keeping
its size and whole-second mtime was hard-linked stale out of the previous
generation, and the single snapshot pass had no authoritative pass to
repair it the way the live path does. It is still one pass against two.
--hard-restart-projects is refused alongside --snapshot, the same way
--shutdown already is: the flag exists for stacks whose database cannot
be backed up hot, which is what a snapshot removes.
Tests: the trailing separator, both teardown behaviours, the new refusal,
and app.main driving the snapshot branch - the caller that runs in
production, which no test had exercised and where the layout defect
therefore stayed invisible. The e2e driver now feeds the resolver the
string shape get_storage_path really produces.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Backing up a live volume with rsync copies a moving target: a database
written to mid-copy lands on disk in a state no engine ever committed.
Stopping the container avoids that at the cost of downtime.
A snapshot removes both. `--snapshot {btrfs,zfs}` with `--snapshot-subject`
freezes the docker root once per run, and every volume copy is then read
from that frozen tree while the containers keep serving. A restore of such
a copy is an ordinary crash recovery, which every supported engine performs
on its own at startup.
An unsupported filesystem or an unknown snapshot kind fails loudly rather
than degrading to a live copy, since a silent fallback would return exactly
the torn backup the mode exists to prevent. `--shutdown` is rejected
alongside `--snapshot` instead of being ignored: under a snapshot no
container is ever stopped, so accepting the flag would promise downtime
semantics the run does not deliver.
Copies out of a snapshot skip rsync's --checksum verification. The source
is immutable for the lifetime of the copy, so size-and-mtime cannot race,
and dropping the second full read roughly halves the I/O per volume.
backup/app.py grew past what one module could carry and is split into
layout, policy and dumps along the lines it already had internally.
Tests: unit coverage for the new snapshot, layout, policy, volume and cli
units; e2e cases drive real btrfs, zfs and ext4 filesystems on loop devices
in a privileged container, including a MariaDB that is written to across
the snapshot and must recover from the restored copy without losing a
committed row. CI installs zfs and sets E2E_REQUIRE_FILESYSTEMS so a
missing kernel module fails the build instead of silently skipping a
filesystem.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each volume is copied twice into the same destination: once hot with the
container running, once cold after it is stopped. The cold pass is meant to
correct the hot one, but it uses rsync's quick check, which compares size and
whole-second mtime.
A pre-allocated 16 MiB WAL segment never changes size. When its last pre-stop
write and postgres' shutdown checkpoint fall in the same whole second, the cold
pass skips it while still replacing global/pg_control, whose previous write was
seconds earlier. The generation then holds a post-shutdown pg_control pointing at
a checkpoint LSN whose WAL bytes are still zero. Restoring it gives
invalid record length at 0/3CB7A20: expected at least 24, got 0
invalid checkpoint record
PANIC: could not locate a valid checkpoint record at 0/3CB7A20
and postgres crash-loops until the restore test gives up after 1200s. Observed in
infinito-nexus-core CI run 30586213932, debian leg; the centos leg of the same
run restored cleanly, and the two generations differ by exactly one 16 MiB WAL
segment.
authoritative is a required keyword rather than an optional flag, so each of the
four call sites states which pass it is. Set, it adds --checksum, so the cold
pass compares content. The hot passes keep the quick check.
Reproduced with real rsync in both directions: without --checksum the WAL stays
zero while pg_control is post-shutdown; with it the pair is consistent. Hardlink
dedup against --link-dest is unaffected, measured at 20/20 with generation size
unchanged.
Rejected alternatives, both measured rather than argued: --ignore-times destroys
dedup (20/20 to 0/20 hardlinks, generation 36 to 72 MiB), and --modify-window=-1
makes correctness depend on the destination filesystem preserving sub-second
mtimes, which silently degrades on NFS or ext3.
COST. The quick check scales with file count; --checksum scales with bytes, read
on both sides, and it runs while the container is stopped. Measured warm-cache on
215 MiB: 0.10s to 0.39s, a factor of 3.9. Derived for disk-bound runs, the extra
stop time stays under a minute up to roughly 4 GB on spinning disk, 15 GB on a
SATA SSD, 60 GB on NVMe. At 1 TB it is 17 minutes on NVMe and over three hours on
spinning disk; at 10 TB it is hours to days. No size threshold is built in - a
guessed one would drop the guarantee exactly where an unrestorable backup costs
most. Volumes at that scale need filesystem snapshots or pg_basebackup rather
than two rsync passes over a live tree, which is a limit of this approach and not
of this change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The dump replay ran statement-by-statement with autocommit. When restore --empty runs against a LIVE database, the pre-clean drops a table, the replay recreates it and autocommits, and a concurrent writer (discourse's mini_scheduler upserting scheduler_stats(id=1)) inserts the same primary key into the empty table before the dump's COPY loads it. The COPY then aborts with a duplicate-key violation under ON_ERROR_STOP and the whole restore fails. Running the replay with --single-transaction keeps the recreated table invisible to other sessions until commit, so the writer can never insert the racing row.
The --empty pre-clean stays multi-statement (\gexec, one DROP per statement): running every DROP in one transaction exhausts max_locks_per_transaction on large schemas (e.g. gitlab).
Extract the pre-clean SQL from the inline string into restore/db/empty_preclean.sql (loaded via dirname(__file__)) and declare it as package-data so it ships in the wheel. Add a unit test guarding the single-transaction/multi-statement split and an e2e that reproduces the live-writer race.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Release metadata (version bump and changelog entry) stays a manual step; agent edits to CHANGELOG.md and pyproject.toml now require explicit operator approval. The local .mcp.json is ignored.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A schema shipping a custom text search dictionary (taiga's english_stem_nostop) survived the pre-clean and aborted the dump replay with a duplicate pg_ts_dict_dictname_index violation under ON_ERROR_STOP. The discovery SELECT now also enumerates user-owned pg_ts_config and pg_ts_dict entries. The string-assertion unit test is replaced by real scenario data in the e2e: the seeded schema now contains an overloaded f()/f(int) pair and the nostop dictionary plus configuration, and the restored database is queried to prove each survives the backup, pre-clean and replay cycle exactly once.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A signature-less DROP FUNCTION aborts with 'function name is not unique' as soon as a schema overloads a name, killing the whole --empty replay under ON_ERROR_STOP. The function branch now emits name(identity args) via pg_get_function_identity_arguments; because that compound must not be identifier-quoted as a whole, quoting moves from the outer format into each branch, guarded by a unit test that pins the per-branch %I quoting.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
