One DO-block ran every DROP in a single transaction and exhausted
max_locks_per_transaction on large schemas (gitlab); emit one DROP per
row and execute via \gexec instead. Also drop user-owned non-public
schemas so a dump that CREATE SCHEMAs (discourse) does not abort on the
existing schema under ON_ERROR_STOP.
Covered by a new DinD e2e test restoring --empty against a fully
populated database with a non-public schema and every object class.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Match --images-no-stop-required and --images-no-backup-required against
the container's exact .Config.Image instead of a substring, so callers
pass full repo:tag references (registry prefix included) and near-miss
image names no longer flip the stop/skip decision. Rename the opt-in
--hard-compose-restart flag to --hard-restart-projects.
The e2e suite pins a SPOT for the DB images and in-container data dirs
(postgres:alpine at /var/lib/postgresql, mariadb:latest at
/var/lib/mysql) and passes exact image refs to the --images-* flags.
BREAKING CHANGE: --images-* now require exact image references, not
substrings; --hard-compose-restart is renamed to --hard-restart-projects.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Backup: a container that vanishes between the docker ps listing and the
swarm-task inspect (--rm one-shots, task-history GC) no longer aborts the
whole backup run; it counts as not stoppable and is skipped.
Restore: the postgres replay streams the dump through a spooled temp file
instead of buffering it three times in memory (multi-GB dumps OOMed the
restore mid-replay), and the superuser-only line filter is COPY-aware: data
rows inside COPY ... FROM stdin blocks pass through untouched, so a row
that happens to start with COMMENT ON EXTENSION or ALTER DEFAULT PRIVILEGES
is no longer silently dropped.
The e2e runner talks to the DinD daemon through docker exec instead of a
host-published tcp://127.0.0.1:2375: port publishing is unreachable from
sandboxed runners and from hosts with broken loopback publishing, and the
unencrypted root API port disappears from the host. The debug tmp dump
shrinks to tar plus docker cp against the DinD container itself.
New coverage: an e2e reproducing the swarm flake end to end (service task
on the volume, nothing whitelisted: the backup must succeed, the very same
task container must keep running, and the service must never replace a
task), unit tests for the COPY-aware filter, the swarm-task probe including
the vanished-container path, filter_stoppable ordering, and the one-session
FOREIGN_KEY_CHECKS drop assembly. Full suite: 35 unit, 9 integration,
30 e2e green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Restore fixes, both hit by the infinito svc-bkp e2e drill:
- mariadb --empty dropped tables one docker exec at a time with SET
FOREIGN_KEY_CHECKS=0 issued in a separate client session, so the
session-scoped toggle never applied and any FK-referenced parent table
(mailu.users) died with ERROR 1451. Issue the toggle and all DROPs in
one session.
- postgres --empty now drops only current_user-owned objects (extension
members like pg_trgm's set_limit are superuser-owned) with IF EXISTS
absorbing CASCADE fallout, and the replay skips superuser-only dump
lines (COMMENT ON EXTENSION, ALTER DEFAULT PRIVILEGES) that abort an
app-user psql run under ON_ERROR_STOP.
Backup fixes:
- pg_dump now runs with --no-owner --no-privileges so future dumps are
replayable by the owning app user in the first place.
- Swarm task containers are never stopped or started manually: the
orchestrator replaces a stopped task and a later docker start fails on
the detached overlay network. filter_stoppable skips them visibly and
the whitelist stop check ignores them.
Validated end to end against a live infinito compose stack: the full
svc-bkp-volume-2-local drill (verify, restore cycle, sql replay for
mailu, keycloak and one more db) passes with these patches applied.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Gracefully handle empty databases.csv by creating header columns and emitting a warning
- Add _empty_df() helper for consistent DataFrame initialization
- Add unit tests for baudolo-seed including empty-file regression case
- Apply minor formatting fixes across backup and e2e test files
https://chatgpt.com/share/69628f0b-8744-800f-b08d-2633e05167da
- read databases.csv with stable types (dtype=str, keep_default_na=False)
- validate database field: require '*' or concrete name (no empty/NaN)
- support Postgres cluster dumps via '*' entries (pg_dumpall)
- write SQL dumps atomically to avoid partial/empty files
- early-skip fully ignored volumes before creating backup directories
- update seed CLI to enforce new contract and update by (instance,database)
- adjust tests: sql dir naming + add E2E coverage for early-skip and '*' seeding
- rename dump-only flag to --dump-only-sql across docs and tests
- update backup logic: skip files/ only for DB volumes when dumps succeed; fallback to files when dumps fail
- extend e2e helpers to support dump_only_sql
- add e2e mixed-run regression test (DB dump => no files/, non-DB => files/)
- add e2e CLI/argparse contract test (--dump-only-sql present, --dump-only rejected)
- fix e2e files test to expect file backups for non-DB volumes in dump-only-sql mode and verify restore
- update changelog + README flag table
https://chatgpt.com/share/69522d9c-ce08-800f-9070-71df3bd779ae
- Change DB backup helpers to return whether a dump was actually produced
- Detect DB containers without successful dumps in --dump-only mode
- Fallback to file backups with a warning instead of skipping silently
- Refactor DB dump logic to return boolean status
- Add E2E test covering dump-only fallback when databases.csv entry is missing
https://chatgpt.com/share/6951a659-2b0c-800f-aafa-3e89ae1eb697
- add `make clean` and run it before `test-e2e` to avoid stale artifacts
- restore: do not hardcode `mysql` for --empty; use detected mariadb/mysql client
- e2e: improve subprocess error output for easier CI debugging
- e2e: adjust MariaDB readiness checks for socket-only root on MariaDB 11
- e2e: add `wait_for_mariadb_sql` and run SQL readiness checks for dedicated TCP user
- e2e: update MariaDB full/no-copy tests to use dedicated user over TCP and consistent credentials
https://chatgpt.com/share/694ecfb8-f8a8-800f-a1c9-a5f410d4ba02
The file restore command previously assumed that the target volume name
was identical to the volume name used in the backup path. This caused
restores to fail (exit code 2) when restoring data from one volume backup
into a different target volume.
Introduce an optional --source-volume argument for `baudolo-restore files`
to explicitly specify the backup source volume while keeping the target
volume unchanged.
- Default behavior remains fully backward-compatible
- Enables restoring backups from volume A into volume B
- Fixes E2E test scenario restoring into a new volume
Tests:
- Update E2E file restore test to use --source-volume
https://chatgpt.com/share/694ec70f-1d7c-800f-b221-9d22e4b0775e
