Commit Graph

10 Commits

Author SHA1 Message Date
f9776ac47a feat(backup)!: exact --images-* matching and --hard-restart-projects
Match --images-no-stop-required and --images-no-backup-required against
the container's exact .Config.Image instead of a substring, so callers
pass full repo:tag references (registry prefix included) and near-miss
image names no longer flip the stop/skip decision. Rename the opt-in
--hard-compose-restart flag to --hard-restart-projects.

The e2e suite pins a SPOT for the DB images and in-container data dirs
(postgres:alpine at /var/lib/postgresql, mariadb:latest at
/var/lib/mysql) and passes exact image refs to the --images-* flags.

BREAKING CHANGE: --images-* now require exact image references, not
substrings; --hard-compose-restart is renamed to --hard-restart-projects.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 18:19:20 +02:00
331931d617 feat(backup)!: opt-in hard restart and mandatory backups-dir
Rename --docker-compose-hard-restart-required to --hard-compose-restart
and change its default from ["mailu"] to [] (nargs="*"): the compose
down/up is now opt-in, so compose hosts pass "mailu" while swarm hosts,
where the dir is a stack whose overlay network collides with compose up,
pass nothing. Make --backups-dir mandatory (no /var/lib/backup/ default)
so a run can never silently target the wrong backup root.

BREAKING CHANGE: the old flag name is removed, the implicit mailu default
is gone, and --backups-dir must be passed explicitly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 14:53:13 +02:00
e9030e8443 fix(backup,restore): make restore drills replayable and leave swarm tasks alone
Restore fixes, both hit by the infinito svc-bkp e2e drill:

- mariadb --empty dropped tables one docker exec at a time with SET
  FOREIGN_KEY_CHECKS=0 issued in a separate client session, so the
  session-scoped toggle never applied and any FK-referenced parent table
  (mailu.users) died with ERROR 1451. Issue the toggle and all DROPs in
  one session.
- postgres --empty now drops only current_user-owned objects (extension
  members like pg_trgm's set_limit are superuser-owned) with IF EXISTS
  absorbing CASCADE fallout, and the replay skips superuser-only dump
  lines (COMMENT ON EXTENSION, ALTER DEFAULT PRIVILEGES) that abort an
  app-user psql run under ON_ERROR_STOP.

Backup fixes:

- pg_dump now runs with --no-owner --no-privileges so future dumps are
  replayable by the owning app user in the first place.
- Swarm task containers are never stopped or started manually: the
  orchestrator replaces a stopped task and a later docker start fails on
  the detached overlay network. filter_stoppable skips them visibly and
  the whitelist stop check ignores them.

Validated end to end against a live infinito compose stack: the full
svc-bkp-volume-2-local drill (verify, restore cycle, sql replay for
mailu, keycloak and one more db) passes with these patches applied.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 08:23:11 +02:00
babadcb038 fix(backup,ci): make databases.csv optional and upgrade Docker CLI in image
- Handle missing or empty databases.csv gracefully with warnings and empty DataFrame
- Add unit tests for robust databases.csv loading behavior
- Adjust seed tests to assert warnings across multiple print calls
- Replace Debian docker.io with docker-ce-cli to avoid Docker API version mismatch
- Install required build tools (curl, gnupg) for Docker repo setup

https://chatgpt.com/share/697e6d9d-6458-800f-9d12-1e337509be4e
2026-01-31 22:01:12 +01:00
d976640312 fix(seed): handle empty databases.csv and add unit tests
- Gracefully handle empty databases.csv by creating header columns and emitting a warning
- Add _empty_df() helper for consistent DataFrame initialization
- Add unit tests for baudolo-seed including empty-file regression case
- Apply minor formatting fixes across backup and e2e test files

https://chatgpt.com/share/69628f0b-8744-800f-b08d-2633e05167da
2026-01-10 18:40:22 +01:00
f3ef86a444 feat(backup): stricter databases.csv semantics + atomic SQL dumps
- read databases.csv with stable types (dtype=str, keep_default_na=False)
- validate database field: require '*' or concrete name (no empty/NaN)
- support Postgres cluster dumps via '*' entries (pg_dumpall)
- write SQL dumps atomically to avoid partial/empty files
- early-skip fully ignored volumes before creating backup directories
- update seed CLI to enforce new contract and update by (instance,database)
- adjust tests: sql dir naming + add E2E coverage for early-skip and '*' seeding
2025-12-29 11:39:57 +01:00
c01ab55f2d test(e2e): add dump-only-sql mixed-run + CLI contract coverage
- rename dump-only flag to --dump-only-sql across docs and tests
- update backup logic: skip files/ only for DB volumes when dumps succeed; fallback to files when dumps fail
- extend e2e helpers to support dump_only_sql
- add e2e mixed-run regression test (DB dump => no files/, non-DB => files/)
- add e2e CLI/argparse contract test (--dump-only-sql present, --dump-only rejected)
- fix e2e files test to expect file backups for non-DB volumes in dump-only-sql mode and verify restore
- update changelog + README flag table

https://chatgpt.com/share/69522d9c-ce08-800f-9070-71df3bd779ae
2025-12-29 08:28:23 +01:00
df32671cec fix(backup): fallback to file backup in dump-only mode when no DB dump is possible
- Change DB backup helpers to return whether a dump was actually produced
- Detect DB containers without successful dumps in --dump-only mode
- Fallback to file backups with a warning instead of skipping silently
- Refactor DB dump logic to return boolean status
- Add E2E test covering dump-only fallback when databases.csv entry is missing

https://chatgpt.com/share/6951a659-2b0c-800f-aafa-3e89ae1eb697
2025-12-28 22:51:12 +01:00
6adafe6b1f fix(backup): log missing db config instead of raising
- Use module logger in backup/db.py
- Skip db dump when no databases.csv entry is present
- Apply black/formatting cleanup across backup/restore/tests

https://chatgpt.com/share/69519d45-b0dc-800f-acb6-6fb8504e9b46
2025-12-28 22:12:31 +01:00
c30b4865d4 refactor: migrate to src/ package + add DinD-based E2E runner with debug artifacts
- Replace legacy standalone scripts with a proper src-layout Python package
  (baudolo backup/restore/configure entrypoints via pyproject.toml)
- Remove old scripts/files (backup-docker-to-local.py, recover-docker-from-local.sh,
  databases.csv.tpl, Todo.md)
- Add Dockerfile to build the project image for local/E2E usage
- Update Makefile: build image and run E2E via external runner script
- Add scripts/test-e2e.sh:
  - start DinD + dedicated network
  - recreate DinD data volume (and shared /tmp volume)
  - pre-pull helper images (alpine-rsync, alpine)
  - load local baudolo:local image into DinD
  - run unittest E2E suite inside DinD and abort on first failure
  - on failure: dump host+DinD diagnostics and archive shared /tmp into artifacts/
- Add artifacts/ debug outputs produced by failing E2E runs (logs, events, tmp archive)

https://chatgpt.com/share/694ec23f-0794-800f-9a59-8365bc80f435
2025-12-26 18:13:26 +01:00