Commit Graph

9 Commits

Author SHA1 Message Date
efcfe88e7f style!: adopt the core lint bar and migrate to it
The package carried no ruff configuration at all, so it ran on the defaults (E4+E7+E9+F) while infinito-nexus-core, its only consumer, holds itself to a far wider selection. Measured against that selection this tree had 224 findings. It now has none.

The selector list is core's verbatim so both repositories answer to one bar. target-version stays py39 rather than core's py311, because requires-python still declares >=3.9 and pyupgrade would otherwise propose syntax the declared minimum cannot run. Every ignore carries its reason: S603/S607 in particular, since running docker and dump binaries from PATH in list form is this tool's whole job and is already injection-safe.

Two conversions are judgement rather than mechanics. os.path.join(dir, '') was the rsync idiom for a trailing separator, which Path drops, so it becomes an explicit os.sep. os.path.abspath stays where Path.resolve() would follow symlinks and let a symlinked volume test as inside the snapshot subject.

BREAKING CHANGE: VersionMismatch is renamed VersionMismatchError.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 01:51:18 +02:00
df1c65ccac feat(backup)!: mandatory repo-name and databases-csv, --only-files, --only-sql
Two defaults could not be right. --repo-name fell back to the literal 'backup-docker-to-local' while its help promised the git repo folder name, which nothing ever derived. --databases-csv pointed inside the installed package directory, where credentials must not live; when it applied, load_databases_df read a missing file as empty and the run finished without a single dump and without an error. Both are required now, --repo-name in the restore CLI too. The file itself may still be absent - babadcb's tolerance is untouched, only the path must be named.

--everything is withdrawn. Its one effect was to ignore --images-no-stop-required, which is what leaving that list empty already does, and its branch was the default path minus the requires_stop check. No caller, no test, and help and README described it differently.

--dump-only-sql becomes --only-sql, and --only-files joins it as the opposite half: no dumps at all, every volume as files. They form a mutually exclusive group. A host that only copies files has no business holding database passwords, so --databases-csv is not required there and is never read.

The smallest valid argv turned out to be written four times across the test tree; it now lives once. Withdrawn flags are listed in one place and proven to exit 2.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:54:39 +02:00
bb647c66ec feat(restore): refuse a dump the target engine cannot read
A restore with --empty destroys before it replays: the pre-clean drops the schema in one session and the dump goes in the next, with no rollback across the two. A dump the engine cannot parse therefore does not fail harmlessly, it leaves an emptied database behind. The version each side is on decides that up front, so the refusal lands before the first session opens.

Both engines state their origin in the dump's own header and spell it differently. Postgres names the source server; MariaDB opens with mariadb-dump's own version and names the server further down, so matching the first number would read the tool on one engine and the server on the other. A pg_dumpall stream carries no version line of its own at all - the first belongs to the first database's embedded pg_dump output, arbitrarily far down - hence the deep scan.

Restoring forward across a major version stays allowed; that is the upgrade path. Only backward is refused, with --no-version-check as the way out.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 05:25:29 +02:00
2129c5e362 build(lint): gate make test on a clean ruff run
ruff was never wired into this repository: no target, no CI step, no pin.
It reported 45 findings across sources and tests, so nothing enforced
what the codebase already mostly followed.

Adds `make ruff` (check + format --check), `make ruff-fix`, and `make
lint` as its alias, and makes `make test` run lint as a fourth parallel
spur. The CI workflow calls `make test`, so it is covered there too. The
linter is pinned in a `lint` extra: a ruff minor bump changes which rules
fire, and with the suite gating on a clean run an unpinned linter would
fail it on an unrelated day.

The 45 findings are fixed rather than configured away. Three needed a
decision instead of the mechanical fix:

- The generation timestamp keeps its local wall clock (DTZ005 waived).
  Generations sort by that name, and UTC would order new ones before the
  existing ones wherever the offset is positive - "newest generation" is
  what every restore path selects on.
- The per-volume `copy` closure now binds volume_name and vol_dir as
  default arguments (B023). It only worked because it is called inside
  the same iteration.
- The two CLI top-level handlers keep their blind except (BLE001
  waived): turning any failure into exit 1 is what a CLI boundary is
  for. The two in run.py did not need it and were narrowed to what they
  actually catch.

Also drops the comments that restate the code: the section banners in
restore/__main__.py, the filename repeated as line 1 of nine test files,
step narration above the statement it narrates, and a block in app.py
documenting parameters that had moved to another module. What names a
trip-wire stays - the snapshot destination rule, the mysql-binary
absence in MariaDB 11 images, the session-scoped FOREIGN_KEY_CHECKS, the
spooled temp file for multi-GB dumps, and the negative control that
loses its discriminating power if it ever passes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 04:35:50 +02:00
a0204fd3ea feat(restore): replay pg_dumpall cluster dumps
A databases.csv row asking for every database of an instance
(database = '*') makes the backup side write <instance>.cluster.backup.sql
via pg_dumpall, and nothing could read it back: the restore CLI knew
files, postgres and mariadb. That dump was stored and unrestorable - a
format whose producer had no consumer.

Adds `baudolo-restore cluster`. Three properties of a cluster stream
shape it, and each one bit during development:

- It recreates databases, and CREATE DATABASE cannot run inside a
  transaction block. So unlike the single-database replay this one must
  NOT be wrapped in --single-transaction. The unit tests now pin both
  contracts against each other.
- It recreates every role including the one the replay connects as, and
  the pre-clean cannot drop the role holding its own session. That
  single CREATE ROLE is filtered out of the stream while its ALTER ROLE
  is kept, because that is what carries the attributes and the password.
  Found by running it: the first replay died on `role "postgres"
  already exists`.
- --empty means more than for one database: the cluster's databases go
  first, then DROP OWNED BY releases what a role still holds in the
  control database, then the roles themselves. The order is pinned by a
  phase column because \gexec would otherwise emit them interleaved, and
  a role cannot be dropped while it still owns a database.

Without --empty the replay stops at the first object that already
exists. Recreating a cluster over a populated one is a decision, not a
default.

The e2e test drills the real thing: two databases and their owning role
are dropped outright and have to come back with their payload and their
ownership intact.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 04:19:03 +02:00
e5da813a9f refactor(restore)!: rsync volumes directly on the host, drop alpine-rsync
Restore now resolves the target volume's mountpoint via docker volume
inspect and rsyncs into it directly, mirroring how the backup path
already reads the mountpoint; the alpine-rsync container and the
--rsync-image flag are gone. The e2e harness mounts /var/lib/docker
read-write in the test container so the direct restore can write, the
same way baudolo runs as root on a real host.

BREAKING CHANGE: the restore 'files' subcommand no longer accepts
--rsync-image; the caller must have write access to the docker volume
root (root on the host), which is baudolo's normal privilege.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 14:53:55 +02:00
6adafe6b1f fix(backup): log missing db config instead of raising
- Use module logger in backup/db.py
- Skip db dump when no databases.csv entry is present
- Apply black/formatting cleanup across backup/restore/tests

https://chatgpt.com/share/69519d45-b0dc-800f-acb6-6fb8504e9b46
2025-12-28 22:12:31 +01:00
4af15d9074 fix(restore): allow restoring files from different source volume
The file restore command previously assumed that the target volume name
was identical to the volume name used in the backup path. This caused
restores to fail (exit code 2) when restoring data from one volume backup
into a different target volume.

Introduce an optional --source-volume argument for `baudolo-restore files`
to explicitly specify the backup source volume while keeping the target
volume unchanged.

- Default behavior remains fully backward-compatible
- Enables restoring backups from volume A into volume B
- Fixes E2E test scenario restoring into a new volume

Tests:
- Update E2E file restore test to use --source-volume

https://chatgpt.com/share/694ec70f-1d7c-800f-b221-9d22e4b0775e
2025-12-26 18:33:58 +01:00
c30b4865d4 refactor: migrate to src/ package + add DinD-based E2E runner with debug artifacts
- Replace legacy standalone scripts with a proper src-layout Python package
  (baudolo backup/restore/configure entrypoints via pyproject.toml)
- Remove old scripts/files (backup-docker-to-local.py, recover-docker-from-local.sh,
  databases.csv.tpl, Todo.md)
- Add Dockerfile to build the project image for local/E2E usage
- Update Makefile: build image and run E2E via external runner script
- Add scripts/test-e2e.sh:
  - start DinD + dedicated network
  - recreate DinD data volume (and shared /tmp volume)
  - pre-pull helper images (alpine-rsync, alpine)
  - load local baudolo:local image into DinD
  - run unittest E2E suite inside DinD and abort on first failure
  - on failure: dump host+DinD diagnostics and archive shared /tmp into artifacts/
- Add artifacts/ debug outputs produced by failing E2E runs (logs, events, tmp archive)

https://chatgpt.com/share/694ec23f-0794-800f-9a59-8365bc80f435
2025-12-26 18:13:26 +01:00