mirror of
https://github.com/kevinveenbirkenbach/docker-volume-backup.git
synced 2025-12-27 11:06:35 +00:00
Kevin Veen-Birkenbach
e0b2e8934e
- clarify separation between file backups (always) and SQL dumps (explicit only) - document correct nested backup directory layout - remove legacy script-based usage and outdated sections - add explicit explanation of database definition scope - update usage examples to current baudolo CLI https://chatgpt.com/share/694ef6d2-7584-800f-a32b-27367f234d1d
218 lines
6.6 KiB
Markdown
218 lines
6.6 KiB
Markdown
# baudolo – Deterministic Backup & Restore for Docker Volumes 📦🔄
|
||
[](https://github.com/sponsors/kevinveenbirkenbach) [](https://www.patreon.com/c/kevinveenbirkenbach) [](https://buymeacoffee.com/kevinveenbirkenbach) [](https://s.veen.world/paypaldonate) [](https://www.gnu.org/licenses/agpl-3.0) [](https://www.docker.com) [](https://www.python.org) [](https://github.com/kevinveenbirkenbach/backup-docker-to-local/stargazers)
|
||
|
||
|
||
`baudolo` is a backup and restore system for Docker volumes with
|
||
**mandatory file backups** and **explicit, deterministic database dumps**.
|
||
It is designed for environments with many Docker services where:
|
||
- file-level backups must always exist
|
||
- database dumps must be intentional, predictable, and auditable
|
||
|
||
## ✨ Key Features
|
||
|
||
- 📦 Incremental Docker volume backups using `rsync --link-dest`
|
||
- 🗄 Optional SQL dumps for:
|
||
- PostgreSQL
|
||
- MariaDB / MySQL
|
||
- 🌱 Explicit database definition for SQL backups (no auto-discovery)
|
||
- 🧾 Backup integrity stamping via `dirval` (Python API)
|
||
- ⏸ Automatic container stop/start when required for consistency
|
||
- 🚫 Whitelisting of containers that do not require stopping
|
||
- ♻️ Modular, maintainable Python architecture
|
||
|
||
|
||
## 🧠 Core Concept (Important!)
|
||
|
||
`baudolo` **separates file backups from database dumps**.
|
||
|
||
- **Docker volumes are always backed up at file level**
|
||
- **SQL dumps are created only for explicitly defined databases**
|
||
|
||
This results in the following behavior:
|
||
|
||
| Database defined | File backup | SQL dump |
|
||
|------------------|-------------|----------|
|
||
| No | ✔ yes | ✘ no |
|
||
| Yes | ✔ yes | ✔ yes |
|
||
|
||
## 📁 Backup Layout
|
||
|
||
Backups are stored in a deterministic, fully nested structure:
|
||
|
||
```text
|
||
<backups-dir>/
|
||
└── <machine-hash>/
|
||
└── <repo-name>/
|
||
└── <timestamp>/
|
||
└── <volume-name>/
|
||
├── files/
|
||
└── sql/
|
||
└── <database>.backup.sql
|
||
```
|
||
|
||
### Meaning of each level
|
||
|
||
* `<machine-hash>`
|
||
SHA256 hash of `/etc/machine-id` (host separation)
|
||
|
||
* `<repo-name>`
|
||
Logical backup namespace (project / stack)
|
||
|
||
* `<timestamp>`
|
||
Backup generation (`YYYYMMDDHHMMSS`)
|
||
|
||
* `<volume-name>`
|
||
Docker volume name
|
||
|
||
* `files/`
|
||
Incremental file backup (rsync)
|
||
|
||
* `sql/`
|
||
Optional SQL dumps (only for defined databases)
|
||
|
||
## 🚀 Installation
|
||
|
||
### Local (editable install)
|
||
|
||
```bash
|
||
python3 -m venv .venv
|
||
source .venv/bin/activate
|
||
pip install -e .
|
||
```
|
||
|
||
## 🌱 Database Definition (SQL Backup Scope)
|
||
|
||
### How SQL backups are defined
|
||
|
||
`baudolo` creates SQL dumps **only** for databases that are **explicitly defined**
|
||
via configuration (e.g. a databases definition file or seeding step).
|
||
|
||
If a database is **not defined**:
|
||
|
||
* its Docker volume is still backed up (files)
|
||
* **no SQL dump is created**
|
||
|
||
> No database definition → file backup only
|
||
> Database definition present → file backup + SQL dump
|
||
|
||
### Why explicit definition?
|
||
|
||
`baudolo` does **not** inspect running containers to guess databases.
|
||
|
||
Databases must be explicitly defined to guarantee:
|
||
|
||
* deterministic backups
|
||
* predictable restore behavior
|
||
* reproducible environments
|
||
* zero accidental production data exposure
|
||
|
||
### Required database metadata
|
||
|
||
Each database definition provides:
|
||
|
||
* database instance (container or logical instance)
|
||
* database name
|
||
* database user
|
||
* database password
|
||
|
||
This information is used by `baudolo` to execute
|
||
`pg_dump`, `pg_dumpall`, or `mariadb-dump`.
|
||
|
||
## 💾 Running a Backup
|
||
|
||
```bash
|
||
baudolo \
|
||
--compose-dir /srv/docker \
|
||
--databases-csv /etc/baudolo/databases.csv \
|
||
--database-containers central-postgres central-mariadb \
|
||
--images-no-stop-required alpine postgres mariadb mysql \
|
||
--images-no-backup-required redis busybox
|
||
```
|
||
|
||
### Common Backup Flags
|
||
|
||
| Flag | Description |
|
||
| --------------- | ------------------------------------------- |
|
||
| `--everything` | Always stop containers and re-run rsync |
|
||
| `--dump-only` | Only create SQL dumps, skip file backups |
|
||
| `--shutdown` | Do not restart containers after backup |
|
||
| `--backups-dir` | Backup root directory (default: `/Backups`) |
|
||
| `--repo-name` | Backup namespace under machine hash |
|
||
|
||
## ♻️ Restore Operations
|
||
|
||
### Restore Volume Files
|
||
|
||
```bash
|
||
baudolo-restore files \
|
||
my-volume \
|
||
<machine-hash> \
|
||
<version> \
|
||
--backups-dir /Backups \
|
||
--repo-name my-repo
|
||
```
|
||
|
||
Restore into a **different target volume**:
|
||
|
||
```bash
|
||
baudolo-restore files \
|
||
target-volume \
|
||
<machine-hash> \
|
||
<version> \
|
||
--source-volume source-volume
|
||
```
|
||
|
||
### Restore PostgreSQL
|
||
|
||
```bash
|
||
baudolo-restore postgres \
|
||
my-volume \
|
||
<machine-hash> \
|
||
<version> \
|
||
--container postgres \
|
||
--db-name appdb \
|
||
--db-password secret \
|
||
--empty
|
||
```
|
||
|
||
### Restore MariaDB / MySQL
|
||
|
||
```bash
|
||
baudolo-restore mariadb \
|
||
my-volume \
|
||
<machine-hash> \
|
||
<version> \
|
||
--container mariadb \
|
||
--db-name shopdb \
|
||
--db-password secret \
|
||
--empty
|
||
```
|
||
|
||
> `baudolo` automatically detects whether `mariadb` or `mysql`
|
||
> is available inside the container
|
||
|
||
## 🔍 Backup Scheme
|
||
|
||
The backup mechanism uses incremental backups with rsync and stamps directories with a unique hash. For more details on the backup scheme, check out [this blog post](https://blog.veen.world/blog/2020/12/26/how-i-backup-dedicated-root-servers/).
|
||
|
||
|
||
## 👨💻 Author
|
||
|
||
**Kevin Veen-Birkenbach**
|
||
- 📧 [kevin@veen.world](mailto:kevin@veen.world)
|
||
- 🌐 [https://www.veen.world/](https://www.veen.world/)
|
||
|
||
## 📜 License
|
||
|
||
This project is licensed under the **GNU Affero General Public License v3.0**. See the [LICENSE](./LICENSE) file for details.
|
||
|
||
## 🔗 More Information
|
||
|
||
- [Docker Volumes Documentation](https://docs.docker.com/storage/volumes/)
|
||
- [Docker Backup Volumes Blog](https://blog.ssdnodes.com/blog/docker-backup-volumes/)
|
||
- [Backup Strategies](https://en.wikipedia.org/wiki/Incremental_backup#Incremental)
|
||
|
||
---
|
||
|
||
Happy Backing Up! 🚀🔐
|