mirror of
https://github.com/kevinveenbirkenbach/docker-volume-backup-cleanup.git
synced 2025-09-16 15:16:05 +02:00
Add CI, dirval-based validator, tests, and docs updates
• Add GitHub Actions workflow (Ubuntu, Python 3.10–3.12) • Add Makefile (test + pkgmgr install note) • Add requirements.yml (pkgmgr: dirval) • Replace shell scripts with parallel validator main.py using dirval • Add unit tests with fake dirval and timeouts • Update README to new repo name and pkgmgr alias 'cleanback' • Add .gitignore for __pycache__ Conversation context: https://chatgpt.com/share/68c309bf-8818-800f-84d9-c4aa74a4544c
This commit is contained in:
115
README.md
115
README.md
@@ -1,32 +1,113 @@
|
||||
# Cleanup Failed Docker Backups
|
||||
[](https://github.com/sponsors/kevinveenbirkenbach) [](https://www.patreon.com/c/kevinveenbirkenbach) [](https://buymeacoffee.com/kevinveenbirkenbach) [](https://s.veen.world/paypaldonate)
|
||||
# Cleanup Failed Backups (cleanback) 🚮⚡
|
||||
|
||||
[](https://github.com/sponsors/kevinveenbirkenbach)
|
||||
[](https://www.patreon.com/c/kevinveenbirkenbach)
|
||||
[](https://buymeacoffee.com/kevinveenbirkenbach)
|
||||
[](https://s.veen.world/paypaldonate)
|
||||
|
||||
This repository hosts a Bash script designed for cleaning up directories within the Docker Volume Backup system. It is intended to be used in conjunction with the [Docker Volume Backup](https://github.com/kevinveenbirkenbach/docker-volume-backup) project.
|
||||
**Repository:** https://github.com/kevinveenbirkenbach/cleanup-failed-backups
|
||||
|
||||
## Description
|
||||
This tool validates and (optionally) cleans up **failed Docker backup directories**.
|
||||
It scans backup folders under `/Backups`, uses [`dirval`](https://github.com/kevinveenbirkenbach/directory-validator) to validate each subdirectory, and lets you delete the ones that fail validation. Validation runs **in parallel** for performance; deletions are controlled and can be interactive or automatic.
|
||||
|
||||
This script operates by traversing subdirectories within a specific main directory and, under certain conditions, proposes their deletion to the user. It is useful in managing backup directories, especially when certain directories can be cleaned up based on the absence of a particular subdirectory and the name of the directories themselves.
|
||||
---
|
||||
|
||||
The script takes two arguments: a backup hash and a trigger directory. It constructs the main directory path using the given backup hash, and then iterates over all items within the main directory. If a directory's name matches a specific date-time-stamp pattern and lacks the specified trigger directory, the script will list its contents and ask for user confirmation to delete the directory.
|
||||
## ✨ Highlights
|
||||
|
||||
For more detailed information about the script's workings, refer to the comments within the `cleanup.sh` script file.
|
||||
- **Parallel validation** of backup subdirectories
|
||||
- Uses **`dirval`** (`directory-validator`) via CLI for robust validation
|
||||
- **Interactive** or **non-interactive** deletion flow (`--yes`)
|
||||
- Supports validating a single backup **ID** or **all** backups
|
||||
|
||||
## Usage
|
||||
---
|
||||
|
||||
To use this script, clone this repository to your local system and run the script with the necessary arguments. The command should be structured as follows:
|
||||
## 📦 Installation
|
||||
|
||||
This project is installable via **pkgmgr** (Kevin’s package manager).
|
||||
|
||||
**New pkgmgr alias:** `cleanback`
|
||||
|
||||
```bash
|
||||
bash cleanup.sh BACKUP_HASH
|
||||
# Install pkgmgr first (if you don't have it):
|
||||
# https://github.com/kevinveenbirkenbach/package-manager
|
||||
|
||||
pkgmgr install cleanback
|
||||
````
|
||||
|
||||
> `dirval` is declared as a dependency (see `requirements.yml`) and will be resolved by pkgmgr.
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Requirements
|
||||
|
||||
* Python 3.8+
|
||||
* `dirval` available on PATH (resolved automatically by `pkgmgr install cleanback`)
|
||||
* Access to `/Backups` directory tree
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Usage
|
||||
|
||||
The executable is `main.py`:
|
||||
|
||||
```bash
|
||||
# Validate a single backup ID (under /Backups/<ID>/backup-docker-to-local)
|
||||
python3 main.py --id <ID>
|
||||
|
||||
# Validate ALL backup IDs under /Backups/*/backup-docker-to-local
|
||||
python3 main.py --all
|
||||
```
|
||||
|
||||
Replace ```BACKUP_HASH``` and ```TRIGGER_DIRECTORY``` with your actual values.
|
||||
### Common options
|
||||
|
||||
## License
|
||||
This project is licensed under the GNU Affero General Public License v3.0. See the LICENSE file for more information.
|
||||
* `--dirval-cmd <path-or-name>` — command to run `dirval` (default: `dirval`)
|
||||
* `--workers <int>` — parallel workers (default: CPU count, min 2)
|
||||
* `--timeout <seconds>` — per-directory validation timeout (float supported; default: 300.0)
|
||||
* `--yes` — **non-interactive**: auto-delete directories that fail validation
|
||||
|
||||
## Author
|
||||
This script is developed by Kevin Veen-Birkenbach. You can reach out to him at kevin@veen.world or visit his website at https://www.veen.world.
|
||||
### Examples
|
||||
|
||||
## Created with Chat GPT
|
||||
https://chat.openai.com/share/01222e15-8e1d-436d-b05b-29f406adb2ea
|
||||
```bash
|
||||
# Validate a single backup and prompt for deletions on failures
|
||||
python3 main.py --id 2024-09-01T12-00-00
|
||||
|
||||
# Validate everything with 8 workers and auto-delete failures
|
||||
python3 main.py --all --workers 8 --yes
|
||||
|
||||
# Use a custom dirval binary and shorter timeout
|
||||
python3 main.py --all --dirval-cmd /usr/local/bin/dirval --timeout 5.0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Tests
|
||||
|
||||
```bash
|
||||
make test
|
||||
```
|
||||
|
||||
This runs the unit tests in `test.py`. Tests create a temporary `/Backups`-like tree and a fake `dirval` to simulate success/failure/timeout behavior.
|
||||
|
||||
---
|
||||
|
||||
## 📁 Project Layout
|
||||
|
||||
* `main.py` — CLI entry point (parallel validator + cleanup)
|
||||
* `test.py` — unit tests
|
||||
* `requirements.yml` — `pkgmgr` dependencies (includes `dirval`)
|
||||
* `Makefile` — `make test` and an informational `make install`
|
||||
|
||||
---
|
||||
|
||||
## 🪪 License
|
||||
|
||||
This project is licensed under the **GNU Affero General Public License v3.0**.
|
||||
See the [LICENSE](LICENSE) file for details.
|
||||
|
||||
---
|
||||
|
||||
## 👤 Author
|
||||
|
||||
**Kevin Veen-Birkenbach**
|
||||
🌐 [https://www.veen.world](https://www.veen.world)
|
||||
📧 [kevin@veen.world](mailto:kevin@veen.world)
|
||||
|
||||
Reference in New Issue
Block a user