mirror of
https://github.com/kevinveenbirkenbach/duplicate-file-handler.git
synced 2025-04-28 19:00:24 +02:00
Compare commits
No commits in common. "2b06bdf6c643598abd4ba5abbec2d886a46da01b" and "89e15dd023aee82190bacaadc337c282b91f5357" have entirely different histories.
2b06bdf6c6
...
89e15dd023
95
README.md
95
README.md
@ -1,94 +1,55 @@
|
||||
# Duplicate File Handler (dufiha) 🔍
|
||||
# Duplicate File Handler
|
||||
|
||||
[](./LICENSE) [](https://github.com/kevinveenbirkenbach/duplicate-file-handler/stargazers)
|
||||
This repository contains a Python script for identifying and handling duplicate files in a directory and its subdirectories based on their MD5 hash. It allows for filtering by file type and provides options for handling duplicates such as deletion, hard linking, or sym linking.
|
||||
|
||||
Duplicate File Handler is a Python CLI tool for identifying and handling duplicate files within one or more directories based on their MD5 hashes. With flexible file-type filtering and multiple action modes, you can efficiently delete duplicates or replace them with hard or symbolic links.
|
||||
## Author
|
||||
- Kevin Veen-Birkenbach
|
||||
- Email: kevin@veen.world
|
||||
- Website: [https://www.veen.world](https://www.veen.world)
|
||||
|
||||
---
|
||||
This repository was enhanced with the help of [OpenAI's ChatGPT](https://chat.openai.com/share/825931d6-1e33-40b0-8dfc-914b3f852eeb).
|
||||
|
||||
## 🛠 Features
|
||||
## Setup
|
||||
To use the script, ensure you have Python installed on your system. No additional libraries are required as the script uses standard Python libraries.
|
||||
|
||||
- **Duplicate Detection:** Computes MD5 hashes for files to find duplicates.
|
||||
- **File Type Filtering:** Process only files with a specified extension.
|
||||
- **Multiple Modification Options:** Choose to delete duplicates, replace them with hard links, or create symbolic links.
|
||||
- **Flexible Modes:** Operate in preview, interactive, or active mode to suit your workflow.
|
||||
- **Parallel Processing:** Utilizes process pooling for efficient scanning of large directories.
|
||||
## Usage
|
||||
|
||||
---
|
||||
### Identifying and Handling Duplicates
|
||||
|
||||
## 📥 Installation
|
||||
|
||||
Install Duplicate File Handler via [Kevin's Package Manager](https://github.com/kevinveenbirkenbach/package-manager) under the alias `dufiha`:
|
||||
`main.py` is a Python script to identify all duplicate files in the specified directories. It can also filter by file type and handle duplicates by deleting them or replacing them with hard or symbolic links.
|
||||
|
||||
```bash
|
||||
package-manager install dufiha
|
||||
python main.py [options] directories
|
||||
```
|
||||
|
||||
This command installs the tool globally, making it available as `dufiha` in your terminal. 🚀
|
||||
#### Options
|
||||
- `--apply-to`: Directories to apply modifications to.
|
||||
- `--modification`: Action to perform on duplicates - `delete`, `hardlink`, `symlink`, or `show` (default).
|
||||
- `--mode`: How to apply the modifications - `act`, `preview`, `interactive` (default: `preview`).
|
||||
- `-f`, `--file-type`: Filter by file type (e.g., `.txt` for text files).
|
||||
|
||||
---
|
||||
### Creating Test File Structure
|
||||
|
||||
## 🚀 Usage
|
||||
|
||||
Run Duplicate File Handler by specifying one or more directories to scan for duplicates:
|
||||
`create_file_structure.py` is a utility script to create a test file structure with duplicate files for testing purposes.
|
||||
|
||||
```bash
|
||||
dufiha [options] directory1 directory2 ...
|
||||
python create_file_structure.py
|
||||
```
|
||||
|
||||
### Options
|
||||
## Example
|
||||
|
||||
- **`--apply-to`**: Directories to which modifications should be applied.
|
||||
- **`--modification`**: Action to perform on duplicates:
|
||||
- `delete` – Delete duplicate files.
|
||||
- `hardlink` – Replace duplicates with hard links.
|
||||
- `symlink` – Replace duplicates with symbolic links.
|
||||
- `show` – Only display duplicate files (default).
|
||||
- **`--mode`**: How to apply modifications:
|
||||
- `act` – Execute changes immediately.
|
||||
- `preview` – Preview changes without making any modifications.
|
||||
- `interactive` – Ask for confirmation before processing each duplicate.
|
||||
- **`-f, --file-type`**: Filter by file type (e.g., `.txt` for text files).
|
||||
|
||||
### Example Commands
|
||||
|
||||
- **Preview duplicate `.txt` files in two directories:**
|
||||
To preview duplicate `.txt` files in `test_dir1` and `test_dir2`:
|
||||
|
||||
```bash
|
||||
dufiha --file-type .txt --mode preview test_dir1 test_dir2
|
||||
python main.py --file-type .txt --mode preview test_dir1 test_dir2
|
||||
```
|
||||
|
||||
- **Interactively delete duplicates in a specific directory:**
|
||||
To interactively delete duplicates in `test_dir2`:
|
||||
|
||||
```bash
|
||||
dufiha --apply-to test_dir2 --modification delete --mode interactive test_dir1 test_dir2
|
||||
python main.py --apply-to test_dir2 --modification delete --mode interactive test_dir1 test_dir2
|
||||
```
|
||||
|
||||
- **Show duplicates without modifying any files:**
|
||||
## License
|
||||
|
||||
```bash
|
||||
dufiha --mode show test_dir1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🧑💻 Author
|
||||
|
||||
Developed by **Kevin Veen-Birkenbach**
|
||||
- 📧 [kevin@veen.world](mailto:kevin@veen.world)
|
||||
- 🌐 [https://www.veen.world](https://www.veen.world)
|
||||
|
||||
This project was enhanced with assistance from [OpenAI's ChatGPT](https://chat.openai.com/share/825931d6-1e33-40b0-8dfc-914b3f852eeb).
|
||||
|
||||
---
|
||||
|
||||
## 📜 License
|
||||
|
||||
This project is licensed under the **GNU Affero General Public License, Version 3, 19 November 2007**.
|
||||
See the [LICENSE](./LICENSE) file for details.
|
||||
|
||||
---
|
||||
|
||||
## 🤝 Contributions
|
||||
|
||||
Contributions are welcome! Please feel free to fork the repository, submit pull requests, or open issues to help improve Duplicate File Handler. Let’s make file management smarter and more efficient! 😊
|
||||
This project is licensed under the terms of the [MIT License](LICENSE).
|
Loading…
x
Reference in New Issue
Block a user