Compare commits

...

12 Commits

8 changed files with 498 additions and 112 deletions

7
.github/FUNDING.yml vendored Normal file
View File

@@ -0,0 +1,7 @@
github: kevinveenbirkenbach
patreon: kevinveenbirkenbach
buy_me_a_coffee: kevinveenbirkenbach
custom: https://s.veen.world/paypaldonate

1
.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
*__pycache__

4
Makefile Normal file
View File

@@ -0,0 +1,4 @@
.PHONY: test
test:
python3 -m unittest test.py

View File

@@ -1,5 +1,7 @@
# Bulk String Replacer CLI (bsr) 🔄
[![GitHub Sponsors](https://img.shields.io/badge/Sponsor-GitHub%20Sponsors-blue?logo=github)](https://github.com/sponsors/kevinveenbirkenbach) [![Patreon](https://img.shields.io/badge/Support-Patreon-orange?logo=patreon)](https://www.patreon.com/c/kevinveenbirkenbach) [![Buy Me a Coffee](https://img.shields.io/badge/Buy%20me%20a%20Coffee-Funding-yellow?logo=buymeacoffee)](https://buymeacoffee.com/kevinveenbirkenbach) [![PayPal](https://img.shields.io/badge/Donate-PayPal-blue?logo=paypal)](https://s.veen.world/paypaldonate)
[![License: AGPL v3](https://img.shields.io/badge/License-AGPL%20v3-blue.svg)](https://www.gnu.org/licenses/agpl-3.0.en.html) [![GitHub stars](https://img.shields.io/github/stars/kevinveenbirkenbach/bulk-string-replacer.svg?style=social)](https://github.com/kevinveenbirkenbach/bulk-string-replacer/stargazers)
Bulk String Replacer CLI (bsr) is a powerful Python-based command-line tool that lets you search and replace strings in file names, folder names, and within file contents across multiple directories. Perfect for performing bulk updates quickly and efficiently, bsr supports recursive traversal, hidden files, and a preview mode so you can review changes before theyre applied. 🔧📂
@@ -8,11 +10,12 @@ Bulk String Replacer CLI (bsr) is a powerful Python-based command-line tool that
## 🛠 Features
- **Comprehensive Replacement:** Replace strings in folder names, file names, and inside file contents.
- **Recursive Processing:** Traverse directories recursively to update all matching files.
- **Hidden Files Support:** Option to include hidden files and directories.
- **Preview Mode:** Preview changes without modifying any files.
- **Verbose Output:** Display detailed logs of the operations performed.
* **Comprehensive Replacement:** Replace strings in folder names, file names, and inside file contents.
* **Recursive Processing:** Traverse directories recursively to update all matching files.
* **Hidden Files Support:** Option to include hidden files and directories.
* **Preview Mode:** Preview changes without modifying any files.
* **Verbose Output:** Display detailed logs of the operations performed.
* **Full-Path Moves:** Match an `old_string` as a relative path (including `/`) and move matching subtrees to a new location.
---
@@ -21,6 +24,7 @@ Bulk String Replacer CLI (bsr) is a powerful Python-based command-line tool that
Install Bulk String Replacer CLI easily via [Kevin's Package Manager](https://github.com/kevinveenbirkenbach/package-manager) under the alias `bsr`:
```bash
package-manager clone bsr
package-manager install bsr
```
@@ -33,36 +37,50 @@ This command makes the tool globally available as `bsr` in your terminal. 🚀
Once installed, run Bulk String Replacer CLI using the alias:
```bash
bsr old_string --new-string "replacement_value" [options] [paths...]
bsr old_string -n "replacement_value" [options] [paths...]
```
### Options
- **`old_string`**: The string to search for and replace.
- **`--new-string`**: The string that will replace `old_string` (default is an empty string).
- **`--recursive`**: Process all subdirectories and files recursively.
- **`--folder`**: Replace occurrences within folder names.
- **`--files`**: Replace occurrences within file names.
- **`--content`**: Replace occurrences inside file contents.
- **`--preview`**: Preview changes without applying them.
- **`--verbose`**: Display detailed logs during execution.
- **`--hidden`**: Include hidden files and directories in the operation.
* **`old_string`**: The string or relative path to search for.
* **`-n, --new`**: The replacement string or new relative path (default is empty string).
* **`-r, --recursive`**: Recurse into all subdirectories and files.
* **`-F, --folders`**: Replace occurrences within folder names.
* **`-f, --files`**: Replace occurrences within file names.
* **`-c, --content`**: Replace occurrences inside file contents.
* **`-P, --path`**: Match `old_string` as a relative path (e.g. `vars/config.yml`) and move matching subtree to `new` relative path.
* **`-p, --preview`**: Preview changes without applying them.
* **`-v, --verbose`**: Display detailed logs during execution.
* **`-H, --hidden`**: Include hidden files and directories in the operation.
### Example Command
### Examples
Replace text within filenames, folder names, and file contents:
```bash
bsr "old_value" --new-string "new_value" --recursive --verbose /path/to/first/directory /path/to/second/directory
bsr "old_value" -n "new_value" -r -F -f -c /path/to/dir
```
Replace `/path/to/first/directory` and `/path/to/second/directory` with the paths you wish to process.
Move every `vars/configuration.yml` to `config/main.yml` in each parent directory:
```bash
bsr "vars/configuration.yml" -n "config/main.yml" -r -P ./
```
Preview a full-path move without changes:
```bash
bsr "vars/configuration.yml" -n "config/main.yml" -r -P -p ./
```
---
## 🧑‍💻 Author
Developed by **Kevin Veen-Birkenbach**
- 📧 [kevin@veen.world](mailto:kevin@veen.world)
- 🌐 [https://www.veen.world/](https://www.veen.world/)
Developed by **Kevin Veen-Birkenbach**
* 📧 [kevin@veen.world](mailto:kevin@veen.world)
* 🌐 [https://www.veen.world/](https://www.veen.world/)
Learn more about the development of this tool in the [original ChatGPT conversation](https://chat.openai.com/share/cfdbc008-8374-47f8-8853-2e00ee27c959).
@@ -70,7 +88,7 @@ Learn more about the development of this tool in the [original ChatGPT conversat
## 📜 License
This project is licensed under the **GNU Affero General Public License, Version 3, 19 November 2007**.
This project is licensed under the **GNU Affero General Public License, Version 3, 19 November 2007**.
For more details, see the [LICENSE](./LICENSE) file.
---

0
__init__.py Normal file
View File

234
main.py Executable file
View File

@@ -0,0 +1,234 @@
#!/usr/bin/env python3
import os
import argparse
def replace_content(path, old_string, new_string, preview, verbose):
"""
Replace occurrences of old_string with new_string inside the file at path.
"""
try:
with open(path, 'r', encoding='utf-8') as f:
content = f.read()
if old_string in content:
new_content = content.replace(old_string, new_string)
print_verbose(f"Replacing content in: {path}", verbose)
if not preview:
with open(path, 'w', encoding='utf-8') as fw:
fw.write(new_content)
except UnicodeDecodeError:
print_verbose(f"Warning: Unicode decode error in file {path}. Skipping.", verbose)
def print_verbose(message, verbose):
if verbose:
print(message)
def process_directory(
base_path,
old_string,
new_string,
recursive,
rename_folders,
rename_files,
replace_in_content,
preview,
verbose,
include_hidden,
rename_paths,
auto_path
):
"""
Traverse the directory tree and perform operations based on flags:
- replace_in_content: replace inside file contents
- rename_files: rename files whose names contain old_string
- rename_folders: rename folders whose names contain old_string
- rename_paths: treat old_string as a relative path and move matching items to new_string path.
When rename_paths is set, prompts occur for Python files unless auto_path is True.
Additionally, for test_*.py files when auto_path is False, prompts to extract 'test_' prefix into a subdirectory.
"""
if rename_paths:
# Full-path moves and optional test_ prefix extraction
for root, dirs, files in os.walk(base_path):
if not include_hidden:
dirs[:] = [d for d in dirs if not d.startswith('.')]
files = [f for f in files if not f.startswith('.')]
for name in files + dirs:
full_src = os.path.join(root, name)
rel = os.path.relpath(full_src, base_path)
if old_string in rel:
new_rel = rel.replace(old_string, new_string)
# Special handling for test_*.py when auto_path is False
if (
os.path.isfile(full_src)
and name.startswith('test_')
and name.endswith('.py')
and not auto_path
):
choice = None
while choice not in ('y', 'n'):
choice = input(
f"File {full_src} is a test module. "
"Extract 'test_' prefix into subdirectory? [y/N]: "
).strip().lower() or 'n'
if choice == 'y':
rel_dir = os.path.dirname(rel)
orig_basename = os.path.basename(rel)
base_noext = orig_basename[:-3] # remove .py
subdir_name = base_noext[len('test_'):]
new_name_part = os.path.basename(new_string)
new_filename = f"test_{new_name_part}.py"
new_rel = os.path.join(rel_dir, subdir_name, new_filename)
full_dst = os.path.join(base_path, new_rel)
print_verbose(f"Moving {full_src}{full_dst}", verbose)
if not preview:
os.makedirs(os.path.dirname(full_dst), exist_ok=True)
os.rename(full_src, full_dst)
if not recursive:
break
# Line-by-line content replacement in Python files
for root, dirs, files in os.walk(base_path):
if not include_hidden:
dirs[:] = [d for d in dirs if not d.startswith('.')]
files = [f for f in files if not f.startswith('.')]
for f in files:
if f.endswith('.py'):
file_path = os.path.join(root, f)
print_verbose(f"Processing Python file: {file_path}", verbose)
try:
with open(file_path, 'r', encoding='utf-8') as ff:
lines = ff.readlines()
except UnicodeDecodeError:
print_verbose(f"Warning: Unicode decode error in file {file_path}. Skipping.", verbose)
continue
old_slash = old_string.replace('/', os.sep)
new_dot = new_string.replace('/', '.')
changed = False
for idx, line in enumerate(lines):
if old_slash in line:
start = max(0, idx - 3)
end = min(len(lines), idx + 4)
print(f"\nContext in {file_path}, line {idx+1}:")
for i in range(start, end):
prefix = '>' if i == idx else ' '
print(f"{prefix} {i+1}: {lines[i].rstrip()}")
if auto_path:
choice = '1'
else:
choice = None
while choice not in ('1', '2'):
choice = input(
f"Replace this line:\n"
f" 1) slash-based: '{old_slash}''{new_string}'\n"
f" 2) dot-based: '{old_slash}''{new_dot}'\n"
f"Choose [1/2]: "
).strip()
if choice == '1':
lines[idx] = line.replace(old_slash, new_string)
else:
lines[idx] = line.replace(old_slash, new_dot)
changed = True
print_verbose(f"Replaced line {idx+1} in {file_path}", verbose)
if changed and not preview:
with open(file_path, 'w', encoding='utf-8') as fw:
fw.writelines(lines)
if not recursive:
break
# Exit early if only path-mode is requested
if not (rename_files or replace_in_content):
return
# Fallback operations: replace content, rename files, rename folders
folders_to_rename = []
for root, dirs, files in os.walk(base_path):
if not include_hidden:
dirs[:] = [d for d in dirs if not d.startswith('.')]
files = [f for f in files if not f.startswith('.')]
if replace_in_content:
for f in files:
replace_content(os.path.join(root, f), old_string, new_string, preview, verbose)
if rename_files:
for f in files:
if old_string in f:
src = os.path.join(root, f)
dst = os.path.join(root, f.replace(old_string, new_string))
print_verbose(f"Renaming file: {src}{dst}", verbose)
if not preview:
os.rename(src, dst)
if rename_folders:
for d in dirs:
if old_string in d:
src = os.path.join(root, d)
dst = os.path.join(root, d.replace(old_string, new_string))
folders_to_rename.append((src, dst))
if not recursive:
break
for src, dst in folders_to_rename:
print_verbose(f"Renaming directory: {src}{dst}", verbose)
if not preview:
os.rename(src, dst)
def main():
parser = argparse.ArgumentParser(
description="Bulk string replacer with optional full-path moves."
)
parser.add_argument('paths', nargs='+', help="Base directories to process.")
parser.add_argument('old_string', help="String or relative path to replace.")
parser.add_argument('-n', '--new', dest='new_string', default='',
help="Replacement string or new relative path.")
parser.add_argument('-r', '--recursive', action='store_true',
help="Recurse into subdirectories.")
parser.add_argument('-F', '--folders', action='store_true',
help="Rename folder names.")
parser.add_argument('-f', '--files', action='store_true',
help="Rename file names.")
parser.add_argument('-c', '--content', action='store_true',
help="Replace inside file contents.")
parser.add_argument('-p', '--preview', action='store_true',
help="Preview only; no changes.")
parser.add_argument('-P', '--path', dest='rename_paths', action='store_true',
help="Treat old_string as path and move matching items.")
parser.add_argument('-y', '--yes', dest='auto_path', action='store_true',
help="Skip prompts and apply default replacements.")
parser.add_argument('-v', '--verbose', action='store_true',
help="Verbose mode.")
parser.add_argument('-H', '--hidden', action='store_true',
help="Include hidden files and folders.")
args = parser.parse_args()
if args.rename_paths and args.folders:
parser.error("Cannot use --path and --folders together.")
base_paths = [os.path.expanduser(p) for p in args.paths]
for base in base_paths:
print_verbose(f"Processing: {base}", args.verbose)
process_directory(
base_path=base,
old_string=args.old_string,
new_string=args.new_string,
recursive=args.recursive,
rename_folders=args.folders,
rename_files=args.files,
replace_in_content=args.content,
preview=args.preview,
verbose=args.verbose,
include_hidden=args.hidden,
rename_paths=args.rename_paths,
auto_path=args.auto_path
)
if __name__ == '__main__':
main()

View File

@@ -1,90 +0,0 @@
import os
import argparse
def replace_content(path, old_string, new_string, preview, verbose):
try:
with open(path, 'r', encoding='utf-8') as f:
content = f.read()
if old_string in content:
new_content = content.replace(old_string, new_string)
print_verbose(f"Replacing content in: {path}", verbose)
if not preview:
with open(path, 'w', encoding='utf-8') as f:
f.write(new_content)
except UnicodeDecodeError as e:
print_verbose(f"Warning: Unicode decode error encountered in file {path}. Skipping file.", verbose)
def print_verbose(content,verbose):
if verbose:
print(content)
def process_directory(base_path, old_string, new_string, recursive, folder, files, content, preview, verbose, hidden):
# Eine Liste, um die Pfade der umzubenennenden Ordner zu speichern
directories_to_rename = []
for root, dirs, filenames in os.walk(base_path):
# Wenn "hidden" nicht gesetzt ist, versteckte Dateien/Ordner aus der Liste entfernen
if not hidden:
dirs[:] = [d for d in dirs if not d.startswith(".")]
filenames = [f for f in filenames if not f.startswith(".")]
if content:
for f in filenames:
replace_content(os.path.join(root, f), old_string, new_string, preview, verbose)
if files:
for f in filenames:
if old_string in f:
old_path = os.path.join(root, f)
new_path = os.path.join(root, f.replace(old_string, new_string))
print_verbose(f"Renaming file from: {old_path} to: {new_path}",verbose)
if not preview:
os.rename(old_path, new_path)
# Pfade von zu ändernden Ordnern speichern
if folder:
for d in dirs:
if old_string in d:
old_path = os.path.join(root, d)
new_path = os.path.join(root, d.replace(old_string, new_string))
directories_to_rename.append((old_path, new_path))
if not recursive:
break
# Ordnernamen ändern nach dem os.walk() Durchlauf
for old_path, new_path in directories_to_rename:
print_verbose(f"Renaming directory from: {old_path} to: {new_path}",verbose)
if not preview:
os.rename(old_path, new_path)
def main():
parser = argparse.ArgumentParser(description="Replace strings in directories and files.")
parser.add_argument('paths', nargs='+', help="Paths in which replacements should be made.")
parser.add_argument('old_string', help="The string to be replaced.")
parser.add_argument('--new-string', dest='new_string', default="", help="The string to replace with. Default is empty string.")
parser.add_argument('--recursive', action='store_true', help="Replace in all subdirectories and files.")
parser.add_argument('--folder', action='store_true', help="Replace in folder names.")
parser.add_argument('--files', action='store_true', help="Replace in file names.")
parser.add_argument('--content', action='store_true', help="Replace inside file contents.")
parser.add_argument('--preview', action='store_true', help="Preview changes without replacing.")
parser.add_argument('--verbose', action='store_true', help="Verbose mode.")
parser.add_argument('--hidden', action='store_true', help="Apply to hidden files and folders.")
args = parser.parse_args()
# Use os.path.expanduser to expand the tilde to the home directory
expanded_paths = [os.path.expanduser(path) for path in args.paths]
for path in expanded_paths:
print_verbose(f"Replacing in path: {path}",args.verbose)
process_directory(path, args.old_string, args.new_string, args.recursive, args.folder, args.files, args.content, args.preview, args.verbose, args.hidden)
if __name__ == "__main__":
main()

212
test.py Normal file
View File

@@ -0,0 +1,212 @@
#!/usr/bin/env python3
import os
import shutil
import tempfile
import unittest
import argparse
from main import process_directory, replace_content, main as cli_main
class TestBulkStringReplacer(unittest.TestCase):
def setUp(self):
# create isolated temp dir
self.base = tempfile.mkdtemp()
def tearDown(self):
shutil.rmtree(self.base)
def create_file(self, relpath, content=''):
full = os.path.join(self.base, relpath)
os.makedirs(os.path.dirname(full), exist_ok=True)
with open(full, 'w', encoding='utf-8') as f:
f.write(content)
return full
def test_test_prefix_extraction_prompt_no_auto(self):
# Setup a test file with test_ prefix
src = self.create_file('dir/test_module.py', 'print("hello")')
# Monkeypatch input to simulate 'y'
inputs = iter(['y'])
original_input = __builtins__['input']
__builtins__['input'] = lambda _: next(inputs)
try:
process_directory(
base_path=self.base,
old_string='module',
new_string='module_new',
recursive=True,
rename_folders=False,
rename_files=False,
replace_in_content=False,
preview=False,
verbose=False,
include_hidden=True,
rename_paths=True,
auto_path=False
)
finally:
__builtins__['input'] = original_input
# After extraction, expect dir/module/test_module_new.py
dst = os.path.join(self.base, 'dir', 'module', 'test_module_new.py')
self.assertTrue(os.path.exists(dst))
def test_test_prefix_default_no_extraction_auto(self):
# Setup a test file with test_ prefix
src = self.create_file('dir/test_file.py', 'print("hello")')
process_directory(
base_path=self.base,
old_string='file',
new_string='file_new',
recursive=True,
rename_folders=False,
rename_files=False,
replace_in_content=False,
preview=False,
verbose=False,
include_hidden=True,
rename_paths=True,
auto_path=True
)
# With auto_path, should do simple path replace: dir/test_file.py -> dir/test_file_new.py
dst = os.path.join(self.base, 'dir', 'test_file_new.py')
self.assertTrue(os.path.exists(dst))
def test_replace_content(self):
f = self.create_file('foo.txt', 'hello OLD world')
replace_content(f, 'OLD', 'NEW', preview=False, verbose=False)
with open(f, 'r', encoding='utf-8') as fp:
self.assertIn('hello NEW world', fp.read())
def test_rename_file(self):
f = self.create_file('OLDfile.txt', '')
process_directory(
base_path=self.base,
old_string='OLD', new_string='NEW',
recursive=False,
rename_folders=False, rename_files=True,
replace_in_content=False,
preview=False, verbose=False,
include_hidden=True, rename_paths=False, auto_path=True
)
self.assertTrue(os.path.exists(os.path.join(self.base, 'NEWfile.txt')))
self.assertFalse(os.path.exists(f))
def test_rename_folder(self):
os.makedirs(os.path.join(self.base, 'OLDfolder', 'x'))
process_directory(
base_path=self.base,
old_string='OLD', new_string='NEW',
recursive=False,
rename_folders=True, rename_files=False,
replace_in_content=False,
preview=False, verbose=False,
include_hidden=True, rename_paths=False, auto_path=True
)
self.assertTrue(os.path.isdir(os.path.join(self.base, 'NEWfolder')))
self.assertFalse(os.path.isdir(os.path.join(self.base, 'OLDfolder')))
def test_full_path_move(self):
cfg = 'vars/configuration.yml'
full = self.create_file(cfg, 'DATA')
process_directory(
base_path=self.base,
old_string='vars/configuration.yml',
new_string='config/main.yml',
recursive=True,
rename_folders=False, rename_files=False,
replace_in_content=False,
preview=False, verbose=False,
include_hidden=True, rename_paths=True, auto_path=True
)
self.assertFalse(os.path.exists(full))
self.assertTrue(os.path.exists(os.path.join(self.base, 'config', 'main.yml')))
def test_path_and_folders_conflict(self):
parser = argparse.ArgumentParser()
parser.add_argument('-P', dest='rename_paths', action='store_true')
parser.add_argument('-F', dest='folders', action='store_true')
args = parser.parse_args(['-P', '-F'])
with self.assertRaises(SystemExit):
if args.rename_paths and args.folders:
parser.error("Cannot use --path and --folders together.")
def test_preview_does_nothing(self):
f = self.create_file('OLD.txt', 'OLD')
os.makedirs(os.path.join(self.base, 'OLDdir'))
process_directory(
base_path=self.base,
old_string='OLD', new_string='NEW',
recursive=True,
rename_folders=True, rename_files=True,
replace_in_content=True,
preview=True, verbose=False,
include_hidden=True, rename_paths=True, auto_path=True
)
self.assertTrue(os.path.exists(f))
self.assertTrue(os.path.isdir(os.path.join(self.base, 'OLDdir')))
with open(f, 'r', encoding='utf-8') as fp:
self.assertIn('OLD', fp.read())
def test_module_path_replacement_in_python_files(self):
content = 'from old/path import func'
src = self.create_file('old/path/module.py', content)
process_directory(
base_path=self.base,
old_string='old/path', new_string='old.path',
recursive=True,
rename_folders=False, rename_files=False,
replace_in_content=False,
preview=False, verbose=False,
include_hidden=True, rename_paths=True, auto_path=True
)
new_path = os.path.join(self.base, 'old.path', 'module.py')
self.assertTrue(os.path.exists(new_path))
with open(new_path, 'r', encoding='utf-8') as fp:
self.assertIn('from old.path import func', fp.read())
def test_non_python_files_are_not_content_updated(self):
content = 'some reference to old/path in text'
txt = self.create_file('old/path/readme.txt', content)
process_directory(
base_path=self.base,
old_string='old/path', new_string='old.path',
recursive=True,
rename_folders=False, rename_files=False,
replace_in_content=False,
preview=False, verbose=False,
include_hidden=True, rename_paths=True, auto_path=True
)
new_txt = os.path.join(self.base, 'old.path', 'readme.txt')
self.assertTrue(os.path.exists(new_txt))
with open(new_txt, 'r', encoding='utf-8') as fp:
self.assertIn('old/path', fp.read())
def test_auto_path_line_level_replacement(self):
# create a Python file with two occurrences
lines = [
'import a\n',
'path = "old/path/to/module"\n',
'print("old/path/example")\n'
]
py = self.create_file('old/path/test.py', ''.join(lines))
process_directory(
base_path=self.base,
old_string='old/path', new_string='new/path',
recursive=True,
rename_folders=False, rename_files=False,
replace_in_content=False,
preview=False, verbose=False,
include_hidden=True, rename_paths=True, auto_path=True
)
new_py = os.path.join(self.base, 'new', 'path', 'test.py')
self.assertTrue(os.path.exists(new_py))
with open(new_py, 'r', encoding='utf-8') as fp:
content = fp.read()
# all replaced slash-based
self.assertIn('path = "new/path/to/module"', content)
self.assertIn('print("new/path/example")', content)
if __name__ == '__main__':
unittest.main()