Compare commits

...

7 Commits

6 changed files with 452 additions and 76 deletions

1
.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
*__pycache__

4
Makefile Normal file
View File

@@ -0,0 +1,4 @@
.PHONY: test
test:
python3 -m unittest test.py

View File

@@ -1,6 +1,6 @@
# Bulk String Replacer CLI (bsr) 🔄 # Bulk String Replacer CLI (bsr) 🔄
[![GitHub Sponsors](https://img.shields.io/badge/Sponsor-GitHub%20Sponsors-blue?logo=github)](https://github.com/sponsors/kevinveenbirkenbach) [![Patreon](https://img.shields.io/badge/Support-Patreon-orange?logo=patreon)](https://www.patreon.com/c/kevinveenbirkenbach) [![Buy Me a Coffee](https://img.shields.io/badge/Buy%20me%20a%20Coffee-Funding-yellow?logo=buymeacoffee)](https://buymeacoffee.com/kevinveenbirkenbach) [![PayPal](https://img.shields.io/badge/Donate-PayPal-blue?logo=paypal)](https://s.veen.world/paypaldonate)
[![GitHub Sponsors](https://img.shields.io/badge/Sponsor-GitHub%20Sponsors-blue?logo=github)](https://github.com/sponsors/kevinveenbirkenbach) [![Patreon](https://img.shields.io/badge/Support-Patreon-orange?logo=patreon)](https://www.patreon.com/c/kevinveenbirkenbach) [![Buy Me a Coffee](https://img.shields.io/badge/Buy%20me%20a%20Coffee-Funding-yellow?logo=buymeacoffee)](https://buymeacoffee.com/kevinveenbirkenbach) [![PayPal](https://img.shields.io/badge/Donate-PayPal-blue?logo=paypal)](https://s.veen.world/paypaldonate)
[![License: AGPL v3](https://img.shields.io/badge/License-AGPL%20v3-blue.svg)](https://www.gnu.org/licenses/agpl-3.0.en.html) [![GitHub stars](https://img.shields.io/github/stars/kevinveenbirkenbach/bulk-string-replacer.svg?style=social)](https://github.com/kevinveenbirkenbach/bulk-string-replacer/stargazers) [![License: AGPL v3](https://img.shields.io/badge/License-AGPL%20v3-blue.svg)](https://www.gnu.org/licenses/agpl-3.0.en.html) [![GitHub stars](https://img.shields.io/github/stars/kevinveenbirkenbach/bulk-string-replacer.svg?style=social)](https://github.com/kevinveenbirkenbach/bulk-string-replacer/stargazers)
@@ -10,11 +10,12 @@ Bulk String Replacer CLI (bsr) is a powerful Python-based command-line tool that
## 🛠 Features ## 🛠 Features
- **Comprehensive Replacement:** Replace strings in folder names, file names, and inside file contents. * **Comprehensive Replacement:** Replace strings in folder names, file names, and inside file contents.
- **Recursive Processing:** Traverse directories recursively to update all matching files. * **Recursive Processing:** Traverse directories recursively to update all matching files.
- **Hidden Files Support:** Option to include hidden files and directories. * **Hidden Files Support:** Option to include hidden files and directories.
- **Preview Mode:** Preview changes without modifying any files. * **Preview Mode:** Preview changes without modifying any files.
- **Verbose Output:** Display detailed logs of the operations performed. * **Verbose Output:** Display detailed logs of the operations performed.
* **Full-Path Moves:** Match an `old_string` as a relative path (including `/`) and move matching subtrees to a new location.
--- ---
@@ -36,36 +37,50 @@ This command makes the tool globally available as `bsr` in your terminal. 🚀
Once installed, run Bulk String Replacer CLI using the alias: Once installed, run Bulk String Replacer CLI using the alias:
```bash ```bash
bsr old_string --new-string "replacement_value" [options] [paths...] bsr old_string -n "replacement_value" [options] [paths...]
``` ```
### Options ### Options
- **`old_string`**: The string to search for and replace. * **`old_string`**: The string or relative path to search for.
- **`--new-string`**: The string that will replace `old_string` (default is an empty string). * **`-n, --new`**: The replacement string or new relative path (default is empty string).
- **`--recursive`**: Process all subdirectories and files recursively. * **`-r, --recursive`**: Recurse into all subdirectories and files.
- **`--folder`**: Replace occurrences within folder names. * **`-F, --folders`**: Replace occurrences within folder names.
- **`--files`**: Replace occurrences within file names. * **`-f, --files`**: Replace occurrences within file names.
- **`--content`**: Replace occurrences inside file contents. * **`-c, --content`**: Replace occurrences inside file contents.
- **`--preview`**: Preview changes without applying them. * **`-P, --path`**: Match `old_string` as a relative path (e.g. `vars/config.yml`) and move matching subtree to `new` relative path.
- **`--verbose`**: Display detailed logs during execution. * **`-p, --preview`**: Preview changes without applying them.
- **`--hidden`**: Include hidden files and directories in the operation. * **`-v, --verbose`**: Display detailed logs during execution.
* **`-H, --hidden`**: Include hidden files and directories in the operation.
### Example Command ### Examples
Replace text within filenames, folder names, and file contents:
```bash ```bash
bsr "old_value" --new-string "new_value" --recursive --verbose /path/to/first/directory /path/to/second/directory bsr "old_value" -n "new_value" -r -F -f -c /path/to/dir
``` ```
Replace `/path/to/first/directory` and `/path/to/second/directory` with the paths you wish to process. Move every `vars/configuration.yml` to `config/main.yml` in each parent directory:
```bash
bsr "vars/configuration.yml" -n "config/main.yml" -r -P ./
```
Preview a full-path move without changes:
```bash
bsr "vars/configuration.yml" -n "config/main.yml" -r -P -p ./
```
--- ---
## 🧑‍💻 Author ## 🧑‍💻 Author
Developed by **Kevin Veen-Birkenbach** Developed by **Kevin Veen-Birkenbach**
- 📧 [kevin@veen.world](mailto:kevin@veen.world)
- 🌐 [https://www.veen.world/](https://www.veen.world/) * 📧 [kevin@veen.world](mailto:kevin@veen.world)
* 🌐 [https://www.veen.world/](https://www.veen.world/)
Learn more about the development of this tool in the [original ChatGPT conversation](https://chat.openai.com/share/cfdbc008-8374-47f8-8853-2e00ee27c959). Learn more about the development of this tool in the [original ChatGPT conversation](https://chat.openai.com/share/cfdbc008-8374-47f8-8853-2e00ee27c959).

0
__init__.py Normal file
View File

248
main.py Normal file → Executable file
View File

@@ -1,90 +1,234 @@
#!/usr/bin/env python3
import os import os
import argparse import argparse
def replace_content(path, old_string, new_string, preview, verbose): def replace_content(path, old_string, new_string, preview, verbose):
"""
Replace occurrences of old_string with new_string inside the file at path.
"""
try: try:
with open(path, 'r', encoding='utf-8') as f: with open(path, 'r', encoding='utf-8') as f:
content = f.read() content = f.read()
if old_string in content: if old_string in content:
new_content = content.replace(old_string, new_string) new_content = content.replace(old_string, new_string)
print_verbose(f"Replacing content in: {path}", verbose) print_verbose(f"Replacing content in: {path}", verbose)
if not preview: if not preview:
with open(path, 'w', encoding='utf-8') as f: with open(path, 'w', encoding='utf-8') as fw:
f.write(new_content) fw.write(new_content)
except UnicodeDecodeError:
print_verbose(f"Warning: Unicode decode error in file {path}. Skipping.", verbose)
except UnicodeDecodeError as e:
print_verbose(f"Warning: Unicode decode error encountered in file {path}. Skipping file.", verbose)
def print_verbose(content,verbose): def print_verbose(message, verbose):
if verbose: if verbose:
print(content) print(message)
def process_directory(base_path, old_string, new_string, recursive, folder, files, content, preview, verbose, hidden): def process_directory(
# Eine Liste, um die Pfade der umzubenennenden Ordner zu speichern base_path,
directories_to_rename = [] old_string,
new_string,
recursive,
rename_folders,
rename_files,
replace_in_content,
preview,
verbose,
include_hidden,
rename_paths,
auto_path
):
"""
Traverse the directory tree and perform operations based on flags:
- replace_in_content: replace inside file contents
- rename_files: rename files whose names contain old_string
- rename_folders: rename folders whose names contain old_string
- rename_paths: treat old_string as a relative path and move matching items to new_string path.
When rename_paths is set, prompts occur for Python files unless auto_path is True.
Additionally, for test_*.py files when auto_path is False, prompts to extract 'test_' prefix into a subdirectory.
"""
if rename_paths:
# Full-path moves and optional test_ prefix extraction
for root, dirs, files in os.walk(base_path):
if not include_hidden:
dirs[:] = [d for d in dirs if not d.startswith('.')]
files = [f for f in files if not f.startswith('.')]
for name in files + dirs:
full_src = os.path.join(root, name)
rel = os.path.relpath(full_src, base_path)
if old_string in rel:
new_rel = rel.replace(old_string, new_string)
for root, dirs, filenames in os.walk(base_path): # Special handling for test_*.py when auto_path is False
# Wenn "hidden" nicht gesetzt ist, versteckte Dateien/Ordner aus der Liste entfernen if (
if not hidden: os.path.isfile(full_src)
dirs[:] = [d for d in dirs if not d.startswith(".")] and name.startswith('test_')
filenames = [f for f in filenames if not f.startswith(".")] and name.endswith('.py')
and not auto_path
):
choice = None
while choice not in ('y', 'n'):
choice = input(
f"File {full_src} is a test module. "
"Extract 'test_' prefix into subdirectory? [y/N]: "
).strip().lower() or 'n'
if choice == 'y':
rel_dir = os.path.dirname(rel)
orig_basename = os.path.basename(rel)
base_noext = orig_basename[:-3] # remove .py
subdir_name = base_noext[len('test_'):]
new_name_part = os.path.basename(new_string)
new_filename = f"test_{new_name_part}.py"
new_rel = os.path.join(rel_dir, subdir_name, new_filename)
if content: full_dst = os.path.join(base_path, new_rel)
for f in filenames: print_verbose(f"Moving {full_src}{full_dst}", verbose)
if not preview:
os.makedirs(os.path.dirname(full_dst), exist_ok=True)
os.rename(full_src, full_dst)
if not recursive:
break
# Line-by-line content replacement in Python files
for root, dirs, files in os.walk(base_path):
if not include_hidden:
dirs[:] = [d for d in dirs if not d.startswith('.')]
files = [f for f in files if not f.startswith('.')]
for f in files:
if f.endswith('.py'):
file_path = os.path.join(root, f)
print_verbose(f"Processing Python file: {file_path}", verbose)
try:
with open(file_path, 'r', encoding='utf-8') as ff:
lines = ff.readlines()
except UnicodeDecodeError:
print_verbose(f"Warning: Unicode decode error in file {file_path}. Skipping.", verbose)
continue
old_slash = old_string.replace('/', os.sep)
new_dot = new_string.replace('/', '.')
changed = False
for idx, line in enumerate(lines):
if old_slash in line:
start = max(0, idx - 3)
end = min(len(lines), idx + 4)
print(f"\nContext in {file_path}, line {idx+1}:")
for i in range(start, end):
prefix = '>' if i == idx else ' '
print(f"{prefix} {i+1}: {lines[i].rstrip()}")
if auto_path:
choice = '1'
else:
choice = None
while choice not in ('1', '2'):
choice = input(
f"Replace this line:\n"
f" 1) slash-based: '{old_slash}''{new_string}'\n"
f" 2) dot-based: '{old_slash}''{new_dot}'\n"
f"Choose [1/2]: "
).strip()
if choice == '1':
lines[idx] = line.replace(old_slash, new_string)
else:
lines[idx] = line.replace(old_slash, new_dot)
changed = True
print_verbose(f"Replaced line {idx+1} in {file_path}", verbose)
if changed and not preview:
with open(file_path, 'w', encoding='utf-8') as fw:
fw.writelines(lines)
if not recursive:
break
# Exit early if only path-mode is requested
if not (rename_files or replace_in_content):
return
# Fallback operations: replace content, rename files, rename folders
folders_to_rename = []
for root, dirs, files in os.walk(base_path):
if not include_hidden:
dirs[:] = [d for d in dirs if not d.startswith('.')]
files = [f for f in files if not f.startswith('.')]
if replace_in_content:
for f in files:
replace_content(os.path.join(root, f), old_string, new_string, preview, verbose) replace_content(os.path.join(root, f), old_string, new_string, preview, verbose)
if files: if rename_files:
for f in filenames: for f in files:
if old_string in f: if old_string in f:
old_path = os.path.join(root, f) src = os.path.join(root, f)
new_path = os.path.join(root, f.replace(old_string, new_string)) dst = os.path.join(root, f.replace(old_string, new_string))
print_verbose(f"Renaming file from: {old_path} to: {new_path}",verbose) print_verbose(f"Renaming file: {src} {dst}", verbose)
if not preview: if not preview:
os.rename(old_path, new_path) os.rename(src, dst)
# Pfade von zu ändernden Ordnern speichern if rename_folders:
if folder:
for d in dirs: for d in dirs:
if old_string in d: if old_string in d:
old_path = os.path.join(root, d) src = os.path.join(root, d)
new_path = os.path.join(root, d.replace(old_string, new_string)) dst = os.path.join(root, d.replace(old_string, new_string))
directories_to_rename.append((old_path, new_path)) folders_to_rename.append((src, dst))
if not recursive: if not recursive:
break break
# Ordnernamen ändern nach dem os.walk() Durchlauf for src, dst in folders_to_rename:
for old_path, new_path in directories_to_rename: print_verbose(f"Renaming directory: {src}{dst}", verbose)
print_verbose(f"Renaming directory from: {old_path} to: {new_path}",verbose)
if not preview: if not preview:
os.rename(old_path, new_path) os.rename(src, dst)
def main(): def main():
parser = argparse.ArgumentParser(description="Replace strings in directories and files.") parser = argparse.ArgumentParser(
parser.add_argument('paths', nargs='+', help="Paths in which replacements should be made.") description="Bulk string replacer with optional full-path moves."
parser.add_argument('old_string', help="The string to be replaced.") )
parser.add_argument('--new-string', dest='new_string', default="", help="The string to replace with. Default is empty string.") parser.add_argument('paths', nargs='+', help="Base directories to process.")
parser.add_argument('--recursive', action='store_true', help="Replace in all subdirectories and files.") parser.add_argument('old_string', help="String or relative path to replace.")
parser.add_argument('--folder', action='store_true', help="Replace in folder names.") parser.add_argument('-n', '--new', dest='new_string', default='',
parser.add_argument('--files', action='store_true', help="Replace in file names.") help="Replacement string or new relative path.")
parser.add_argument('--content', action='store_true', help="Replace inside file contents.") parser.add_argument('-r', '--recursive', action='store_true',
parser.add_argument('--preview', action='store_true', help="Preview changes without replacing.") help="Recurse into subdirectories.")
parser.add_argument('--verbose', action='store_true', help="Verbose mode.") parser.add_argument('-F', '--folders', action='store_true',
parser.add_argument('--hidden', action='store_true', help="Apply to hidden files and folders.") help="Rename folder names.")
parser.add_argument('-f', '--files', action='store_true',
help="Rename file names.")
parser.add_argument('-c', '--content', action='store_true',
help="Replace inside file contents.")
parser.add_argument('-p', '--preview', action='store_true',
help="Preview only; no changes.")
parser.add_argument('-P', '--path', dest='rename_paths', action='store_true',
help="Treat old_string as path and move matching items.")
parser.add_argument('-y', '--yes', dest='auto_path', action='store_true',
help="Skip prompts and apply default replacements.")
parser.add_argument('-v', '--verbose', action='store_true',
help="Verbose mode.")
parser.add_argument('-H', '--hidden', action='store_true',
help="Include hidden files and folders.")
args = parser.parse_args() args = parser.parse_args()
# Use os.path.expanduser to expand the tilde to the home directory if args.rename_paths and args.folders:
expanded_paths = [os.path.expanduser(path) for path in args.paths] parser.error("Cannot use --path and --folders together.")
for path in expanded_paths: base_paths = [os.path.expanduser(p) for p in args.paths]
print_verbose(f"Replacing in path: {path}",args.verbose) for base in base_paths:
process_directory(path, args.old_string, args.new_string, args.recursive, args.folder, args.files, args.content, args.preview, args.verbose, args.hidden) print_verbose(f"Processing: {base}", args.verbose)
process_directory(
base_path=base,
old_string=args.old_string,
new_string=args.new_string,
recursive=args.recursive,
rename_folders=args.folders,
rename_files=args.files,
replace_in_content=args.content,
preview=args.preview,
verbose=args.verbose,
include_hidden=args.hidden,
rename_paths=args.rename_paths,
auto_path=args.auto_path
)
if __name__ == "__main__": if __name__ == '__main__':
main() main()

212
test.py Normal file
View File

@@ -0,0 +1,212 @@
#!/usr/bin/env python3
import os
import shutil
import tempfile
import unittest
import argparse
from main import process_directory, replace_content, main as cli_main
class TestBulkStringReplacer(unittest.TestCase):
def setUp(self):
# create isolated temp dir
self.base = tempfile.mkdtemp()
def tearDown(self):
shutil.rmtree(self.base)
def create_file(self, relpath, content=''):
full = os.path.join(self.base, relpath)
os.makedirs(os.path.dirname(full), exist_ok=True)
with open(full, 'w', encoding='utf-8') as f:
f.write(content)
return full
def test_test_prefix_extraction_prompt_no_auto(self):
# Setup a test file with test_ prefix
src = self.create_file('dir/test_module.py', 'print("hello")')
# Monkeypatch input to simulate 'y'
inputs = iter(['y'])
original_input = __builtins__['input']
__builtins__['input'] = lambda _: next(inputs)
try:
process_directory(
base_path=self.base,
old_string='module',
new_string='module_new',
recursive=True,
rename_folders=False,
rename_files=False,
replace_in_content=False,
preview=False,
verbose=False,
include_hidden=True,
rename_paths=True,
auto_path=False
)
finally:
__builtins__['input'] = original_input
# After extraction, expect dir/module/test_module_new.py
dst = os.path.join(self.base, 'dir', 'module', 'test_module_new.py')
self.assertTrue(os.path.exists(dst))
def test_test_prefix_default_no_extraction_auto(self):
# Setup a test file with test_ prefix
src = self.create_file('dir/test_file.py', 'print("hello")')
process_directory(
base_path=self.base,
old_string='file',
new_string='file_new',
recursive=True,
rename_folders=False,
rename_files=False,
replace_in_content=False,
preview=False,
verbose=False,
include_hidden=True,
rename_paths=True,
auto_path=True
)
# With auto_path, should do simple path replace: dir/test_file.py -> dir/test_file_new.py
dst = os.path.join(self.base, 'dir', 'test_file_new.py')
self.assertTrue(os.path.exists(dst))
def test_replace_content(self):
f = self.create_file('foo.txt', 'hello OLD world')
replace_content(f, 'OLD', 'NEW', preview=False, verbose=False)
with open(f, 'r', encoding='utf-8') as fp:
self.assertIn('hello NEW world', fp.read())
def test_rename_file(self):
f = self.create_file('OLDfile.txt', '')
process_directory(
base_path=self.base,
old_string='OLD', new_string='NEW',
recursive=False,
rename_folders=False, rename_files=True,
replace_in_content=False,
preview=False, verbose=False,
include_hidden=True, rename_paths=False, auto_path=True
)
self.assertTrue(os.path.exists(os.path.join(self.base, 'NEWfile.txt')))
self.assertFalse(os.path.exists(f))
def test_rename_folder(self):
os.makedirs(os.path.join(self.base, 'OLDfolder', 'x'))
process_directory(
base_path=self.base,
old_string='OLD', new_string='NEW',
recursive=False,
rename_folders=True, rename_files=False,
replace_in_content=False,
preview=False, verbose=False,
include_hidden=True, rename_paths=False, auto_path=True
)
self.assertTrue(os.path.isdir(os.path.join(self.base, 'NEWfolder')))
self.assertFalse(os.path.isdir(os.path.join(self.base, 'OLDfolder')))
def test_full_path_move(self):
cfg = 'vars/configuration.yml'
full = self.create_file(cfg, 'DATA')
process_directory(
base_path=self.base,
old_string='vars/configuration.yml',
new_string='config/main.yml',
recursive=True,
rename_folders=False, rename_files=False,
replace_in_content=False,
preview=False, verbose=False,
include_hidden=True, rename_paths=True, auto_path=True
)
self.assertFalse(os.path.exists(full))
self.assertTrue(os.path.exists(os.path.join(self.base, 'config', 'main.yml')))
def test_path_and_folders_conflict(self):
parser = argparse.ArgumentParser()
parser.add_argument('-P', dest='rename_paths', action='store_true')
parser.add_argument('-F', dest='folders', action='store_true')
args = parser.parse_args(['-P', '-F'])
with self.assertRaises(SystemExit):
if args.rename_paths and args.folders:
parser.error("Cannot use --path and --folders together.")
def test_preview_does_nothing(self):
f = self.create_file('OLD.txt', 'OLD')
os.makedirs(os.path.join(self.base, 'OLDdir'))
process_directory(
base_path=self.base,
old_string='OLD', new_string='NEW',
recursive=True,
rename_folders=True, rename_files=True,
replace_in_content=True,
preview=True, verbose=False,
include_hidden=True, rename_paths=True, auto_path=True
)
self.assertTrue(os.path.exists(f))
self.assertTrue(os.path.isdir(os.path.join(self.base, 'OLDdir')))
with open(f, 'r', encoding='utf-8') as fp:
self.assertIn('OLD', fp.read())
def test_module_path_replacement_in_python_files(self):
content = 'from old/path import func'
src = self.create_file('old/path/module.py', content)
process_directory(
base_path=self.base,
old_string='old/path', new_string='old.path',
recursive=True,
rename_folders=False, rename_files=False,
replace_in_content=False,
preview=False, verbose=False,
include_hidden=True, rename_paths=True, auto_path=True
)
new_path = os.path.join(self.base, 'old.path', 'module.py')
self.assertTrue(os.path.exists(new_path))
with open(new_path, 'r', encoding='utf-8') as fp:
self.assertIn('from old.path import func', fp.read())
def test_non_python_files_are_not_content_updated(self):
content = 'some reference to old/path in text'
txt = self.create_file('old/path/readme.txt', content)
process_directory(
base_path=self.base,
old_string='old/path', new_string='old.path',
recursive=True,
rename_folders=False, rename_files=False,
replace_in_content=False,
preview=False, verbose=False,
include_hidden=True, rename_paths=True, auto_path=True
)
new_txt = os.path.join(self.base, 'old.path', 'readme.txt')
self.assertTrue(os.path.exists(new_txt))
with open(new_txt, 'r', encoding='utf-8') as fp:
self.assertIn('old/path', fp.read())
def test_auto_path_line_level_replacement(self):
# create a Python file with two occurrences
lines = [
'import a\n',
'path = "old/path/to/module"\n',
'print("old/path/example")\n'
]
py = self.create_file('old/path/test.py', ''.join(lines))
process_directory(
base_path=self.base,
old_string='old/path', new_string='new/path',
recursive=True,
rename_folders=False, rename_files=False,
replace_in_content=False,
preview=False, verbose=False,
include_hidden=True, rename_paths=True, auto_path=True
)
new_py = os.path.join(self.base, 'new', 'path', 'test.py')
self.assertTrue(os.path.exists(new_py))
with open(new_py, 'r', encoding='utf-8') as fp:
content = fp.read()
# all replaced slash-based
self.assertIn('path = "new/path/to/module"', content)
self.assertIn('print("new/path/example")', content)
if __name__ == '__main__':
unittest.main()