Overall optimations

.github/FUNDING.yml

@@ -0,0 +1,7 @@
+github: kevinveenbirkenbach
+
+patreon: kevinveenbirkenbach
+
+buy_me_a_coffee: kevinveenbirkenbach
+
+custom: https://s.veen.world/paypaldonate

README.md

@@ -1,62 +1,136 @@
-# Linux Image Manager
+# Linux Image Manager 🖥️🛠️
-[![License: GPL v3](https://img.shields.io/badge/License-GPL%20v3-blue.svg)](./LICENSE.txt) 
 
-This repository contains some shell scripts to download and configure linux images and to transfer them to a storage.
+[![GitHub Sponsors](https://img.shields.io/badge/Sponsor-GitHub%20Sponsors-blue?logo=github)](https://github.com/sponsors/kevinveenbirkenbach) [![Patreon](https://img.shields.io/badge/Support-Patreon-orange?logo=patreon)](https://www.patreon.com/c/kevinveenbirkenbach) [![Buy Me a Coffee](https://img.shields.io/badge/Buy%20me%20a%20Coffee-Funding-yellow?logo=buymeacoffee)](https://buymeacoffee.com/kevinveenbirkenbach) [![PayPal](https://img.shields.io/badge/Donate-PayPal-blue?logo=paypal)](https://s.veen.world/paypaldonate)
 
-## Virtual Btrfs RAID1 Setup
+[![License: GPL v3](https://img.shields.io/badge/License-GPL%20v3-blue.svg)](./LICENSE.txt) [![GitHub stars](https://img.shields.io/github/stars/kevinveenbirkenbach/linux-image-manager.svg?style=social)](https://github.com/kevinveenbirkenbach/linux-image-manager/stargazers)
 
-To setup a virtual btrfs encrypted raid 1 execute: 
+Linux Image Manager (lim) is a powerful collection of shell scripts for downloading, configuring, and managing Linux images. Whether you're setting up encrypted storage, configuring a virtual Btrfs RAID1, performing backups, or chrooting into an image, this tool makes Linux image administration simple and efficient. 🚀
+
+> **Note:** In this project, `lim` is an alias for the **main.py** wrapper script which orchestrates the execution of the various shell scripts.
+
+## Features ✨
+
+- **Image Download & Setup:** Automatically download and prepare Linux distributions.
+- **Encrypted Storage:** Configure LUKS encryption for secure image management.
+- **Virtual RAID1:** Easily set up virtual Btrfs RAID1 for data redundancy.
+- **Backup & Restore:** Create image backups from devices using dd.
+- **Chroot Environment:** Easily enter a chroot shell to maintain or modify Linux images.
+- **Automated Procedures:** Simplify partitioning, formatting, mounting, and more.
+
+## Installation 📦
+
+Install Linux Image Manager quickly using [Kevin's Package Manager](https://github.com/kevinveenbirkenbach/package-manager) under the alias `lim`. Just run:
 
 ```bash
-  bash scripts/encryption/storage/raid1/setup.sh
+package-manager install lim
 ```
 
-## Setup
+This command makes Linux Image Manager globally available as `lim` in your terminal. The `lim` alias points to the **main.py** wrapper script.
 
-To install a Linux distribution execute:
+## Usage ⚙️
 
-```bash
+The **main.py** wrapper provides a unified interface to run the different shell scripts included in this project. It supports various script types and allows you to pass additional parameters. The built-in `--help` option displays detailed usage information.
-  sudo bash ./scripts/image/setup.sh
-```
 
-### Cleanup
+### Available Script Types
 
-To cleanup the image setup execute:
+- **Image Setup (`--type image`):**  
-```bash
+  Executes the Linux image setup located at `scripts/image/setup.sh`. This setup:
-fuser -k /dev/mapper/linux-image-manager-*; 
+  - Creates partitions and formats them.
-umount -f /dev/mapper/linux-image-manager-*; 
+  - Transfers the Linux image file to the device.
-fuser -k /tmp/linux-image-manager-*; 
+  - Configures boot and root partitions.
-umount -f /tmp/linux-image-manager-*;
-```
 
-Additional you can unmount the device with a command like
+- **Single Drive Encryption Setup (`--type single`):**  
+  Executes the single-drive encryption setup from `scripts/encryption/storage/single_drive/setup.sh`. This setup:
+  - Sets up disk encryption using LUKS on one drive.
+  - Configures a Btrfs file system for secure storage.
 
-```bash
+- **RAID1 Encryption Setup (`--type raid1`):**  
-umount -f /dev/sd*;
+  Executes the RAID1 encryption setup found at `scripts/encryption/storage/raid1/setup.sh`. This setup:
-```
+  - Configures a virtual RAID1 with two drives.
+  - Uses LUKS encryption and a Btrfs RAID1 file system for redundancy.
 
-### Verification
+- **Backup Image Setup (`--type backup`):**  
-To verify that the unmounting was successfull, check the result of
+  Executes the backup image setup located at `scripts/image/backup.sh`. This setup:
-```bash
+  - Creates an image backup from a memory device to a file.
-mount
+  - Uses `dd` to transfer the image from the specified device to an image file.
-```
 
-## Chroot
+- **Chroot Environment Setup (`--type chroot`):**  
+  Executes the chroot setup from `scripts/image/chroot.sh`. This setup:
+  - Mounts partitions and configures the chroot environment for a Linux image.
+  - Provides a shell within the Linux image for system maintenance.
 
-To chroot into a Linux distribution on a storage execute:
+### Command-Line Options
 
-```bash
+- **`--type`**  
-  sudo bash ./scripts/image/chroot.sh
+  **(Required)** Choose the type of script to execute. Options include: `image`, `single`, `raid1`, `backup`, and `chroot`.
-```
 
-## Backup
+- **`--extra`**  
+  **(Optional)** Pass any extra parameters directly to the selected shell script.
 
-To backup a image execute:
+- **`--auto-confirm`**  
+  **(Optional)** Automatically bypass the confirmation prompt before executing the selected script.
 
-```bash
+- **`--help`**  
-  sudo bash ./scripts/image/backup.sh
+  **(Optional)** Displays detailed help information about the command-line options and usage of the wrapper. Simply run:
-```
+  ```bash
+  lim --help
+  ```
+  to view the complete help message.
 
-## License
+### Example Commands
 
-The ["GNU GENERAL PUBLIC LICENSE Version 3"](./LICENSE.txt) applies to this project.
+- **Display Help:**  
+  ```bash
+  lim --help
+  ```
+
+- **Show Information About the Image Setup:**  
+  ```bash
+  lim --type image --info
+  ```
+
+- **Execute the Linux Image Setup (with extra parameters):**  
+  ```bash
+  lim --type image --extra --some-option value
+  ```
+
+- **Run the Single Drive Encryption Setup without a confirmation prompt:**  
+  ```bash
+  lim --type single --auto-confirm
+  ```
+
+- **Execute the RAID1 Encryption Setup:**  
+  ```bash
+  lim --type raid1
+  ```
+
+- **Perform a Backup of an Image:**  
+  ```bash
+  lim --type backup
+  ```
+
+- **Enter a Chroot Environment for a Linux Image:**  
+  ```bash
+  lim --type chroot
+  ```
+
+For additional details on each script and further configuration options, please refer to the `scripts/` and `configuration/` directories.
+
+## Configuration & Customization 🔧
+
+Customize your environment in the `configuration/` folder:
+- **General Packages:** Contains common packages for all setup scripts.
+- **Server LUKS Packages:** Contains packages needed for setting up LUKS encryption on servers.
+
+## License 📜
+
+This project is licensed under the GNU General Public License Version 3. See the [LICENSE.txt](./LICENSE.txt) file for details.
+
+## Contact & Support 💬
+
+- **Author:** Kevin Veen-Birkenbach  
+- **Email:** [kevin@veen.world](mailto:kevin@veen.world)  
+- **Website:** [https://www.veen.world/](https://www.veen.world/)
+
+Feel free to contribute, report issues, or get in touch. Happy Linux managing! 😊
+```

main.py

@@ -0,0 +1,118 @@
+#!/usr/bin/env python3
+import subprocess
+import os
+import argparse
+import sys
+
+def run_script(script_path, extra_args):
+    if not os.path.exists(script_path):
+        print(f"[ERROR] Script not found at {script_path}")
+        exit(1)
+    command = ["sudo", "bash", script_path] + extra_args
+    print(f"[INFO] Running command: {' '.join(command)}")
+    # Pass the parent's stdout and stderr so that progress output shows in real time.
+    result = subprocess.run(command, stdout=sys.stdout, stderr=sys.stderr)
+    if result.returncode != 0:
+        print(f"[ERROR] Script exited with code {result.returncode}")
+        exit(result.returncode)
+    print("[SUCCESS] Script executed successfully.")
+
+def main():
+    # Use os.path.realpath to get the actual path of this file regardless of symlinks.
+    repo_root = os.path.dirname(os.path.realpath(__file__))
+
+    # Define available scripts along with their descriptions.
+    setup_scripts = {
+        "image": {
+            "path": os.path.join(repo_root, "scripts", "image", "setup.sh"),
+            "description": (
+                "Linux Image Setup:\n"
+                "  - Creates partitions and formats them.\n"
+                "  - Transfers the Linux image file to the device.\n"
+                "  - Configures boot and root partitions."
+            )
+        },
+        "single": {
+            "path": os.path.join(repo_root, "scripts", "encryption", "storage", "single_drive", "setup.sh"),
+            "description": (
+                "Single Drive Encryption Setup:\n"
+                "  - Sets up disk encryption using LUKS on one drive.\n"
+                "  - Configures a Btrfs file system for secure storage."
+            )
+        },
+        "raid1": {
+            "path": os.path.join(repo_root, "scripts", "encryption", "storage", "raid1", "setup.sh"),
+            "description": (
+                "RAID1 Encryption Setup:\n"
+                "  - Configures a virtual RAID1 with two drives.\n"
+                "  - Uses LUKS encryption and a Btrfs RAID1 file system for redundancy."
+            )
+        },
+        "backup": {
+            "path": os.path.join(repo_root, "scripts", "image", "backup.sh"),
+            "description": (
+                "Backup Image Setup:\n"
+                "  - Creates an image backup from a memory device to a file.\n"
+                "  - Uses dd to transfer the image from the specified device to an image file."
+            )
+        },
+        "chroot": {
+            "path": os.path.join(repo_root, "scripts", "image", "chroot.sh"),
+            "description": (
+                "Chroot Environment Setup:\n"
+                "  - Mounts partitions and configures the chroot environment for a Linux image.\n"
+                "  - Provides a shell within the Linux image for system maintenance."
+            )
+        }
+    }
+
+    parser = argparse.ArgumentParser(
+        description="Wrapper for executing various scripts from Linux Image Manager.",
+        epilog=(
+            "Available script types:\n"
+            "  image  - Linux Image Setup\n"
+            "  single - Single Drive Encryption Setup\n"
+            "  raid1  - RAID1 Encryption Setup\n"
+            "  backup - Backup Image Setup\n"
+            "  chroot - Chroot Environment Setup\n\n"
+            "Additional Options:\n"
+            "  --extra         Pass extra parameters to the selected script.\n"
+            "  --auto-confirm  Bypass the confirmation prompt before execution.\n"
+            "  --help          Display this help message and exit."
+        ),
+        formatter_class=argparse.RawDescriptionHelpFormatter
+    )
+    parser.add_argument("--type", required=True, choices=list(setup_scripts.keys()),
+                        help="Select the script type to execute. Options: " + ", ".join(setup_scripts.keys()))
+    parser.add_argument("--extra", nargs=argparse.REMAINDER, default=[],
+                        help="Extra parameters to pass to the selected script.")
+    parser.add_argument("--auto-confirm", action="store_true",
+                        help="Automatically confirm execution without prompting the user.")
+
+    args = parser.parse_args()
+
+    script_info = setup_scripts[args.type]
+    print("[INFO] Selected script type:", args.type)
+    print("[INFO] Description:")
+    print(script_info["description"])
+    print("[INFO] Script path:", script_info["path"])
+    if args.extra:
+        print("[INFO] Extra parameters provided:", " ".join(args.extra))
+    else:
+        print("[INFO] No extra parameters provided.")
+
+    if not args.auto_confirm:
+        try:
+            input("Press Enter to execute the script or Ctrl+C to cancel...")
+        except KeyboardInterrupt:
+            print("\n[ERROR] Execution aborted by user.")
+            exit(1)
+
+    run_script(script_info["path"], args.extra)
+
+if __name__ == "__main__":
+    try:
+        main()
+    except KeyboardInterrupt:
+        print("\n[ERROR] Execution aborted by user.")
+        exit(1)

requirements.yml

@@ -0,0 +1,2 @@
+pacman:
+  - pv

scripts/base.sh

@@ -95,12 +95,32 @@ set_device_path(){
   info "Optimal blocksize set to: $OPTIMAL_BLOCKSIZE" || error
 }
 
+print_partition_table_info() {
+  echo "##########################################################################################"
+  echo "Note on Partition Table Deletion:"
+  echo "---------------------------------------------"
+  echo "• MBR (Master Boot Record):"
+  echo "  - Typically occupies the first sector (512 bytes), i.e., 1 block."
+  echo ""
+  echo "• GPT (GUID Partition Table):"
+  echo "  - Uses a protective MBR (1 block), a GPT header (1 block),"
+  echo "    and usually a partition entry array that takes up about 32 blocks."
+  echo "  - Total: approximately 34 blocks (assuming a 512-byte block size)."
+  echo ""
+  echo "Recommendation: For deleting a GPT partition table, use a block size of 512 bytes"
+  echo "                and overwrite at least 34 blocks to ensure the entire table is cleared."
+  echo "##########################################################################################"
+}
+
 overwrite_device() {
+  # Call the function to display the information.
+  print_partition_table_info
+
   question "Should $device_path be overwritten with zeros before copying? (y/N/block count)" && read -r copy_zeros_to_device
   case "$copy_zeros_to_device" in
     y)
       info "Overwriting entire device..." &&
-      dd if=/dev/zero of="$device_path" bs="$OPTIMAL_BLOCKSIZE" status=progress || error "Overwriting $device_path failed."
+      dd if=/dev/zero of="$device_path" bs="$OPTIMAL_BLOCKSIZE" status=progress && sync || error "Overwriting $device_path failed."
       ;;
     N|'')
       info "Skipping Overwriting..."
@@ -111,7 +131,7 @@ overwrite_device() {
     *)
       if [[ "$copy_zeros_to_device" =~ ^[0-9]+$ ]]; then
         info "Overwriting $copy_zeros_to_device blocks..." &&
-        dd if=/dev/zero of="$device_path" bs="$OPTIMAL_BLOCKSIZE" count="$copy_zeros_to_device" status=progress || error "Overwriting $device_path failed."
+        dd if=/dev/zero of="$device_path" bs="$OPTIMAL_BLOCKSIZE" count="$copy_zeros_to_device" status=progress && sync || error "Overwriting $device_path failed."
       else
         error "Invalid input. Block count must be a number."
       fi

scripts/encryption/storage/base.sh

@@ -31,10 +31,11 @@ create_luks_key_and_update_cryptab(){
     then
       warning "File already exists. Overwriting!"
   fi
-  sudo dd if=/dev/urandom of="$secret_key_path" bs=512 count=8 &&
+  sudo dd if=/dev/urandom of="$secret_key_path" bs=512 count=8 && sync &&
   
   info "Opening and closing device to verify that everything works fine..." &&
-  sudo cryptsetup -v luksClose "$1" || info "No need to luksClose $1." &&
+  sudo cryptsetup -v luksClose "$1" || info "No need to luksClose $1. Device isn't open." &&
+  sudo cryptsetup luksAddKey $2 $secret_key_path &&
   sudo cryptsetup -v luksOpen "$2" "$1" --key-file="$secret_key_path" &&
   sudo cryptsetup -v luksClose "$1" &&
   info "Reading UUID..." &&

scripts/encryption/storage/raid1/mount_on_boot.sh

@@ -3,7 +3,13 @@
 # shellcheck disable=SC2154  # Referenced but not assigned
 # shellcheck disable=SC2015 #Deactivate bool hint
 source "$(dirname "$(readlink -f "${0}")")/base.sh" || (echo "Loading base.sh failed." && exit 1)
-info "Automount raid1 encrypted storages..." &&
+info "Activate Automount raid1 encrypted storages..." &&
+echo ""
+for dev in $(lsblk -dno NAME); do
+    if sudo cryptsetup isLuks /dev/$dev 2>/dev/null; then
+        info "/dev/$dev is a LUKS encrypted storage device."
+    fi
+done
 set_raid1_devices_mount_partition_and_mapper_paths &&
 create_luks_key_and_update_cryptab "$mapper_name_1" "$device_path_1" &&
 info "Creating mount folder unter \"$mount_path_1\"..." &&

scripts/image/backup.sh

@@ -25,6 +25,6 @@ question "Please confirm by pushing \"Enter\". To cancel use \"Ctrl + Alt + C\""
 read -r bestaetigung && echo "$bestaetigung";
 
 info "Imagetransfer starts. This can take a while..." &&
-dd if="$device_path" of="$ofi" bs=1M status=progress || error "\"dd\" failed.";
+dd if="$device_path" of="$ofi" bs=1M status=progress && sync || error "\"dd\" failed.";
 
 success "Imagetransfer successfull." && exit 0;

scripts/image/setup.sh

@@ -147,6 +147,10 @@ case "$operation_system" in
               base_download_url="https://download.manjaro.org/gnome/22.1.3/"
               image_name="manjaro-gnome-22.1.3-230529-linux61.iso"
               ;;
+            "24")
+              base_download_url="https://download.manjaro.org/gnome/24.2.1/"
+              image_name="manjaro-gnome-24.2.1-241216-linux612.iso"
+              ;;  
             "raspberrypi")
               # at the moment just optimized for raspberry pi 4
               base_download_url="https://github.com/manjaro-arm/rpi4-images/releases/download/23.02/"
@@ -241,18 +245,28 @@ if [ -z "$image_checksum" ]; then
     done
 fi
 
-if [[ -v image_checksum ]]
+if [[ -v image_checksum ]]; then
-  then
+  info "A checksum is defined for the image."
-    (info "Checking md5 checksum..." && echo "$image_checksum $image_path"| md5sum -c -) ||
+  info "Checksums verify file integrity to ensure that the file was not corrupted during download."
-    (info "Checking sha1 checksum..." && echo "$image_checksum $image_path"| sha1sum -c -) ||
+  info "The script will try verifying the integrity using MD5, then SHA1, and finally SHA256 if needed."
-    (info "Checking sha256 checksum..." && echo "$image_checksum $image_path"| sha256sum -c -) ||
+  
-    error "Verification failed. HINT: Force the download of the image."
+  info "Trying MD5 checksum verification..."
-  else
+  (info "Checking md5 checksum..." && echo "$image_checksum $image_path" | md5sum -c -) ||
-    warning "Verification is not possible. No checksum is defined."
+  (warning "MD5 verification failed. This may indicate data corruption." &&
+   info "Trying SHA1 checksum verification for a secondary integrity check..." &&
+   info "Checking sha1 checksum..." && echo "$image_checksum $image_path" | sha1sum -c -) ||
+  (warning "SHA1 verification failed. Attempting SHA256 verification for thoroughness." &&
+   info "SHA256 provides a more robust check and is used as a final integrity measure." &&
+   info "Checking sha256 checksum..." && echo "$image_checksum $image_path" | sha256sum -c -) ||
+  error "Verification failed. HINT: Force the download of the image."
+else
+  warning "No checksum is defined. Skipping checksum verification."
 fi
 
-info "Verifying signature..."
+info "Note: Checksums verify integrity but do not confirm authenticity."
+info "Proceeding to signature verification, which ensures the file comes from a trusted source."
 signature_download_url="$download_url.sig"
+info "Attempting to download the image signature from: $signature_download_url"
 info "Try to download image signature from $signature_download_url."
 
 if wget -q --method=HEAD "$signature_download_url"; then
@@ -384,7 +398,7 @@ if [ "$transfer_image" = "y" ]
       elif [ "${image_path: -4}" = ".iso" ]
         then
           info "Transfering .iso file..." &&
-          sudo dd if="$image_path" of="$device_path" bs="$OPTIMAL_BLOCKSIZE" conv=fsync status=progress &&
+          pv "$image_path" | sudo dd of="$device_path" bs="$OPTIMAL_BLOCKSIZE" conv=fsync &&
           sync ||
           error
       elif [ "${image_path: -3}" = ".xz" ]