Added more debugging info

01_FEATURES.md

@@ -23,4 +23,4 @@ No need to be a Linux or Docker expert! CyMaIS simplifies deployment with intuit
 ## Open Source Trust & Transparency 🔓
 As an open-source project, CyMaIS guarantees transparency, security, and community-driven development, ensuring continuous improvements and adherence to industry best practices.
 
-For further information, check out the full [User Guide](04_USER_GUIDE.md) and [Enterprise Solutions](10_ENTERPRISE_SOLUTIONS.md).
+For further information, check out the [application glosar](roles/application_glosar), [applications ordered by category](roles/application_categories) and the [detailled ansible role descriptions](roles/ansible_role_glosar).

05_CUSTOMER_GUIDE.md

@@ -2,12 +2,13 @@
 
 Are you looking for a **reliable IT infrastructure** for your business or organization? **CyMaIS** is here to help!
 
-### Who Can Benefit? 🎯
-✅ **Small & Medium Businesses** - Automate IT setup & security
-✅ **Enterprises** - Scale deployments with modular architecture
-✅ **NGOs & Organizations** - Secure, cost-effective infrastructure solutions
+## Who Can Benefit? 🎯
+✅ **Small & Medium Businesses** - IT infrastructure with everything included what you need. E.g. data clouds, mailservers, vpn's, homepages, documentation tools, etc.
+✅ **Enterprises** - Scale the solutions for Small & Medium Businesses up for an unlimeted amount of users
+✅ **NGOs & Organizations** - Secure, cost-effective infrastructure solutions on Open Source Base
+✅ **Journalists & Content Creators** - Host your content on your own servers, share it via the Fediverse and avoid cencorship
 
-### Why Choose CyMaIS? 🚀
+## Why Choose CyMaIS? 🚀
 - **Fast Deployment** - Get your IT setup running in minutes
 - **Security First** - Encrypted backups, 2FA, and secure logins
 - **Scalable & Customizable** - Adapts to your specific needs

06_ADMINISTRATOR_GUIDE.md

@@ -0,0 +1,22 @@
+# Administrator Guide 🖥️
+
+This guide is for **system administrators** who are deploying and managing CyMaIS infrastructure.
+
+## Setting Up CyMaIS 🏗️
+Follow these guides to install and configure CyMaIS:
+- [Setup Guide](07_SETUP_GUIDE.md)
+- [Configuration Guide](08_CONFIGURATION.md)
+- [Deployment Guide](09_DEPLOY.md)
+
+## Key Responsibilities 🔧
+- **User Management** - Configure LDAP, Keycloak, and user permissions.
+- **Security & Backups** - Set up `backup-remote-to-local`, `backup-data-to-usb`, and `system-security` roles.
+- **Application Hosting** - Deploy services like `Nextcloud`, `Matrix`, `Gitea`, and more.
+- **Networking & VPN** - Configure `WireGuard`, `OpenVPN`, and `Nginx Reverse Proxy`.
+
+## Managing & Updating CyMaIS 🔄
+- Regularly update services using `update-docker`, `update-pacman`, or `update-apt`.
+- Monitor system health with `health-btrfs`, `health-nginx`, and `health-docker-container`.
+- Automate system maintenance with `system-maintenance-lock`, `cleanup-backups-service`, and `restart-docker`.
+
+For more details, refer to the specific guides above.

10_DEVELOPER_GUIDE.rst

@@ -3,12 +3,6 @@ Developer Guide
 
 Welcome to the **CyMaIS Developer Guide**! This guide provides essential information for developers who want to contribute to the CyMaIS open-source project.
 
-Getting Started
----------------
-To understand the overall structure of CyMaIS, start by reviewing the available YAML configuration files:
-
-- :doc:`docs/generated/yaml_index`
-
 Explore CyMaIS Solutions
 ------------------------
 CyMaIS offers various solutions for IT infrastructure automation. Learn more about the available applications:
@@ -18,17 +12,24 @@ CyMaIS offers various solutions for IT infrastructure automation. Learn more abo
 
 For Developers
 --------------
-### Understanding Ansible Roles
+
+Understanding Ansible Roles
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
 CyMaIS is powered by **Ansible** roles to automate deployments. Developers can explore the technical details of our roles here:
 
 - :doc:`roles/ansible_role_glosar`
 
-### Contributing to CyMaIS
+Contributing to CyMaIS
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
 Want to contribute to the project or explore the source code? Check out our **GitHub repository**:
 
 - `CyMaIS GitHub Repository <https://github.com/kevinveenbirkenbach/cymais/tree/master/roles>`_
 
-### Contribution Guidelines
+Contribution Guidelines
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
 1. **Fork the Repository** – Start by forking the CyMaIS repository.
 2. **Create a New Branch** – Make changes in a dedicated branch.
 3. **Follow Coding Standards** – Ensure your code is well-documented and follows best practices.

10_ENTERPRISE_SOLUTIONS.md

@@ -2,13 +2,13 @@
 
 **CyMaIS** provides powerful **enterprise-grade IT infrastructure solutions**, enabling businesses to scale securely and efficiently.
 
-### How CyMaIS Helps Enterprises 🔧
+## How CyMaIS Helps Enterprises 🔧
 - **Automated Deployment** - Set up secure servers & workstations effortlessly
 - **Advanced Security** - Integrated 2FA, LDAP, encrypted storage
 - **High Availability** - Scalable infrastructure for growing enterprises
 - **Compliance & Audit Logs** - Maintain regulatory standards
 
-### Use Cases 💼
+## Use Cases 💼
 ✅ **Cloud-Based Infrastructure** (Docker, Kubernetes, CI/CD pipelines)
 ✅ **Enterprise Networking & VPN** (WireGuard, OpenVPN, Firewall rules)
 ✅ **Database & Business Apps** (PostgreSQL, Nextcloud, ERP systems)

11_INVESTOR_INFORMATIONS.md

@@ -2,12 +2,12 @@
 
 🚀 **CyMaIS is seeking investors** to expand its reach and continue development. With an increasing demand for automated IT solutions, **CyMaIS has the potential to revolutionize IT infrastructure management.**
 
-### Market Potential 📈
+## Market Potential 📈
 - **$500B+ Global IT Infrastructure Market**
 - Growing **open-source adoption** across enterprises
 - Increasing need for **automation & cybersecurity**
 
-### Why Invest in CyMaIS? 🔥
+## Why Invest in CyMaIS? 🔥
 - **Unique Automation Approach** - Pre-configured roles for quick IT setup
 - **Security & Compliance Focus** - Built-in security best practices
 - **Scalability** - Modular framework adaptable to various industries

README.md

@@ -1,18 +1,19 @@
 
-# README.md
-
 # CyMaIS - Cyber Master Infrastructure Solution 🚀
 
+[![GitHub Sponsors](https://img.shields.io/badge/Sponsor-GitHub%20Sponsors-blue?logo=github)](https://github.com/sponsors/kevinveenbirkenbach) [![Patreon](https://img.shields.io/badge/Support-Patreon-orange?logo=patreon)](https://www.patreon.com/c/kevinveenbirkenbach) [![Buy Me a Coffee](https://img.shields.io/badge/Buy%20me%20a%20Coffee-Funding-yellow?logo=buymeacoffee)](https://buymeacoffee.com/kevinveenbirkenbach) [![PayPal](https://img.shields.io/badge/Donate-PayPal-blue?logo=paypal)](https://s.veen.world/paypaldonate)
+
 Welcome to **CyMaIS (Cyber Master Infrastructure Solution)**, a powerful automation framework that simplifies IT infrastructure setup and management. Whether you are an **end-user** looking to access cloud services securely or an **administrator** responsible for deploying and maintaining infrastructure, CyMaIS provides a seamless and secure solution.
 
 ![CyMaIS Logo](assets/img/logo.png)
 
 ## What is CyMaIS? 📌
-CyMaIS leverages **Docker, Linux, and Ansible** to provide an automated and modular infrastructure solution. With more then **148 pre-configured roles**, it supports a wide range of applications, from cloud services to local server management and desktop workstation setups.
+CyMaIS leverages **Docker, Linux, and Ansible** to provide an automated and modular infrastructure solution. With more then **150 pre-configured roles**, it supports a wide range of applications, from cloud services to local server management and desktop workstation setups.
 
 ## Guides 📖
 - **[User Guide](04_USER_GUIDE.md)** - For end-users accessing cloud apps like Nextcloud, Matrix, and more.
 - **[Administrator Guide](06_ADMINISTRATOR_GUIDE.md)** - For system administrators deploying CyMaIS.
+- **[Customer Guide](05_CUSTOMER_GUIDE.md)** - For customers which are interested in an infrastructure setup
 
 ## Key Features 🎯
 - **Automated IT deployment** 📦 - Pre-built roles for server and PC setups
@@ -21,6 +22,8 @@ CyMaIS leverages **Docker, Linux, and Ansible** to provide an automated and modu
 - **Backup & recovery solutions** 💾 - Automate data security and prevent loss
 - **Infrastructure monitoring & maintenance** 📊 - Keep your system running optimally
 
+More informations about the features you will find [here](01_FEATURES.md).
+
 ## Get Started 🚀
 1. **Install CyMaIS** via [Kevin's Package Manager](https://github.com/kevinveenbirkenbach/package-manager)
 2. **Setup CyMaIS** using:
@@ -30,6 +33,4 @@ CyMaIS leverages **Docker, Linux, and Ansible** to provide an automated and modu
 3. **Explore Commands** with:
    ```sh
    cymais --help
-   ```
-
-For detailed setup instructions, check out the [Setup Guide](07_SETUP_GUIDE.md) and [Configuration Guide](08_CONFIGURATION.md).
+   ```

docs/.gitignore

@@ -1,5 +1,8 @@
 assets/img/*
 !assets/img/.gitkeep
-build/*
+output/*
+!output/.gitkeep
 generated/*
-!generated/.gitkeep
+!generated/.gitkeep
+requirements/*
+!requirements/.gitkeep

docs/Dockerfile

@@ -2,26 +2,34 @@ ARG DOCKER_PYTHON_VERSION
 FROM python:${DOCKER_PYTHON_VERSION}
 
 ARG SPHINX_SOURCE_DIR
-ARG SPHINX_BUILD_DIR
+ARG SPHINX_OUTPUT_DIR
 ARG SPHINX_EXEC_DIR
-ARG SPHINX_APP_DIR
+ARG SPHINX_DOCKER_EXEC_DIR
 ARG SPHINX_SOURCE_DIR_RELATIVE
 
 # Set the environment variables so they are available during build for Makefile
 ENV SPHINX_SOURCE_DIR=${SPHINX_SOURCE_DIR}
-ENV SPHINX_BUILD_DIR=${SPHINX_BUILD_DIR}
+ENV SPHINX_OUTPUT_DIR=${SPHINX_OUTPUT_DIR}
+ENV SPHINX_REQUIREMENTS_DIR=${SPHINX_EXEC_DIR}/requirements
 
-# Install required packages
-RUN apt-get update && apt-get install -y make curl
 
 # Set the working directory
-WORKDIR ${SPHINX_APP_DIR}
+WORKDIR ${SPHINX_DOCKER_EXEC_DIR}
+
+# Update and install make
+RUN apt-get update && apt install -y make
 
 # Copy the project files into the container
-COPY ${SPHINX_SOURCE_DIR_RELATIVE} ${SPHINX_APP_DIR}
+COPY ${SPHINX_SOURCE_DIR_RELATIVE} ${SPHINX_DOCKER_EXEC_DIR}
+
+# Build the requirement files
+RUN cd ${SPHINX_EXEC_DIR} && make extract-requirements
+
+# Install required packages
+RUN xargs -a ${SPHINX_REQUIREMENTS_DIR}/apt.txt apt install -y
 
 # Install Python packages via requirements.txt
-RUN cd ${SPHINX_EXEC_DIR} && pip install --upgrade pip && pip install -r requirements.txt
+RUN pip install --upgrade pip && pip install -r ${SPHINX_REQUIREMENTS_DIR}/pip.txt
 
 # Build the HTML documentation using Sphinx with the defined directories
 RUN cd ${SPHINX_EXEC_DIR} && make html
@@ -30,4 +38,4 @@ RUN cd ${SPHINX_EXEC_DIR} && make html
 EXPOSE 8000
 
 # Start a simple HTTP server to serve the built documentation
-CMD python -m http.server 8000 --directory "${SPHINX_BUILD_DIR}html/"
+CMD python -m http.server 8000 --directory "${SPHINX_OUTPUT_DIR}html/"

docs/Makefile

@@ -1,14 +1,33 @@
-# Minimal Makefile for Sphinx documentation
-#
-# You can set these variables from the command line, and also
-# from the environment
-SPHINXOPTS         		?=	-c .
-SPHINXBUILD        		?=	sphinx-build
-SPHINX_SOURCE_DIR  		?=	../
-SPHINX_BUILD_DIR   		?=	./build
-SPHINX_GENERATED_DIR 	=	$(SPHINX_BUILD_DIR)/../generated
+# PARAMETER (with default values)
 
-.PHONY: help install copy-images apidoc remove-generated html generate Makefile
+# Directory which cointains the Makefile
+SPHINX_EXEC_DIR			?= .
+
+# Directory from which the sources will be read
+SPHINX_SOURCE_DIR  		?= ../
+
+# Directory which contains the builded files
+SPHINX_OUTPUT_DIR   	?= ./output
+
+# Args parsed to the sphinx-build command
+SPHINXOPTS         		?= -c $(SPHINX_EXEC_DIR)
+
+# CONSTANTS
+
+# Sphinx build command
+SPHINX_BUILD_COMMAND    = sphinx-build
+
+# Directory which contains the auto generated files
+SPHINX_GENERATED_DIR 	= $(SPHINX_OUTPUT_DIR)/../generated
+
+# Directory which contains the extracted requirement files
+SPHINX_REQUIREMENTS_DIR = $(SPHINX_EXEC_DIR)/requirements
+
+.PHONY: help install copy-images apidoc remove-generated html generate extract-requirements Makefile
+
+extract-requirements:
+	@echo "Creating requirement files"
+	bash ./scripts/extract-requirements.sh "$(SPHINX_EXEC_DIR)/requirements.yml" "$(SPHINX_REQUIREMENTS_DIR)/apt.txt" "$(SPHINX_REQUIREMENTS_DIR)/pip.txt"
 
 # Copy images before running any Sphinx command (except for help)
 copy-images:
@@ -42,20 +61,20 @@ remove-generated:
 	- find $(SPHINX_GENERATED_DIR)/ -type f ! -name '.gitkeep' -delete
 
 help:
-	@$(SPHINXBUILD) -M help "$(SPHINX_SOURCE_DIR)" "$(SPHINX_BUILD_DIR)" $(SPHINXOPTS) $(O)
+	@$(SPHINX_BUILD_COMMAND) -M help "$(SPHINX_SOURCE_DIR)" "$(SPHINX_OUTPUT_DIR)" $(SPHINXOPTS) $(O)
 
 html: copy-images generate
 	@echo "Building Sphinx documentation..."
-	$(SPHINXBUILD) -M html "$(SPHINX_SOURCE_DIR)" "$(SPHINX_BUILD_DIR)" $(SPHINXOPTS)
+	$(SPHINX_BUILD_COMMAND) -M html "$(SPHINX_SOURCE_DIR)" "$(SPHINX_OUTPUT_DIR)" $(SPHINXOPTS)
 
 just-html:
-	@$(SPHINXBUILD) -M html "$(SPHINX_SOURCE_DIR)" "$(SPHINX_BUILD_DIR)" $(SPHINXOPTS)
+	@$(SPHINX_BUILD_COMMAND) -M html "$(SPHINX_SOURCE_DIR)" "$(SPHINX_OUTPUT_DIR)" $(SPHINXOPTS)
 
 
 clean: remove-generated
-	@$(SPHINXBUILD) -M clean "$(SPHINX_SOURCE_DIR)" "$(SPHINX_BUILD_DIR)" $(SPHINXOPTS) $(O)
+	@$(SPHINX_BUILD_COMMAND) -M clean "$(SPHINX_SOURCE_DIR)" "$(SPHINX_OUTPUT_DIR)" $(SPHINXOPTS) $(O)
 
 # Catch-all target: route all unknown targets to Sphinx using the new
 # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
 %: Makefile
-	@$(SPHINXBUILD) -M $@ "$(SPHINX_SOURCE_DIR)" "$(SPHINX_BUILD_DIR)" $(SPHINXOPTS) $(O)
+	@$(SPHINX_BUILD_COMMAND) -M $@ "$(SPHINX_SOURCE_DIR)" "$(SPHINX_OUTPUT_DIR)" $(SPHINXOPTS) $(O)

docs/generators/ansible_roles.py

@@ -15,7 +15,7 @@ def convert_md_to_rst(md_content):
         return result.stdout.decode("utf-8")
     except subprocess.CalledProcessError as e:
         print("Error converting Markdown to reStructuredText:", e)
-        return md_content  # Falls Pandoc fehlschlägt, nutze das Original als Fallback
+        return md_content
 
 def generate_ansible_roles_doc(roles_dir, output_dir):
     """Generates reStructuredText documentation for Ansible roles."""

docs/requirements.txt

@@ -1,9 +0,0 @@
-myst-parser
-sphinx
-sphinxawesome-theme
-docutils
-sphinx-jinja
-sphinxcontrib-yaml
-pathspec
-markdown2
-pandoc

docs/requirements.yml

@@ -0,0 +1,13 @@
+apt:
+  make
+  curl
+  pandoc
+pip:
+  myst-parser
+  sphinx
+  sphinxawesome-theme
+  docutils
+  sphinx-jinja
+  sphinxcontrib-yaml
+  pathspec
+  markdown2

docs/scripts/extract-requirements.sh

@@ -0,0 +1,38 @@
+#!/bin/bash
+
+# Check if correct number of arguments is given
+if [[ $# -ne 3 ]]; then
+  echo "Usage: $0 <input_file> <apt_output_file> <pip_output_file>"
+  echo "Input: $0 <$1> <$2> <$3>"
+  exit 1
+fi
+
+input_file="$1"
+apt_file="$2"
+pip_file="$3"
+
+# Clear the output files
+> "$apt_file"
+> "$pip_file"
+
+current_section=""
+
+while IFS= read -r line; do
+  [[ -z "$line" ]] && continue
+
+  if [[ "$line" == apt:* ]]; then
+    current_section="apt"
+    continue
+  elif [[ "$line" == pip:* ]]; then
+    current_section="pip"
+    continue
+  fi
+
+  package=$(echo "$line" | sed 's/^[[:space:]]*//')
+
+  if [[ "$current_section" == "apt" ]]; then
+    echo "$package" >> "$apt_file"
+  elif [[ "$current_section" == "pip" ]]; then
+    echo "$package" >> "$pip_file"
+  fi
+done < "$input_file"

main.py

@@ -2,6 +2,7 @@
 
 import argparse
 import subprocess
+import os
 
 def run_ansible_vault(action, filename, password_file):
     """Execute an ansible-vault command with the specified action on a file."""
@@ -32,6 +33,10 @@ def run_ansible_playbook(inventory:str, playbook:str, modes:[bool], limit:str=No
     subprocess.run(cmd, check=True)
 
 def main():
+    # Change to script dir to execute all folders relative to their
+    script_dir = os.path.dirname(os.path.realpath(__file__))
+    os.chdir(script_dir)
+    
     parser = argparse.ArgumentParser(description="CyMaIS Ansible Deployment and Vault Management")
     subparsers = parser.add_subparsers(dest="command", required=True)
 
@@ -72,7 +77,7 @@ def main():
         }
 
         # Use a fixed playbook file "playbook.yml"
-        run_ansible_playbook(args.inventory, "playbook.yml", modes, args.limit, args.password_file, args.verbose)
+        run_ansible_playbook(args.inventory, f"{script_dir}/playbook.yml", modes, args.limit, args.password_file, args.verbose)
 
 if __name__ == "__main__":
     main()

roles/docker-sphinx/tasks/main.yml

@@ -10,7 +10,7 @@
     state: directory
     mode: '0755'
 
-- name: pull the source repository to build the Sphinx documentation from {{ applications.sphinx.repository_sphinx_source }}
+- name: "pull the source repository to build the Sphinx documentation from {{ applications.sphinx.repository_sphinx_source }} to {{ host_sphinx_source_dir_absolute }}"
   git:
     repo: "{{ applications.sphinx.repository_sphinx_source }}"
     dest: "{{ host_sphinx_source_dir_absolute }}"
@@ -26,10 +26,10 @@
     domain: 	"{{ domains[application_id] }}"
     http_port: 	"{{ ports.localhost.http[application_id] }}"
 
-- name: "create {{ sphinx_docker_file }}"
+- name: "create {{ sphinx_host_dockerfile }}"
   copy:
-    src: "Dockerfile"
-    dest: "{{ sphinx_docker_file }}"
+    src:  "{{ sphinx_control_node_dockerfile }}"
+    dest: "{{ sphinx_host_dockerfile }}"
     mode: '770'
     force: yes
   notify: docker compose project build and setup

roles/docker-sphinx/templates/docker-compose.yml.j2

@@ -4,12 +4,12 @@ services:
       context:  .
       dockerfile: Dockerfile
       args:
-        SPHINX_SOURCE_DIR: {{docker_source_dir}}
-        SPHINX_BUILD_DIR: {{docker_output_dir}}
-        SPHINX_EXEC_DIR: {{docker_exec_dir}}
-        SPHINX_APP_DIR: {{docker_app_dir}}
+        SPHINX_SOURCE_DIR:          {{docker_source_dir}}
+        SPHINX_OUTPUT_DIR:          {{docker_output_dir}}
+        SPHINX_EXEC_DIR:            {{docker_exec_dir}}
+        SPHINX_DOCKER_EXEC_DIR:     {{docker_app_dir}}
         SPHINX_SOURCE_DIR_RELATIVE: {{host_sphinx_source_dir_relative}}
-        DOCKER_PYTHON_VERSION: {{applications[application_id].version}}
+        DOCKER_PYTHON_VERSION:      {{applications[application_id].version}}
     ports:
       - "127.0.0.1:{{ports.localhost.http[application_id]}}:8000"
     healthcheck:

roles/docker-sphinx/vars/main.yml

@@ -1,11 +1,12 @@
 application_id:                   "sphinx"
 
-host_sphinx_source_dir_relative:  "volumes/source/"                                                             # Place where the sphinx source repository is stored on the host
-host_sphinx_source_dir_absolute:  "{{docker_compose.directories.instance}}{{host_sphinx_source_dir_relative}}"  # Place where the sphinx source repository is stored on the host
+host_sphinx_source_dir_relative:  "volumes/source/"                                                                     # Place where the sphinx source repository is stored on the host
+host_sphinx_source_dir_absolute:  "{{docker_compose.directories.instance}}{{host_sphinx_source_dir_relative}}"          # Place where the sphinx source repository is stored on the host
 
-docker_app_dir:                   "/app/"                                                                       # Folder in which the application is running
-docker_source_dir:                "{{docker_app_dir}}"                                                          # Folder which is used to be screened
-docker_output_dir:                "/docs/"                                                                      # Folder to which the output is fuuuucking putted!
-docker_exec_dir:                  "{{docker_app_dir}}{{applications.sphinx.sphinx_exec_dir_relative}}"          # Folder which contains the sphinxs makefile and logic
+docker_app_dir:                   "/app/"                                                                               # Folder in which the application is running
+docker_source_dir:                "{{docker_app_dir}}"                                                                  # Folder which is used to be screened
+docker_output_dir:                "/output/"                                                                            # Folder to which the output is fuuuucking putted!
+docker_exec_dir:                  "{{ [ docker_app_dir, applications.sphinx.sphinx_exec_dir_relative ] | path_join }}"  # Folder which contains the sphinxs makefile and logic
 
-sphinx_docker_file:               "{{ docker_compose.directories.instance }}Dockerfile"                         # Path to the Dockerfile to build sphinx
+sphinx_host_dockerfile:           "{{ docker_compose.directories.instance }}Dockerfile"                                 # Path to the Dockerfile to build sphinx on the server
+sphinx_control_node_dockerfile:   "{{ [ playbook_dir, 'docs/Dockerfile' ] | path_join }}"                               # Path to the Dockerfile on the control node