mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2025-09-09 19:57:16 +02:00
Compare commits
21 Commits
7362accab0
...
39d2e6c0fa
Author | SHA1 | Date | |
---|---|---|---|
39d2e6c0fa | |||
69ad91ee91 | |||
5cd94c1d0a | |||
a0a61ad304 | |||
50c502d331 | |||
575df76ec3 | |||
db384c6261 | |||
2108702a2b | |||
66198ca1ec | |||
1f43536018 | |||
94bb060a5b | |||
8c411a21c7 | |||
3fdd900ed8 | |||
f548faa80f | |||
9668e74139 | |||
d0bd33fee3 | |||
ae5f021b8d | |||
dd1aab70fb | |||
e4ff99e336 | |||
ed0cd9b8c0 | |||
22b4342300 |
64
README.md
64
README.md
@@ -1,31 +1,50 @@
|
||||
|
||||
# CyMaIS - Cyber Master Infrastructure Solution 🚀
|
||||
# IT-Infrastructure Automation Framework 🚀
|
||||
|
||||
[](https://github.com/sponsors/kevinveenbirkenbach) [](https://www.patreon.com/c/kevinveenbirkenbach) [](https://buymeacoffee.com/kevinveenbirkenbach) [](https://s.veen.world/paypaldonate)
|
||||
|
||||
Welcome to **CyMaIS (Cyber Master Infrastructure Solution)**, a powerful automation framework that simplifies IT infrastructure setup and management. Whether you are an **end-user** looking to access cloud services securely or an **administrator** responsible for deploying and maintaining infrastructure, CyMaIS provides a seamless and secure solution.
|
||||
---
|
||||
|
||||

|
||||
|
||||
## What is CyMaIS? 📌
|
||||
CyMaIS leverages **Docker, Linux, and Ansible** to provide an automated and modular infrastructure solution. With more then **150 pre-configured roles**, it supports a wide range of applications, from cloud services to local server management and desktop workstation setups.
|
||||
|
||||
## Guides 📖
|
||||
- **[User Guide](docs/guides/user/Readme.md)** - For end-users accessing cloud apps like Nextcloud, Matrix, and more.
|
||||
- **[Administrator Guide](docs/guides/administrator/Readme.md)** - For system administrators deploying CyMaIS.
|
||||
- **[Customer Guide](docs/guides/customer/Readme.md)** - For customers which are interested in an infrastructure setup
|
||||
- **[Developer Guide](docs/guides/developer/index)** - For developers which are interested in participating
|
||||
- **[Investor Guide](docs/guides/investor/Readme.md)** - For investors which like to get a share in the project
|
||||
**CyMaIS** is an **automated, modular infrastructure framework** built on **Docker**, **Linux**, and **Ansible**, equally suited for cloud services, local server management, and desktop workstations. At its core lies a **web-based desktop with single sign-on**—backed by an **LDAP directory** and **OIDC**—granting **seamless access** to an almost limitless portfolio of self-hosted applications. It fully supports **ActivityPub applications** and is **Fediverse-compatible**, while integrated **monitoring**, **alerting**, **cleanup**, **self-healing**, **automated updates**, and **backup solutions** provide everything an organization needs to run at scale.
|
||||
|
||||
---
|
||||
|
||||
## Key Features 🎯
|
||||
- **Automated IT deployment** 📦 - Pre-built roles for server and PC setups
|
||||
- **Enterprise-ready security** 🔒 - Supports LDAP, Keycloak, 2FA, and encrypted storage
|
||||
- **Scalability & flexibility** 📈 - Modular approach for small teams to large enterprises
|
||||
- **Backup & recovery solutions** 💾 - Automate data security and prevent loss
|
||||
- **Infrastructure monitoring & maintenance** 📊 - Keep your system running optimally
|
||||
|
||||
* **Automated Deployment** 📦
|
||||
Turn up servers and workstations in minutes with ready-made Ansible roles.
|
||||
|
||||
* **Enterprise-Grade Security** 🔒
|
||||
Centralized user management via LDAP & OIDC (Keycloak), plus optional 2FA and encrypted storage.
|
||||
|
||||
* **Modular Scalability** 📈
|
||||
Grow from small teams to global enterprises by composing only the services you need.
|
||||
|
||||
* **Fediverse & ActivityPub Support** 🌐
|
||||
Seamlessly integrate Mastodon, Peertube, Matrix and other ActivityPub apps out of the box.
|
||||
|
||||
* **Self-Healing & Maintenance** ⚙️
|
||||
Automated cleanup, container healing, and auto-updates keep infrastructure healthy without human intervention.
|
||||
|
||||
* **Monitoring, Alerting & Analytics** 📊
|
||||
Built-in system, application, and security monitoring with multi-channel notifications.
|
||||
|
||||
* **Backup & Disaster Recovery** 💾
|
||||
Scheduled backups and scripted recovery processes to safeguard your data.
|
||||
|
||||
* **Continuous Updates** 🔄
|
||||
Automatic patching and version upgrades across the stack.
|
||||
|
||||
* **Application Ecosystem** 🚀
|
||||
A curated suite of self-hosted apps—from **project management**, **version control**, and **CI/CD** to **chat**, **video conferencing**, **CMS**, **e-learning**, **social networking**, and **e-commerce**—all seamlessly integrated.
|
||||
|
||||
More informations about the features you will find [here](docs/overview/Features.md).
|
||||
|
||||
---
|
||||
|
||||
## Get Started 🚀
|
||||
|
||||
### Use it online 🌐
|
||||
@@ -36,9 +55,20 @@ Give CyMaIS a spin at cymais.cloud – sign up in seconds, click around, and see
|
||||
1. **Install CyMaIS** via [Kevin's Package Manager](https://github.com/kevinveenbirkenbach/package-manager)
|
||||
2. **Setup CyMaIS** using:
|
||||
```sh
|
||||
pkgmgr setup cymais
|
||||
pkgmgr install cymais
|
||||
```
|
||||
3. **Explore Commands** with:
|
||||
```sh
|
||||
cymais --help
|
||||
```
|
||||
```
|
||||
---
|
||||
|
||||
## License ⚖️
|
||||
|
||||
CyMaIS is distributed under the **CyMaIS NonCommercial License**. Please see [LICENSE.md](LICENSE.md) for full terms.
|
||||
|
||||
---
|
||||
|
||||
## Professional Setup & Support 💼
|
||||
|
||||
For expert installation and configuration visit [cybermaster.space](https://cybermaster.space/) or write to us at **[contact@cymais.cloud](mailto:contact@cymais.cloud)**.
|
||||
|
@@ -47,7 +47,7 @@ def main():
|
||||
for role_dir in sorted(roles_dir.iterdir()):
|
||||
role_name = role_dir.name
|
||||
vars_main = role_dir / "vars" / "main.yml"
|
||||
config_file = role_dir / "vars" / "configuration.yml"
|
||||
config_file = role_dir / "config" / "main.yml"
|
||||
|
||||
if not vars_main.exists():
|
||||
print(f"[!] Skipping {role_name}: vars/main.yml missing")
|
||||
@@ -68,7 +68,7 @@ def main():
|
||||
continue
|
||||
|
||||
if not config_file.exists():
|
||||
print(f"[!] Skipping {role_name}: vars/configuration.yml missing")
|
||||
print(f"[!] Skipping {role_name}: config/main.yml missing")
|
||||
continue
|
||||
|
||||
config_data = load_yaml_file(config_file)
|
||||
|
@@ -113,14 +113,21 @@ def generate_playbook_entries(roles_dir, prefix=None):
|
||||
entries = []
|
||||
for role_name in sorted_role_names:
|
||||
role = roles[role_name]
|
||||
|
||||
# --- new validation block ---
|
||||
if role.get('application_id') is None:
|
||||
raise ValueError(f"Role '{role_name}' is missing an application_id")
|
||||
# ----------------------------
|
||||
|
||||
app_id = role['application_id']
|
||||
entries.append(
|
||||
f"- name: setup {role['application_id']}\n"
|
||||
f" when: ('{role['application_id']}' | application_allowed(group_names, allowed_applications))\n"
|
||||
f"- name: setup {app_id}\n"
|
||||
f" when: ('{app_id}' | application_allowed(group_names, allowed_applications))\n"
|
||||
f" include_role:\n"
|
||||
f" name: {role['role_name']}\n"
|
||||
)
|
||||
entries.append(
|
||||
f"- name: flush handlers after {role['application_id']}\n"
|
||||
f"- name: flush handlers after {app_id}\n"
|
||||
f" meta: flush_handlers\n"
|
||||
)
|
||||
|
||||
@@ -136,7 +143,7 @@ def main():
|
||||
)
|
||||
parser.add_argument(
|
||||
'-p', '--prefix',
|
||||
help='Only include roles whose names start with this prefix (e.g. web-app-, desktop-)',
|
||||
help='Only include roles whose names start with this prefix (e.g. web-app-, desk-)',
|
||||
default=None
|
||||
)
|
||||
parser.add_argument(
|
||||
|
@@ -110,7 +110,7 @@ def build_users(defs, primary_domain, start_id, become_pwd):
|
||||
|
||||
def load_user_defs(roles_directory):
|
||||
"""
|
||||
Scan all roles/*/meta/users.yml files and merge any 'users:' sections.
|
||||
Scan all roles/*/users/main.yml files and merge any 'users:' sections.
|
||||
|
||||
Args:
|
||||
roles_directory (str): Path to the directory containing role subdirectories.
|
||||
@@ -121,7 +121,7 @@ def load_user_defs(roles_directory):
|
||||
Raises:
|
||||
ValueError: On invalid format or conflicting override values.
|
||||
"""
|
||||
pattern = os.path.join(roles_directory, '*/meta/users.yml')
|
||||
pattern = os.path.join(roles_directory, '*/users/main.yml')
|
||||
files = sorted(glob.glob(pattern))
|
||||
merged = OrderedDict()
|
||||
|
||||
@@ -165,11 +165,11 @@ def dictify(data):
|
||||
|
||||
def parse_args():
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Generate a users.yml by merging all roles/*/meta/users.yml definitions.'
|
||||
description='Generate a users.yml by merging all roles/*/users/main.yml definitions.'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--roles-dir', '-r', required=True,
|
||||
help='Directory containing roles (e.g., roles/*/meta/users.yml).'
|
||||
help='Directory containing roles (e.g., roles/*/users/main.yml).'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--output', '-o', required=True,
|
||||
|
@@ -17,7 +17,7 @@ class InventoryManager:
|
||||
self.vault_pw = vault_pw
|
||||
self.overrides = overrides
|
||||
self.inventory = YamlHandler.load_yaml(inventory_path)
|
||||
self.schema = YamlHandler.load_yaml(role_path / "meta" / "schema.yml")
|
||||
self.schema = YamlHandler.load_yaml(role_path / "schema" / "main.yml")
|
||||
self.app_id = self.load_application_id(role_path)
|
||||
|
||||
self.vault_handler = VaultHandler(vault_pw)
|
||||
@@ -38,7 +38,7 @@ class InventoryManager:
|
||||
target = apps.setdefault(self.app_id, {})
|
||||
|
||||
# Load the data from vars/main.yml
|
||||
vars_file = self.role_path / "vars" / "configuration.yml"
|
||||
vars_file = self.role_path / "config" / "main.yml"
|
||||
data = YamlHandler.load_yaml(vars_file)
|
||||
|
||||
# Check if 'central-database' is enabled in the features section of data
|
||||
|
@@ -12,10 +12,10 @@ CyMaIS covers everything from essential system setups to advanced configurations
|
||||
Every business is unique, and so is CyMaIS! With a modular architecture, it adapts to specific requirements, whether for startups, growing businesses, NGOs, or large enterprises.
|
||||
|
||||
## Proactive Monitoring & Maintenance 🔍
|
||||
With automated updates, system health checks, and security audits, CyMaIS ensures your infrastructure is always up-to-date and running smoothly. Roles such as `monitor-bot-docker-container`, `monitor-bot-btrfs`, and `monitor-bot-webserver` help monitor system integrity.
|
||||
With automated updates, system health checks, and security audits, CyMaIS ensures your infrastructure is always up-to-date and running smoothly. Roles such as `mon-bot-docker-container`, `mon-bot-btrfs`, and `mon-bot-webserver` help monitor system integrity.
|
||||
|
||||
## Uncompromised Security 🔒
|
||||
Security is a top priority! CyMaIS includes robust security features like full-disk encryption recommendations, 2FA enforcement, encrypted server deployments (`web-app-keycloak`, `service-openldap`), and secure backup solutions (`backup-remote-to-local`, `backup-data-to-usb`).
|
||||
Security is a top priority! CyMaIS includes robust security features like full-disk encryption recommendations, 2FA enforcement, encrypted server deployments (`web-app-keycloak`, `svc-openldap`), and secure backup solutions (`bkp-remote-to-local`, `bkp-data-to-usb`).
|
||||
|
||||
## User-Friendly with Expert Support 👩💻
|
||||
No need to be a Linux or Docker expert! CyMaIS simplifies deployment with intuitive role-based automation. Documentation and community support make IT administration accessible to all experience levels.
|
||||
|
@@ -10,13 +10,13 @@ Follow these guides to install and configure CyMaIS:
|
||||
|
||||
## Key Responsibilities 🔧
|
||||
- **User Management** - Configure LDAP, Keycloak, and user permissions.
|
||||
- **Security & Backups** - Set up `backup-remote-to-local`, `backup-data-to-usb`, and `core-security` roles.
|
||||
- **Security & Backups** - Set up `bkp-remote-to-local`, `bkp-data-to-usb`, and `core-security` roles.
|
||||
- **Application Hosting** - Deploy services like `Nextcloud`, `Matrix`, `Gitea`, and more.
|
||||
- **Networking & VPN** - Configure `WireGuard`, `OpenVPN`, and `Nginx Reverse Proxy`.
|
||||
|
||||
## Managing & Updating CyMaIS 🔄
|
||||
- Regularly update services using `update-docker`, `update-pacman`, or `update-apt`.
|
||||
- Monitor system health with `monitor-bot-btrfs`, `monitor-bot-webserver`, and `monitor-bot-docker-container`.
|
||||
- Automate system maintenance with `maintenance-lock`, `cleanup-backups-service`, and `maintenance-docker-restart`.
|
||||
- Monitor system health with `mon-bot-btrfs`, `mon-bot-webserver`, and `mon-bot-docker-container`.
|
||||
- Automate system maintenance with `maint-lock`, `cln-backups-service`, and `maint-docker-restart`.
|
||||
|
||||
For more details, refer to the specific guides above.
|
@@ -91,9 +91,9 @@ Now that you have defined the application settings, domain, and application ID,
|
||||
command: pkgmgr path cymais-my_service
|
||||
register: path_cymais_my_service_output
|
||||
|
||||
- name: "include role webserver-proxy-domain for {{ application_id }}"
|
||||
- name: "include role srv-web-proxy-domain for {{ application_id }}"
|
||||
include_role:
|
||||
name: webserver-proxy-domain
|
||||
name: srv-web-proxy-domain
|
||||
vars:
|
||||
domain: "{{ domains | get_domain(application_id) }}"
|
||||
http_port: "{{ ports.localhost.http[application_id] }}"
|
||||
|
@@ -30,10 +30,10 @@ def load_configuration(application_id, key):
|
||||
except Exception:
|
||||
md = {}
|
||||
if md.get('application_id') == application_id:
|
||||
cf = os.path.join(roles_dir, role, 'vars', 'configuration.yml')
|
||||
cf = os.path.join(roles_dir, role, "config" , "main.yml")
|
||||
if not os.path.exists(cf):
|
||||
raise AnsibleFilterError(
|
||||
f"Role '{role}' declares '{application_id}' but missing configuration.yml"
|
||||
f"Role '{role}' declares '{application_id}' but missing config/main.yml"
|
||||
)
|
||||
config_path = cf
|
||||
break
|
||||
@@ -41,7 +41,7 @@ def load_configuration(application_id, key):
|
||||
# 2) fallback nested
|
||||
if config_path is None:
|
||||
for role in os.listdir(roles_dir):
|
||||
cf = os.path.join(roles_dir, role, 'vars', 'configuration.yml')
|
||||
cf = os.path.join(roles_dir, role, "config" , "main.yml")
|
||||
if not os.path.exists(cf):
|
||||
continue
|
||||
try:
|
||||
@@ -55,7 +55,7 @@ def load_configuration(application_id, key):
|
||||
# 3) fallback flat
|
||||
if config_path is None:
|
||||
for role in os.listdir(roles_dir):
|
||||
cf = os.path.join(roles_dir, role, 'vars', 'configuration.yml')
|
||||
cf = os.path.join(roles_dir, role, "config" , "main.yml")
|
||||
if not os.path.exists(cf):
|
||||
continue
|
||||
try:
|
||||
@@ -74,7 +74,7 @@ def load_configuration(application_id, key):
|
||||
try:
|
||||
parsed = yaml.safe_load(open(config_path)) or {}
|
||||
except Exception as e:
|
||||
raise AnsibleFilterError(f"Error loading configuration.yml at {config_path}: {e}")
|
||||
raise AnsibleFilterError(f"Error loading config/main.yml at {config_path}: {e}")
|
||||
|
||||
# detect nested vs flat
|
||||
is_nested = isinstance(parsed, dict) and (application_id in parsed)
|
||||
|
@@ -1,4 +1,3 @@
|
||||
# roles/<your-role>/filter_plugins/redirect_filters.py
|
||||
from ansible.errors import AnsibleFilterError
|
||||
|
||||
class FilterModule(object):
|
||||
|
@@ -13,23 +13,23 @@ system_maintenance_lock_timeout_restart_docker: "{{system_maintenance_lock_ti
|
||||
|
||||
### Defined Services for Backup Tasks
|
||||
system_maintenance_backup_services:
|
||||
- "backup-docker-to-local"
|
||||
- "backup-remote-to-local"
|
||||
- "backup-data-to-usb"
|
||||
- "backup-docker-to-local-everything"
|
||||
- "bkp-docker-to-local"
|
||||
- "bkp-remote-to-local"
|
||||
- "bkp-data-to-usb"
|
||||
- "bkp-docker-to-local-everything"
|
||||
|
||||
### Defined Services for System Cleanup
|
||||
system_maintenance_cleanup_services:
|
||||
- "cleanup-backups"
|
||||
- "cleanup-disc-space"
|
||||
- "cleanup-failed-docker-backups"
|
||||
- "cln-backups"
|
||||
- "cln-disc-space"
|
||||
- "cln-failed-docker-backups"
|
||||
|
||||
### Services that Manipulate the System
|
||||
system_maintenance_manipulation_services:
|
||||
- "maintenance-docker-heal"
|
||||
- "maint-docker-heal"
|
||||
- "update-docker"
|
||||
- "maintenance-docker-storage-optimizer"
|
||||
- "maintenance-docker-restart"
|
||||
- "maint-docker-storage-optimizer"
|
||||
- "maint-docker-restart"
|
||||
|
||||
## Total System Maintenance Services
|
||||
system_maintenance_services: "{{ system_maintenance_backup_services + system_maintenance_cleanup_services + system_maintenance_manipulation_services }}"
|
||||
|
@@ -3,4 +3,4 @@
|
||||
path_administrator_home: "/home/administrator/"
|
||||
path_administrator_scripts: "/opt/scripts/"
|
||||
path_docker_compose_instances: "/opt/docker/"
|
||||
path_system_lock_script: "/opt/scripts/maintenance-lock.py"
|
||||
path_system_lock_script: "/opt/scripts/maint-lock.py"
|
@@ -14,32 +14,32 @@ For a complete list of role categories and detailed definitions, see:
|
||||
- **core-***
|
||||
Fundamental system configuration (SSH, journald, sudo, etc.)
|
||||
|
||||
- **generic-***
|
||||
Generic helpers and language/tool installers (e.g. `generic-git`, `generic-locales`, `generic-timer`)
|
||||
- **gen-***
|
||||
Generic helpers and language/tool installers (e.g. `gen-git`, `gen-locales`, `gen-timer`)
|
||||
|
||||
- **desktop-***
|
||||
Desktop environment and application roles (e.g. `desktop-gnome`, `desktop-browser`, `desktop-libreoffice`)
|
||||
- **desk-***
|
||||
Desktop environment and application roles (e.g. `desk-gnome`, `utils-desk-browser`, `desk-libreoffice`)
|
||||
|
||||
---
|
||||
|
||||
## Webserver & HTTP
|
||||
|
||||
- **webserver-core**
|
||||
- **srv-web-core**
|
||||
Installs and configures the base Nginx server.
|
||||
|
||||
- **webserver-tls-***
|
||||
- **srv-web-tls-***
|
||||
Manages TLS certificates and renewal (formerly “https”).
|
||||
|
||||
- **webserver-proxy-***
|
||||
- **srv-web-proxy-***
|
||||
Proxy and vhost orchestration roles (domain setup, OAuth2 proxy, etc.)
|
||||
|
||||
- **webserver-injector-***
|
||||
- **srv-web-injector-***
|
||||
HTML response modifiers: CSS, JS, Matomo tracking, iframe notifier.
|
||||
|
||||
- **webserver-composer**
|
||||
- **srv-web-composer**
|
||||
Aggregates multiple sub-filters into one include for your vhost.
|
||||
|
||||
- **web-service-***
|
||||
- **web-svc-***
|
||||
Static‐content servers (assets, HTML, legal pages, file hosting).
|
||||
|
||||
- **web-app-***
|
||||
@@ -49,17 +49,17 @@ For a complete list of role categories and detailed definitions, see:
|
||||
|
||||
## Network
|
||||
|
||||
- **network-***
|
||||
- **net-***
|
||||
Network infrastructure (DNS records, WireGuard, Let’s Encrypt entrypoints).
|
||||
|
||||
- **service-***
|
||||
- **svc-***
|
||||
Docker‐deployed services that aren’t “apps” (RDBMS, LDAP, Redis, OpenLDAP).
|
||||
|
||||
---
|
||||
|
||||
## Monitoring & Alerting
|
||||
|
||||
- **monitor-bot-***
|
||||
- **mon-bot-***
|
||||
“Bot”-style health checks with alerts via Telegram, email, etc.
|
||||
|
||||
- **monitor-core-***
|
||||
@@ -72,20 +72,20 @@ For a complete list of role categories and detailed definitions, see:
|
||||
|
||||
## Maintenance & Healing
|
||||
|
||||
- **maintenance-***
|
||||
- **maint-***
|
||||
Periodic maintenance tasks (Btrfs balancing, swapfile management).
|
||||
|
||||
- **maintenance-docker-***
|
||||
- **maint-docker-***
|
||||
Automated recovery and restarts for Docker Compose workloads.
|
||||
|
||||
- **cleanup-***
|
||||
- **cln-***
|
||||
Housekeeping tasks (old backups, certs, log rotation).
|
||||
|
||||
---
|
||||
|
||||
## Backup & Restore
|
||||
|
||||
- **backup-***
|
||||
- **bkp-***
|
||||
Local and remote backup strategies for files, Docker volumes, databases.
|
||||
|
||||
---
|
||||
@@ -111,9 +111,9 @@ For a complete list of role categories and detailed definitions, see:
|
||||
---
|
||||
|
||||
> **Tip:** To find a role quickly, search for its prefix:
|
||||
> `core-`, `generic-`, `desktop-`, `webserver-`, `web-service-`, `web-app-`,
|
||||
> `network-`, `service-`, `monitor-`, `alert-`, `maintenance-`, `cleanup-`,
|
||||
> `backup-`, `update-`, `pkgmgr-`, `user-`.
|
||||
> `core-`, `gen-`, `desk-`, `srv-web-`, `web-svc-`, `web-app-`,
|
||||
> `net-`, `svc-`, `monitor-`, `alert-`, `maint-`, `cln-`,
|
||||
> `bkp-`, `update-`, `pkgmgr-`, `user-`.
|
||||
|
||||
---
|
||||
|
||||
|
@@ -23,4 +23,4 @@ galaxy_info:
|
||||
issue_tracker_url: "https://s.veen.world/cymaisissues"
|
||||
documentation: "https://s.veen.world/cymais"
|
||||
dependencies:
|
||||
- generic-msmtp
|
||||
- gen-msmtp
|
||||
|
@@ -1,5 +0,0 @@
|
||||
- name: "reload backup-data-to-usb.cymais.service"
|
||||
systemd:
|
||||
name: backup-data-to-usb.cymais.service
|
||||
state: reloaded
|
||||
daemon_reload: yes
|
@@ -1,9 +0,0 @@
|
||||
- name: "reload backup-docker-to-local-everything.cymais.service"
|
||||
systemd:
|
||||
name: backup-docker-to-local-everything.cymais.service
|
||||
daemon_reload: yes
|
||||
|
||||
- name: "reload backup-docker-to-local.cymais.service"
|
||||
systemd:
|
||||
name: backup-docker-to-local.cymais.service
|
||||
daemon_reload: yes
|
@@ -1,4 +0,0 @@
|
||||
- name: "reload backup-remote-to-local service"
|
||||
systemd:
|
||||
name: backup-remote-to-local.cymais.service
|
||||
daemon_reload: yes
|
@@ -21,7 +21,7 @@ def main():
|
||||
machine_id = subprocess.run(["sha256sum", "/etc/machine-id"], capture_output=True, text=True).stdout.strip()[:64]
|
||||
print(f"machine id: {machine_id}")
|
||||
|
||||
versions_path = os.path.join(backup_to_usb_destination_path, f"{machine_id}/backup-data-to-usb/")
|
||||
versions_path = os.path.join(backup_to_usb_destination_path, f"{machine_id}/bkp-data-to-usb/")
|
||||
print(f"versions path: {versions_path}")
|
||||
|
||||
if not os.path.isdir(versions_path):
|
5
roles/bkp-data-to-usb/handlers/main.yml
Normal file
5
roles/bkp-data-to-usb/handlers/main.yml
Normal file
@@ -0,0 +1,5 @@
|
||||
- name: "reload bkp-data-to-usb.cymais.service"
|
||||
systemd:
|
||||
name: bkp-data-to-usb.cymais.service
|
||||
state: reloaded
|
||||
daemon_reload: yes
|
@@ -23,5 +23,5 @@ galaxy_info:
|
||||
issue_tracker_url: https://s.veen.world/cymaisissues
|
||||
documentation: https://s.veen.world/cymais
|
||||
dependencies:
|
||||
- cleanup-backups-service
|
||||
- maintenance-lock
|
||||
- cln-backups-service
|
||||
- maint-lock
|
@@ -1,6 +1,6 @@
|
||||
- name: Copy backup script to the scripts directory
|
||||
copy:
|
||||
src: backup-data-to-usb.python
|
||||
src: bkp-data-to-usb.python
|
||||
dest: "{{ backup_to_usb_script_path }}"
|
||||
owner: root
|
||||
group: root
|
||||
@@ -8,9 +8,9 @@
|
||||
|
||||
- name: Copy systemd service to systemd directory
|
||||
template:
|
||||
src: backup-data-to-usb.service.j2
|
||||
dest: /etc/systemd/system/backup-data-to-usb.cymais.service
|
||||
src: bkp-data-to-usb.service.j2
|
||||
dest: /etc/systemd/system/bkp-data-to-usb.cymais.service
|
||||
owner: root
|
||||
group: root
|
||||
mode: '0644'
|
||||
notify: reload backup-data-to-usb.cymais.service
|
||||
notify: reload bkp-data-to-usb.cymais.service
|
@@ -6,7 +6,7 @@ OnFailure=alert-compose.cymais@%n.service
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/bin/python {{ backup_to_usb_script_path }} {{backup_to_usb_source}} {{backup_to_usb_destination}}
|
||||
ExecStartPost=/bin/systemctl start cleanup-backups.cymais.service
|
||||
ExecStartPost=/bin/systemctl start cln-backups.cymais.service
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
@@ -1,4 +1,4 @@
|
||||
backup_to_usb_script_path: "/usr/local/sbin/backup-data-to-usb.python"
|
||||
backup_to_usb_script_path: "/usr/local/sbin/bkp-data-to-usb.python"
|
||||
backup_to_usb_destination: "{{backup_to_usb_mount}}{{backup_to_usb_destination_subdirectory}}"
|
||||
backups_folder_path: "{{backup_to_usb_destination}}"
|
||||
systemctl_mount_service_name: "{{ backup_to_usb_mount | trim('/') | replace('/', '-') }}.mount"
|
@@ -2,7 +2,7 @@
|
||||
|
||||
## Description
|
||||
|
||||
This Ansible role installs the [directory-validator](https://github.com/kevinveenbirkenbach/directory-validator.git). It is used by the backup-docker-to-local and cleanup-failed-docker-backups roles to verify whether backups have been successfully created.
|
||||
This Ansible role installs the [directory-validator](https://github.com/kevinveenbirkenbach/directory-validator.git). It is used by the bkp-docker-to-local and cln-failed-docker-backups roles to verify whether backups have been successfully created.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -16,5 +16,5 @@ The primary purpose of this role is to validate backup directories. By fetching
|
||||
|
||||
- **Git Repository Pull:** Automatically pulls the latest version of the directory-validator from Git.
|
||||
- **Idempotent Execution:** Ensures that the repository is fetched only once per playbook run.
|
||||
- **Backup Verification:** Serves as a validation step for other backup-related roles.
|
||||
- **Backup Verification:** Serves as a validation step for other bkp-related roles.
|
||||
- **Configurable Destination:** The target folder is customizable via the `backup_directory_validator_folder` variable.
|
@@ -2,16 +2,16 @@
|
||||
|
||||
## Description
|
||||
|
||||
This Ansible role automates the process of backing up Docker volumes to a local folder. It pulls the [backup-docker-to-local repository](https://github.com/kevinveenbirkenbach/backup-docker-to-local.git), installs required software, configures systemd services for both standard and "everything" backup modes, and seeds backup database entries as needed.
|
||||
This Ansible role automates the process of backing up Docker volumes to a local folder. It pulls the [bkp-docker-to-local repository](https://github.com/kevinveenbirkenbach/bkp-docker-to-local.git), installs required software, configures systemd services for both standard and "everything" backup modes, and seeds backup database entries as needed.
|
||||
|
||||
## Overview
|
||||
|
||||
Optimized for Archlinux, this role ensures that Docker volume backups are performed reliably with minimal manual intervention. It integrates with several dependent roles to verify backup success and manage related tasks, including:
|
||||
- [backup-directory-validator](../backup-directory-validator/) – Validates backup directories.
|
||||
- [cleanup-failed-docker-backups](../cleanup-failed-docker-backups/) – Cleans up unsuccessful backup attempts.
|
||||
- [generic-timer](../generic-timer/) – Schedules recurring backup tasks.
|
||||
- [backup-provider](../backup-provider/) – Manages backup sources.
|
||||
- [maintenance-lock](../maintenance-lock/) – Ensures coordinated maintenance operations.
|
||||
- [bkp-directory-validator](../bkp-directory-validator/) – Validates backup directories.
|
||||
- [cln-failed-docker-backups](../cln-failed-docker-backups/) – Cleans up unsuccessful backup attempts.
|
||||
- [gen-timer](../gen-timer/) – Schedules recurring backup tasks.
|
||||
- [bkp-provider](../bkp-provider/) – Manages backup sources.
|
||||
- [maint-lock](../maint-lock/) – Ensures coordinated maintenance operations.
|
||||
|
||||
## Purpose
|
||||
|
||||
@@ -20,7 +20,7 @@ Backup Docker Volumes to Local is a comprehensive solution that leverages rsync
|
||||
## Features
|
||||
|
||||
- **Required Software Installation:** Installs necessary packages (e.g., lsof, python-pandas) via pacman.
|
||||
- **Git Repository Pull:** Automatically pulls the latest version of the [backup-docker-to-local repository](https://github.com/kevinveenbirkenbach/backup-docker-to-local.git).
|
||||
- **Git Repository Pull:** Automatically pulls the latest version of the [bkp-docker-to-local repository](https://github.com/kevinveenbirkenbach/bkp-docker-to-local.git).
|
||||
- **Systemd Service Configuration:** Deploys and reloads two systemd service templates to manage backup tasks.
|
||||
- **Database Seeding:** Includes tasks to seed and manage a backup database (`databases.csv`) for tracking backup details.
|
||||
- **Dependency Integration:** Works in conjunction with the dependent roles listed above to verify and manage backups.
|
9
roles/bkp-docker-to-local/handlers/main.yml
Normal file
9
roles/bkp-docker-to-local/handlers/main.yml
Normal file
@@ -0,0 +1,9 @@
|
||||
- name: "reload bkp-docker-to-local-everything.cymais.service"
|
||||
systemd:
|
||||
name: bkp-docker-to-local-everything.cymais.service
|
||||
daemon_reload: yes
|
||||
|
||||
- name: "reload bkp-docker-to-local.cymais.service"
|
||||
systemd:
|
||||
name: bkp-docker-to-local.cymais.service
|
||||
daemon_reload: yes
|
@@ -23,8 +23,8 @@ galaxy_info:
|
||||
issue_tracker_url: "https://s.veen.world/cymaisissues"
|
||||
documentation: "https://s.veen.world/cymais"
|
||||
dependencies:
|
||||
- backup-provider
|
||||
- bkp-provider
|
||||
- alert-compose
|
||||
- cleanup-failed-docker-backups
|
||||
- maintenance-lock
|
||||
- backup-directory-validator
|
||||
- cln-failed-docker-backups
|
||||
- maint-lock
|
||||
- bkp-directory-validator
|
@@ -2,11 +2,11 @@
|
||||
include_role:
|
||||
name: pkgmgr-install
|
||||
vars:
|
||||
package_name: backup-docker-to-local
|
||||
package_name: bkp-docker-to-local
|
||||
when: run_once_backup_docker_to_local is not defined
|
||||
|
||||
- name: Retrieve backup-docker-to-local path from pkgmgr
|
||||
command: pkgmgr path backup-docker-to-local
|
||||
- name: Retrieve bkp-docker-to-local path from pkgmgr
|
||||
command: pkgmgr path bkp-docker-to-local
|
||||
register: pkgmgr_output
|
||||
changed_when: false
|
||||
when: run_once_backup_docker_to_local is not defined
|
||||
@@ -17,18 +17,18 @@
|
||||
changed_when: false
|
||||
when: run_once_backup_docker_to_local is not defined
|
||||
|
||||
- name: configure backup-docker-to-local-everything.cymais.service
|
||||
- name: configure bkp-docker-to-local-everything.cymais.service
|
||||
template:
|
||||
src: backup-docker-to-local-everything.service.j2
|
||||
dest: /etc/systemd/system/backup-docker-to-local-everything.cymais.service
|
||||
notify: reload backup-docker-to-local-everything.cymais.service
|
||||
src: bkp-docker-to-local-everything.service.j2
|
||||
dest: /etc/systemd/system/bkp-docker-to-local-everything.cymais.service
|
||||
notify: reload bkp-docker-to-local-everything.cymais.service
|
||||
when: run_once_backup_docker_to_local is not defined
|
||||
|
||||
- name: configure backup-docker-to-local.cymais.service
|
||||
- name: configure bkp-docker-to-local.cymais.service
|
||||
template:
|
||||
src: backup-docker-to-local.service.j2
|
||||
dest: /etc/systemd/system/backup-docker-to-local.cymais.service
|
||||
notify: reload backup-docker-to-local.cymais.service
|
||||
src: bkp-docker-to-local.service.j2
|
||||
dest: /etc/systemd/system/bkp-docker-to-local.cymais.service
|
||||
notify: reload bkp-docker-to-local.cymais.service
|
||||
when: run_once_backup_docker_to_local is not defined
|
||||
|
||||
- name: set service_name to the name of the current role
|
||||
@@ -36,9 +36,9 @@
|
||||
service_name: "{{ role_name }}"
|
||||
when: run_once_backup_docker_to_local is not defined
|
||||
|
||||
- name: "include role for generic-timer for {{service_name}}"
|
||||
- name: "include role for gen-timer for {{service_name}}"
|
||||
include_role:
|
||||
name: generic-timer
|
||||
name: gen-timer
|
||||
vars:
|
||||
on_calendar: "{{on_calendar_backup_docker_to_local}}"
|
||||
when: run_once_backup_docker_to_local is not defined
|
@@ -1,9 +1,9 @@
|
||||
[Unit]
|
||||
Description=backup docker volumes to local folder
|
||||
OnFailure=alert-compose.cymais@%n.service cleanup-failed-docker-backups.cymais.service
|
||||
OnFailure=alert-compose.cymais@%n.service cln-failed-docker-backups.cymais.service
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStartPre=/bin/sh -c '/usr/bin/python {{ path_system_lock_script }} {{ system_maintenance_services | join(' ') }} --ignore {{ system_maintenance_backup_services | reject('equalto', 'backup-docker-to-local') | join(' ') }} --timeout "{{system_maintenance_lock_timeout_backup_services}}"'
|
||||
ExecStartPre=/bin/sh -c '/usr/bin/python {{ path_system_lock_script }} {{ system_maintenance_services | join(' ') }} --ignore {{ system_maintenance_backup_services | reject('equalto', 'bkp-docker-to-local') | join(' ') }} --timeout "{{system_maintenance_lock_timeout_backup_services}}"'
|
||||
ExecStart=/bin/sh -c '/usr/bin/python {{backup_docker_to_local_folder}}backup-docker-to-local.py --compose-dir {{path_docker_compose_instances}} --everything'
|
||||
ExecStartPost=/bin/sh -c '/bin/systemctl start maintenance-docker-heal.cymais.service &'
|
||||
ExecStartPost=/bin/sh -c '/bin/systemctl start maint-docker-heal.cymais.service &'
|
@@ -1,9 +1,9 @@
|
||||
[Unit]
|
||||
Description=backup docker volumes to local folder
|
||||
OnFailure=alert-compose.cymais@%n.service cleanup-failed-docker-backups.cymais.service
|
||||
OnFailure=alert-compose.cymais@%n.service cln-failed-docker-backups.cymais.service
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStartPre=/bin/sh -c '/usr/bin/python {{ path_system_lock_script }} {{ system_maintenance_services | join(' ') }} --ignore {{ system_maintenance_backup_services | reject('equalto', 'backup-docker-to-local-everything') | join(' ') }} --timeout "{{system_maintenance_lock_timeout_backup_services}}"'
|
||||
ExecStartPre=/bin/sh -c '/usr/bin/python {{ path_system_lock_script }} {{ system_maintenance_services | join(' ') }} --ignore {{ system_maintenance_backup_services | reject('equalto', 'bkp-docker-to-local-everything') | join(' ') }} --timeout "{{system_maintenance_lock_timeout_backup_services}}"'
|
||||
ExecStart=/bin/sh -c '/usr/bin/python {{backup_docker_to_local_folder}}backup-docker-to-local.py --compose-dir {{path_docker_compose_instances}}'
|
||||
ExecStartPost=/bin/sh -c '/bin/systemctl start maintenance-docker-heal.cymais.service &'
|
||||
ExecStartPost=/bin/sh -c '/bin/systemctl start maint-docker-heal.cymais.service &'
|
@@ -12,8 +12,8 @@ hashed_machine_id="$($get_hashed_machine_id | head -c 64)"
|
||||
get_backup_types="find /Backups/$hashed_machine_id/ -maxdepth 1 -type d -execdir basename {} ;";
|
||||
|
||||
|
||||
# @todo This configuration is not scalable yet. If other backup services then backup-docker-to-local are integrated, this logic needs to be optimized
|
||||
get_version_directories="ls -d /Backups/$hashed_machine_id/backup-docker-to-local/*"
|
||||
# @todo This configuration is not scalable yet. If other backup services then bkp-docker-to-local are integrated, this logic needs to be optimized
|
||||
get_version_directories="ls -d /Backups/$hashed_machine_id/bkp-docker-to-local/*"
|
||||
last_version_directory="$($get_version_directories | tail -1)"
|
||||
rsync_command="sudo rsync --server --sender -blogDtpre.iLsfxCIvu . $last_version_directory/"
|
||||
|
@@ -8,7 +8,7 @@ This role sets up and manages the host as a backup provider. It establishes the
|
||||
|
||||
Optimized for automated backup processes, this role:
|
||||
- Configures the host to provide backup services.
|
||||
- Integrates seamlessly with the [backup-provider-user](../backup-provider-user/README.md) and [cleanup-backups-timer](../cleanup-backups-timer/README.md) roles.
|
||||
- Integrates seamlessly with the [bkp-provider-user](../bkp-provider-user/README.md) and [cln-backups-timer](../cln-backups-timer/README.md) roles.
|
||||
- Lays the foundation for secure and extensible backup operations.
|
||||
|
||||
## Purpose
|
@@ -23,5 +23,5 @@ galaxy_info:
|
||||
issue_tracker_url: "https://s.veen.world/cymaisissues"
|
||||
documentation: "https://s.veen.world/cymais"
|
||||
dependencies:
|
||||
- backup-provider-user
|
||||
- cleanup-backups-timer
|
||||
- bkp-provider-user
|
||||
- cln-backups-timer
|
@@ -9,17 +9,17 @@ To track what the service is doing, execute one of the following commands:
|
||||
#### Using systemctl
|
||||
|
||||
```bash
|
||||
watch -n2 "systemctl status backup-remote-to-local.cymais.service"
|
||||
watch -n2 "systemctl status bkp-remote-to-local.cymais.service"
|
||||
```
|
||||
|
||||
#### Using journalctl
|
||||
|
||||
```bash
|
||||
journalctl -fu backup-remote-to-local.cymais.service
|
||||
journalctl -fu bkp-remote-to-local.cymais.service
|
||||
```
|
||||
|
||||
### Viewing History
|
||||
|
||||
```bash
|
||||
sudo journalctl -u backup-remote-to-local.cymais.service
|
||||
sudo journalctl -u bkp-remote-to-local.cymais.service
|
||||
```
|
@@ -17,14 +17,14 @@ Backup Remote to Local is a robust solution for retrieving backup data from remo
|
||||
- **Remote Backup Retrieval:** Pulls backups from a remote server using secure SSH connections.
|
||||
- **Incremental Backup with rsync:** Uses rsync with options for archive, backup, and hard linking to efficiently manage changes.
|
||||
- **Retry Logic:** Implements a retry mechanism to handle transient network issues or remote errors.
|
||||
- **Integration with Other Roles:** Works alongside roles like backup-directory-validator, cleanup-failed-docker-backups, generic-timer, backup-provider, and maintenance-lock.
|
||||
- **Integration with Other Roles:** Works alongside roles like bkp-directory-validator, cln-failed-docker-backups, gen-timer, bkp-provider, and maint-lock.
|
||||
- **Administrative Debugging:** Detailed debug instructions and administrative tasks are provided in a separate file.
|
||||
|
||||
## Other Resources
|
||||
|
||||
- **Backup Scheme:**
|
||||

|
||||
More details can be found in [this blog post](https://www.veen.world/2020/12/26/how-i-backup-dedicated-root-servers/).
|
||||

|
||||
More details can be found in [this blog post](https://blog.veen.world/2020/12/26/how-i-backup-dedicated-root-servers/).
|
||||
|
||||
## Administration & Debugging
|
||||
|
4
roles/bkp-remote-to-local/handlers/main.yml
Normal file
4
roles/bkp-remote-to-local/handlers/main.yml
Normal file
@@ -0,0 +1,4 @@
|
||||
- name: "reload bkp-remote-to-local service"
|
||||
systemd:
|
||||
name: bkp-remote-to-local.cymais.service
|
||||
daemon_reload: yes
|
@@ -25,9 +25,9 @@ galaxy_info:
|
||||
issue_tracker_url: "https://s.veen.world/cymaisissues"
|
||||
documentation: "https://s.veen.world/cymais"
|
||||
dependencies:
|
||||
- generic-git
|
||||
- gen-git
|
||||
- alert-compose
|
||||
- cleanup-backups-timer
|
||||
- cleanup-failed-docker-backups
|
||||
- maintenance-lock
|
||||
- cln-backups-timer
|
||||
- cln-failed-docker-backups
|
||||
- maint-lock
|
||||
- user-root
|
@@ -4,17 +4,17 @@
|
||||
state: directory
|
||||
mode: 0755
|
||||
|
||||
- name: create backup-remote-to-local.sh
|
||||
- name: create bkp-remote-to-local.sh
|
||||
copy:
|
||||
src: backup-remote-to-local.sh
|
||||
dest: "{{docker_backup_remote_to_local_folder}}backup-remote-to-local.sh"
|
||||
src: bkp-remote-to-local.sh
|
||||
dest: "{{docker_backup_remote_to_local_folder}}bkp-remote-to-local.sh"
|
||||
mode: 0755
|
||||
|
||||
- name: create backup-remote-to-local.cymais.service
|
||||
- name: create bkp-remote-to-local.cymais.service
|
||||
template:
|
||||
src: backup-remote-to-local.service.j2
|
||||
dest: /etc/systemd/system/backup-remote-to-local.cymais.service
|
||||
notify: reload backup-remote-to-local service
|
||||
src: bkp-remote-to-local.service.j2
|
||||
dest: /etc/systemd/system/bkp-remote-to-local.cymais.service
|
||||
notify: reload bkp-remote-to-local service
|
||||
|
||||
- name: create backups-remote-to-local.sh
|
||||
template:
|
||||
@@ -26,9 +26,9 @@
|
||||
set_fact:
|
||||
service_name: "{{ role_name }}"
|
||||
|
||||
- name: "include role for generic-timer for {{service_name}}"
|
||||
- name: "include role for gen-timer for {{service_name}}"
|
||||
include_role:
|
||||
name: generic-timer
|
||||
name: gen-timer
|
||||
vars:
|
||||
on_calendar: "{{on_calendar_backup_remote_to_local}}"
|
||||
|
@@ -3,6 +3,6 @@
|
||||
hosts="{{ pull_remote_backups | join(' ') }}";
|
||||
errors=0
|
||||
for host in $hosts; do
|
||||
bash {{ docker_backup_remote_to_local_folder }}backup-remote-to-local.sh $host || ((errors+=1));
|
||||
bash {{ docker_backup_remote_to_local_folder }}bkp-remote-to-local.sh $host || ((errors+=1));
|
||||
done;
|
||||
exit $errors;
|
@@ -1,6 +1,6 @@
|
||||
[Unit]
|
||||
Description=pull remote backups
|
||||
OnFailure=alert-compose.cymais@%n.service cleanup-failed-docker-backups.cymais.service
|
||||
OnFailure=alert-compose.cymais@%n.service cln-failed-docker-backups.cymais.service
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
@@ -1 +1 @@
|
||||
docker_backup_remote_to_local_folder: "{{path_administrator_scripts}}backup-remote-to-local/"
|
||||
docker_backup_remote_to_local_folder: "{{path_administrator_scripts}}bkp-remote-to-local/"
|
@@ -1,5 +0,0 @@
|
||||
- name: "reload cleanup-backups.cymais.service"
|
||||
systemd:
|
||||
name: cleanup-backups.cymais.service
|
||||
enabled: yes
|
||||
daemon_reload: yes
|
@@ -1 +0,0 @@
|
||||
cleanup_backups_directory: "{{path_administrator_scripts}}cleanup-backups/"
|
@@ -1,6 +0,0 @@
|
||||
- name: "Reload and restart cleanup-certs.cymais.service"
|
||||
systemd:
|
||||
name: cleanup-certs.cymais.service
|
||||
enabled: yes
|
||||
daemon_reload: yes
|
||||
state: restarted
|
@@ -1,5 +0,0 @@
|
||||
- name: "reload cleanup-disc-space.cymais.service"
|
||||
systemd:
|
||||
name: cleanup-disc-space.cymais.service
|
||||
enabled: yes
|
||||
daemon_reload: yes
|
@@ -1,26 +0,0 @@
|
||||
- name: "create {{cleanup_disc_space_folder}}"
|
||||
file:
|
||||
path: "{{cleanup_disc_space_folder}}"
|
||||
state: directory
|
||||
mode: 0755
|
||||
|
||||
- name: create cleanup-disc-space.sh
|
||||
template:
|
||||
src: cleanup-disc-space.sh.j2
|
||||
dest: "{{cleanup_disc_space_folder}}cleanup-disc-space.sh"
|
||||
|
||||
- name: create cleanup-disc-space.cymais.service
|
||||
template:
|
||||
src: cleanup-disc-space.service.j2
|
||||
dest: /etc/systemd/system/cleanup-disc-space.cymais.service
|
||||
notify: reload cleanup-disc-space.cymais.service
|
||||
|
||||
- name: set service_name to the name of the current role
|
||||
set_fact:
|
||||
service_name: "{{ role_name }}"
|
||||
|
||||
- name: "include role for generic-timer for {{service_name}}"
|
||||
include_role:
|
||||
name: generic-timer
|
||||
vars:
|
||||
on_calendar: "{{on_calendar_cleanup_disc_space}}"
|
@@ -1 +0,0 @@
|
||||
cleanup_disc_space_folder: "{{path_administrator_scripts}}cleanup-disc-space/"
|
@@ -1,5 +0,0 @@
|
||||
- name: "Reload cleanup-failed-docker-backups.cymais.service"
|
||||
systemd:
|
||||
name: cleanup-failed-docker-backups.cymais.service
|
||||
enabled: yes
|
||||
daemon_reload: yes
|
5
roles/cln-backups-service/handlers/main.yml
Normal file
5
roles/cln-backups-service/handlers/main.yml
Normal file
@@ -0,0 +1,5 @@
|
||||
- name: "reload cln-backups.cymais.service"
|
||||
systemd:
|
||||
name: cln-backups.cymais.service
|
||||
enabled: yes
|
||||
daemon_reload: yes
|
@@ -22,6 +22,6 @@ galaxy_info:
|
||||
issue_tracker_url: "https://s.veen.world/cymaisissues"
|
||||
documentation: "https://s.veen.world/cymais"
|
||||
dependencies:
|
||||
- generic-python-pip
|
||||
- gen-python-pip
|
||||
- alert-compose
|
||||
- maintenance-lock
|
||||
- maint-lock
|
@@ -13,17 +13,17 @@
|
||||
mode: 0755
|
||||
when: run_once_cleanup_backups_service is not defined
|
||||
|
||||
- name: create cleanup-backups.py
|
||||
- name: create cln-backups.py
|
||||
copy:
|
||||
src: "cleanup-backups.py"
|
||||
dest: "{{cleanup_backups_directory}}cleanup-backups.py"
|
||||
src: "cln-backups.py"
|
||||
dest: "{{cleanup_backups_directory}}cln-backups.py"
|
||||
when: run_once_cleanup_backups_service is not defined
|
||||
|
||||
- name: create cleanup-backups.cymais.service
|
||||
- name: create cln-backups.cymais.service
|
||||
template:
|
||||
src: "cleanup-backups.service.j2"
|
||||
dest: "/etc/systemd/system/cleanup-backups.cymais.service"
|
||||
notify: reload cleanup-backups.cymais.service
|
||||
src: "cln-backups.service.j2"
|
||||
dest: "/etc/systemd/system/cln-backups.cymais.service"
|
||||
notify: reload cln-backups.cymais.service
|
||||
when: run_once_cleanup_backups_service is not defined
|
||||
|
||||
- name: run the cleanup_backups_service tasks once
|
@@ -5,4 +5,4 @@ OnFailure=alert-compose.cymais@%n.service
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStartPre=/bin/sh -c '/usr/bin/python {{ path_system_lock_script }} {{ system_maintenance_services | join(' ') }} --ignore {{system_maintenance_cleanup_services| join(' ') }} --timeout "{{system_maintenance_lock_timeout_backup_services}}"'
|
||||
ExecStart=/bin/sh -c '/usr/bin/python {{cleanup_backups_directory}}cleanup-backups.py --backups-folder-path {{backups_folder_path}} --maximum-backup-size-percent {{size_percent_maximum_backup}}'
|
||||
ExecStart=/bin/sh -c '/usr/bin/python {{cleanup_backups_directory}}cln-backups.py --backups-folder-path {{backups_folder_path}} --maximum-backup-size-percent {{size_percent_maximum_backup}}'
|
1
roles/cln-backups-service/vars/main.yml
Normal file
1
roles/cln-backups-service/vars/main.yml
Normal file
@@ -0,0 +1 @@
|
||||
cleanup_backups_directory: "{{path_administrator_scripts}}cln-backups/"
|
@@ -2,13 +2,13 @@
|
||||
|
||||
## Description
|
||||
|
||||
This role sets up a systemd timer to schedule the periodic cleanup of old backups. It leverages the cleanup-backups-service role to perform the actual cleanup operation.
|
||||
This role sets up a systemd timer to schedule the periodic cleanup of old backups. It leverages the cln-backups-service role to perform the actual cleanup operation.
|
||||
|
||||
## Overview
|
||||
|
||||
Optimized for automated maintenance, this role:
|
||||
- Sets a fact for the service name.
|
||||
- Integrates with the [generic-timer](../generic-timer/README.md) role to schedule cleanup-backups tasks at defined intervals.
|
||||
- Integrates with the [gen-timer](../gen-timer/README.md) role to schedule cln-backups tasks at defined intervals.
|
||||
|
||||
## Purpose
|
||||
|
||||
@@ -17,5 +17,5 @@ The primary purpose of this role is to automate the scheduling of backup cleanup
|
||||
## Features
|
||||
|
||||
- **Timer Scheduling:** Configures a systemd timer to trigger the backup cleanup service.
|
||||
- **Role Integration:** Works in conjunction with the cleanup-backups-service role.
|
||||
- **Role Integration:** Works in conjunction with the cln-backups-service role.
|
||||
- **Idempotency:** Ensures the timer tasks execute only once per playbook run.
|
@@ -1,7 +1,7 @@
|
||||
---
|
||||
galaxy_info:
|
||||
author: "Kevin Veen-Birkenbach"
|
||||
description: "Schedules periodic cleanup of old backups by configuring a systemd timer to trigger the cleanup-backups-service role."
|
||||
description: "Schedules periodic cleanup of old backups by configuring a systemd timer to trigger the cln-backups-service role."
|
||||
license: "CyMaIS NonCommercial License (CNCL)"
|
||||
license_url: "https://s.veen.world/cncl"
|
||||
company: |
|
||||
@@ -22,4 +22,4 @@ galaxy_info:
|
||||
issue_tracker_url: "https://s.veen.world/cymaisissues"
|
||||
documentation: "https://s.veen.world/cymais"
|
||||
dependencies:
|
||||
- cleanup-backups-service
|
||||
- cln-backups-service
|
@@ -1,11 +1,11 @@
|
||||
- name: set service_name to cleanup-backups
|
||||
- name: set service_name to cln-backups
|
||||
set_fact:
|
||||
service_name: "cleanup-backups"
|
||||
service_name: "cln-backups"
|
||||
when: run_once_cleanup_backups_timer is not defined
|
||||
|
||||
- name: "include role for generic-timer for {{service_name}}"
|
||||
- name: "include role for gen-timer for {{service_name}}"
|
||||
include_role:
|
||||
name: generic-timer
|
||||
name: gen-timer
|
||||
vars:
|
||||
on_calendar: "{{on_calendar_cleanup_backups}}"
|
||||
when: run_once_cleanup_backups_timer is not defined
|
@@ -7,8 +7,8 @@ This Ansible role automates the detection, revocation and deletion of unused Let
|
||||
## Overview
|
||||
|
||||
- Installs the `certreap` cleanup tool using the `pkgmgr-install` role
|
||||
- Deploys and configures a `cleanup-certs.cymais.service` systemd unit
|
||||
- (Optionally) Sets up a recurring cleanup via a systemd timer using the `generic-timer` role
|
||||
- Deploys and configures a `cln-certs.cymais.service` systemd unit
|
||||
- (Optionally) Sets up a recurring cleanup via a systemd timer using the `gen-timer` role
|
||||
- Integrates with `alert-compose` to send failure notifications
|
||||
- Ensures idempotent execution with a `run_once_cleanup_certs` flag
|
||||
|
||||
@@ -18,16 +18,16 @@ This Ansible role automates the detection, revocation and deletion of unused Let
|
||||
Uses `pkgmgr-install` to install the `certreap` binary.
|
||||
|
||||
- **Systemd Service Configuration**
|
||||
Deploys `cleanup-certs.cymais.service` and reloads/restarts it on changes.
|
||||
Deploys `cln-certs.cymais.service` and reloads/restarts it on changes.
|
||||
|
||||
- **Systemd Timer Scheduling**
|
||||
Optionally wires in a timer via the `generic-timer` role, controlled by the `on_calendar_cleanup_certs` variable.
|
||||
Optionally wires in a timer via the `gen-timer` role, controlled by the `on_calendar_cleanup_certs` variable.
|
||||
|
||||
- **Smart Execution Logic**
|
||||
Prevents multiple runs in one play by setting a `run_once_cleanup_certs` fact.
|
||||
|
||||
- **Failure Notification**
|
||||
Triggers `alert-compose.cymais@cleanup-certs.cymais.service` on failure.
|
||||
Triggers `alert-compose.cymais@cln-certs.cymais.service` on failure.
|
||||
|
||||
## Further Resources
|
||||
|
6
roles/cln-certs/handlers/main.yml
Normal file
6
roles/cln-certs/handlers/main.yml
Normal file
@@ -0,0 +1,6 @@
|
||||
- name: "Reload and restart cln-certs.cymais.service"
|
||||
systemd:
|
||||
name: cln-certs.cymais.service
|
||||
enabled: yes
|
||||
daemon_reload: yes
|
||||
state: restarted
|
@@ -5,11 +5,11 @@
|
||||
package_name: certreap
|
||||
when: run_once_cleanup_certs is not defined
|
||||
|
||||
- name: configure cleanup-certs.cymais.service
|
||||
- name: configure cln-certs.cymais.service
|
||||
template:
|
||||
src: cleanup-certs.service.j2
|
||||
dest: /etc/systemd/system/cleanup-certs.cymais.service
|
||||
notify: Reload and restart cleanup-certs.cymais.service
|
||||
src: cln-certs.service.j2
|
||||
dest: /etc/systemd/system/cln-certs.cymais.service
|
||||
notify: Reload and restart cln-certs.cymais.service
|
||||
when: run_once_cleanup_certs is not defined
|
||||
|
||||
- name: set service_name to the name of the current role
|
||||
@@ -17,9 +17,9 @@
|
||||
service_name: "{{ role_name }}"
|
||||
when: run_once_cleanup_certs is not defined
|
||||
|
||||
- name: "include role for generic-timer for {{service_name}}"
|
||||
- name: "include role for gen-timer for {{service_name}}"
|
||||
include_role:
|
||||
name: generic-timer
|
||||
name: gen-timer
|
||||
vars:
|
||||
on_calendar: "{{ on_calendar_cleanup_certs }}"
|
||||
when: run_once_cleanup_certs is not defined
|
5
roles/cln-disc-space/handlers/main.yml
Normal file
5
roles/cln-disc-space/handlers/main.yml
Normal file
@@ -0,0 +1,5 @@
|
||||
- name: "reload cln-disc-space.cymais.service"
|
||||
systemd:
|
||||
name: cln-disc-space.cymais.service
|
||||
enabled: yes
|
||||
daemon_reload: yes
|
@@ -23,4 +23,4 @@ galaxy_info:
|
||||
documentation: "https://s.veen.world/cymais"
|
||||
dependencies:
|
||||
- alert-compose
|
||||
- maintenance-lock
|
||||
- maint-lock
|
26
roles/cln-disc-space/tasks/main.yml
Normal file
26
roles/cln-disc-space/tasks/main.yml
Normal file
@@ -0,0 +1,26 @@
|
||||
- name: "create {{cleanup_disc_space_folder}}"
|
||||
file:
|
||||
path: "{{cleanup_disc_space_folder}}"
|
||||
state: directory
|
||||
mode: 0755
|
||||
|
||||
- name: create cln-disc-space.sh
|
||||
template:
|
||||
src: cln-disc-space.sh.j2
|
||||
dest: "{{cleanup_disc_space_folder}}cln-disc-space.sh"
|
||||
|
||||
- name: create cln-disc-space.cymais.service
|
||||
template:
|
||||
src: cln-disc-space.service.j2
|
||||
dest: /etc/systemd/system/cln-disc-space.cymais.service
|
||||
notify: reload cln-disc-space.cymais.service
|
||||
|
||||
- name: set service_name to the name of the current role
|
||||
set_fact:
|
||||
service_name: "{{ role_name }}"
|
||||
|
||||
- name: "include role for gen-timer for {{service_name}}"
|
||||
include_role:
|
||||
name: gen-timer
|
||||
vars:
|
||||
on_calendar: "{{on_calendar_cleanup_disc_space}}"
|
@@ -5,4 +5,4 @@ OnFailure=alert-compose.cymais@%n.service
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStartPre=/bin/sh -c '/usr/bin/python {{ path_system_lock_script }} {{ system_maintenance_services | join(' ') }} --ignore {{system_maintenance_cleanup_services| join(' ') }} --timeout "{{system_maintenance_lock_timeout_backup_services}}"'
|
||||
ExecStart=/bin/sh -c '/bin/bash {{cleanup_disc_space_folder}}cleanup-disc-space.sh {{size_percent_cleanup_disc_space}}'
|
||||
ExecStart=/bin/sh -c '/bin/bash {{cleanup_disc_space_folder}}cln-disc-space.sh {{size_percent_cleanup_disc_space}}'
|
@@ -24,7 +24,7 @@ if [ "$force_freeing" = true ]; then
|
||||
|
||||
{% if backups_folder_path is defined and size_percent_maximum_backup is defined %}
|
||||
echo "cleaning up backups" &&
|
||||
python {{path_administrator_scripts}}cleanup-backups/cleanup-backups.py --backups-folder-path {{backups_folder_path}} --maximum-backup-size-percent {{size_percent_maximum_backup}} || exit 2
|
||||
python {{path_administrator_scripts}}cln-backups/cln-backups.py --backups-folder-path {{backups_folder_path}} --maximum-backup-size-percent {{size_percent_maximum_backup}} || exit 2
|
||||
{% endif %}
|
||||
|
||||
if pacman -Qs $package > /dev/null ; then
|
1
roles/cln-disc-space/vars/main.yml
Normal file
1
roles/cln-disc-space/vars/main.yml
Normal file
@@ -0,0 +1 @@
|
||||
cleanup_disc_space_folder: "{{path_administrator_scripts}}cln-disc-space/"
|
@@ -1,4 +1,4 @@
|
||||
# cleanup-domains
|
||||
# cln-domains
|
||||
|
||||
## Description
|
||||
|
@@ -21,4 +21,4 @@ galaxy_info:
|
||||
issue_tracker_url: "https://s.veen.world/cymaisissues"
|
||||
documentation: "https://s.veen.world/cymais"
|
||||
dependencies:
|
||||
- webserver-core
|
||||
- srv-web-core
|
@@ -2,15 +2,15 @@
|
||||
|
||||
## Description
|
||||
|
||||
This role cleans up failed Docker backups by pulling a [Git repository](https://github.com/kevinveenbirkenbach/cleanup-failed-docker-backups) that contains cleanup scripts and configuring a systemd service to execute them. It ensures that failed or incomplete backups are removed to free up disk space and maintain a healthy backup environment.
|
||||
This role cleans up failed Docker backups by pulling a [Git repository](https://github.com/kevinveenbirkenbach/cln-failed-docker-backups) that contains cleanup scripts and configuring a systemd service to execute them. It ensures that failed or incomplete backups are removed to free up disk space and maintain a healthy backup environment.
|
||||
|
||||
## Overview
|
||||
|
||||
Optimized for backup maintenance, this role:
|
||||
- Clones the cleanup-failed-docker-backups repository.
|
||||
- Clones the cln-failed-docker-backups repository.
|
||||
- Configures a systemd service to run the cleanup script.
|
||||
- Integrates with the [generic-timer](../generic-timer/README.md) role to schedule periodic cleanup.
|
||||
- Works in conjunction with the backup-directory-validator role for additional verification.
|
||||
- Integrates with the [gen-timer](../gen-timer/README.md) role to schedule periodic cleanup.
|
||||
- Works in conjunction with the bkp-directory-validator role for additional verification.
|
||||
|
||||
## Purpose
|
||||
|
||||
@@ -21,4 +21,4 @@ The primary purpose of this role is to remove failed Docker backups automaticall
|
||||
- **Repository Cloning:** Retrieves the latest cleanup scripts from a Git repository.
|
||||
- **Service Configuration:** Sets up a systemd service to run the cleanup tasks.
|
||||
- **Timer Integration:** Schedules periodic cleanup through a systemd timer.
|
||||
- **Dependency Integration:** Works with backup-directory-validator to enhance backup integrity.
|
||||
- **Dependency Integration:** Works with bkp-directory-validator to enhance backup integrity.
|
5
roles/cln-failed-docker-backups/handlers/main.yml
Normal file
5
roles/cln-failed-docker-backups/handlers/main.yml
Normal file
@@ -0,0 +1,5 @@
|
||||
- name: "Reload cln-failed-docker-backups.cymais.service"
|
||||
systemd:
|
||||
name: cln-failed-docker-backups.cymais.service
|
||||
enabled: yes
|
||||
daemon_reload: yes
|
@@ -23,5 +23,5 @@ galaxy_info:
|
||||
documentation: "https://s.veen.world/cymais"
|
||||
dependencies:
|
||||
- alert-compose
|
||||
- maintenance-lock
|
||||
- backup-directory-validator
|
||||
- maint-lock
|
||||
- bkp-directory-validator
|
@@ -2,26 +2,26 @@
|
||||
include_role:
|
||||
name: pkgmgr-install
|
||||
vars:
|
||||
package_name: cleanup-failed-docker-backups
|
||||
package_name: cln-failed-docker-backups
|
||||
when: run_once_cleanup_failed_docker_backups is not defined
|
||||
|
||||
- name: Retrieve backup-docker-to-local path from pkgmgr
|
||||
command: pkgmgr path cleanup-failed-docker-backups
|
||||
- name: Retrieve bkp-docker-to-local path from pkgmgr
|
||||
command: pkgmgr path cln-failed-docker-backups
|
||||
register: pkgmgr_output
|
||||
changed_when: false
|
||||
when: run_once_cleanup_failed_docker_backups is not defined
|
||||
|
||||
- name: Set fact for backup_docker_to_local_cleanup_script
|
||||
set_fact:
|
||||
backup_docker_to_local_cleanup_script: "{{ pkgmgr_output.stdout.rstrip('/') ~ '/cleanup-all.sh' }}"
|
||||
backup_docker_to_local_cleanup_script: "{{ pkgmgr_output.stdout.rstrip('/') ~ '/cln-all.sh' }}"
|
||||
changed_when: false
|
||||
when: run_once_cleanup_failed_docker_backups is not defined
|
||||
|
||||
- name: configure cleanup-failed-docker-backups.cymais.service
|
||||
- name: configure cln-failed-docker-backups.cymais.service
|
||||
template:
|
||||
src: cleanup-failed-docker-backups.service.j2
|
||||
dest: /etc/systemd/system/cleanup-failed-docker-backups.cymais.service
|
||||
notify: Reload cleanup-failed-docker-backups.cymais.service
|
||||
src: cln-failed-docker-backups.service.j2
|
||||
dest: /etc/systemd/system/cln-failed-docker-backups.cymais.service
|
||||
notify: Reload cln-failed-docker-backups.cymais.service
|
||||
when: run_once_cleanup_failed_docker_backups is not defined
|
||||
|
||||
- name: set service_name to the name of the current role
|
||||
@@ -29,9 +29,9 @@
|
||||
service_name: "{{ role_name }}"
|
||||
when: run_once_cleanup_failed_docker_backups is not defined
|
||||
|
||||
- name: "include role for generic-timer for {{service_name}}"
|
||||
- name: "include role for gen-timer for {{service_name}}"
|
||||
include_role:
|
||||
name: generic-timer
|
||||
name: gen-timer
|
||||
vars:
|
||||
on_calendar: "{{on_calendar_cleanup_failed_docker}}"
|
||||
when: run_once_cleanup_failed_docker_backups is not defined
|
@@ -22,7 +22,7 @@ This Ansible role manages the configuration of `systemd-journald` on target host
|
||||
|
||||
- Customizable retention and runtime limits
|
||||
- Seamless restarts on config update
|
||||
- Integration with `monitor-bot-journalctl` for downstream monitoring
|
||||
- Integration with `mon-bot-journalctl` for downstream monitoring
|
||||
|
||||
## Usage
|
||||
|
||||
|
@@ -17,4 +17,4 @@ galaxy_info:
|
||||
documentation: "https://github.com/kevinveenbirkenbach/cymais/roles/core-journalctl"
|
||||
run_after: []
|
||||
dependencies:
|
||||
- monitor-bot-journalctl
|
||||
- mon-bot-journalctl
|
||||
|
@@ -1,4 +1,4 @@
|
||||
# desktop-bluray-player
|
||||
# desk-bluray-player
|
||||
|
||||
## Description
|
||||
|
@@ -1,2 +1,2 @@
|
||||
dependencies:
|
||||
- generic-java
|
||||
- gen-java
|
@@ -1,10 +1,10 @@
|
||||
# README for PC-Docker Playbook
|
||||
|
||||
## Overview
|
||||
This playbook, `desktop-docker`, is part of a larger collection housed within the `cymais` repository. It is specifically tailored for setting up Docker and Docker Compose on personal computers (PCs) used for development purposes. The primary goal is to facilitate a development environment on individual workstations rather than configuring servers for hosting or distributing Docker images.
|
||||
This playbook, `desk-docker`, is part of a larger collection housed within the `cymais` repository. It is specifically tailored for setting up Docker and Docker Compose on personal computers (PCs) used for development purposes. The primary goal is to facilitate a development environment on individual workstations rather than configuring servers for hosting or distributing Docker images.
|
||||
|
||||
## Contents
|
||||
The `main.yml` file in the `desktop-docker` role consists of two primary tasks:
|
||||
The `main.yml` file in the `desk-docker` role consists of two primary tasks:
|
||||
|
||||
1. **Install Docker**: This task uses the `community.general.pacman` module to install `docker` and `docker-compose` on the system. It ensures that these packages are present on the PC.
|
||||
|
||||
@@ -20,7 +20,7 @@ The playbook is designed for developers who require Docker in their local develo
|
||||
## Running the Playbook
|
||||
To run this playbook:
|
||||
1. Clone the `cymais` repository.
|
||||
2. Navigate to the `roles/desktop-docker` directory.
|
||||
2. Navigate to the `roles/desk-docker` directory.
|
||||
3. Run the playbook using the appropriate Ansible commands, ensuring that you have the necessary privileges.
|
||||
|
||||
## Important Notes
|
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user