Huge role refactoring/cleanup. Other commits will propably follow. Because some bugs will exist. Still important for longrun and also for auto docs/help/slideshow generation

2025-08-29 15:06:26 +02:00 · 2025-07-08 23:43:13 +02:00
parent 6b87a049d4
commit 563d5fd528
1242 changed files with 2301 additions and 1355 deletions
--- a/roles/web-app-mastodon/Administration.md
+++ b/roles/web-app-mastodon/Administration.md
@@ -0,0 +1,39 @@
+# Administration
+
+## 🗑️ Cleanup (Remove Instance & Volumes)
+```bash
+cd {{path_docker_compose_instances}}mastodon/
+docker-compose down
+docker volume rm mastodon_data mastodon_database mastodon_redis
+cd {{path_docker_compose_instances}} &&
+rm -vR {{path_docker_compose_instances}}mastodon
+```
+
+## 🔍 Access Mastodon Terminal
+```bash
+docker-compose exec -it web /bin/bash
+```
+
+## 🛠️ Set File Permissions
+After setting up Mastodon, apply the correct file permissions:
+```bash
+docker-compose exec -it -u root web chown -R 991:991 public
+```
+
+# 📦 Database Management
+
+## 🏗️ Running Database Migrations
+Ensure all required database structures are up to date:
+```bash
+docker compose exec -it web bash -c "RAILS_ENV=production bin/rails db:migrate"
+```
+
+# 🚀 Performance Optimization
+
+## 🗑️ Delete Cache & Recompile Assets
+```bash
+docker-compose exec web bundle exec rails assets:precompile
+docker-compose restart
+```
+
+This ensures your Mastodon instance is loading the latest assets after updates.
--- a/roles/web-app-mastodon/Installation.md
+++ b/roles/web-app-mastodon/Installation.md
@@ -0,0 +1,22 @@
+# ⚙️ Configuration & Setup
+
+## 🔧 Create Credentials
+Run the following command to generate a new configuration setup:
+```bash
+    docker pull ghcr.io/mastodon/mastodon:latest
+    # Secret Generation
+    docker run --rm ghcr.io/mastodon/mastodon:latest bundle exec rails secret
+    docker run --rm ghcr.io/mastodon/mastodon:latest bundle exec rails secret
+    # Vapid Key Generation
+    docker run --rm ghcr.io/mastodon/mastodon:latest bundle exec rails mastodon:webpush:generate_vapid_key
+    # ACTIVE_RECORD_ENCRYPTION Generation
+    docker run --rm ghcr.io/mastodon/mastodon:latest bin/rails db:encryption:init
+```
+
+## 🔄 Setup with an Existing Configuration
+```bash
+docker-compose run --rm web bundle exec rails db:migrate
+```
+
+## 🔐 OIDC (OpenID Connect) Authentication Support
+This Mastodon role now **fully supports OpenID Connect (OIDC)**, allowing seamless authentication via identity providers like **Keycloak, Auth0, Google, or other OIDC-compliant services**.
--- a/roles/web-app-mastodon/README.md
+++ b/roles/web-app-mastodon/README.md
@@ -0,0 +1,45 @@
+# Mastodon
+
+## Description
+
+Dive into a decentralized social experience with Mastodon, a vibrant platform that redefines online communication with its federated, community-driven approach. With a rich set of features focused on privacy, scalability, and customization, Mastodon empowers users to create, share, and interact in an open social network.
+
+## Overview
+
+This role deploys Mastodon using Docker, streamlining the installation and configuration of a full-featured social networking platform. Mastodon is built to support federation across multiple instances, offering robust content moderation, real-time updates, and flexible API integrations. Its advanced architecture—including separate services for the web frontend, streaming API, and background job processing—ensures high performance and scalability for large communities.
+
+For detailed configuration and operational instructions, please refer to the following files in this directory:
+- [Installation.md](./Installation.md)
+- [Administration.md](./Administration.md)
+- [Upgrade.md](./Upgrade.md)
+- [User_Administration.md](./User_Administration.md)
+
+## Features
+
+- **Decentralized Network:** Connect with users across multiple instances in a federated social media ecosystem.
+- **Real-Time Streaming:** Enjoy dynamic updates and real-time content delivery through dedicated streaming services.
+- **Robust Content Moderation:** Utilize powerful moderation tools to manage community interactions and maintain safe spaces.
+- **Scalable Architecture:** Benefit from a multi-service, Docker-based setup that supports high user loads and seamless background processing.
+- **Flexible Authentication:** Integrated support for OpenID Connect (OIDC) simplifies user login and enhances security.
+- **Customizable User Experience:** Configure themes, timeline settings, and notification options to tailor the social experience to your community.
+
+## Further Resources
+
+- [Mastodon Official Website](https://joinmastodon.org/)
+- [Mastodon Documentation](https://docs.joinmastodon.org/)
+- [Mastodon with Docker & Traefik](https://goneuland.de/mastodon-mit-web-app-und-traefik-installieren/)
+- [Mastodon Configuration Guide](https://gist.github.com/TrillCyborg/84939cd4013ace9960031b803a0590c4)
+- [Check Website Availability](https://www.2daygeek.com/linux-command-check-website-is-up-down-alive/)
+- [Personal Mastodon Setup](https://vitobotta.com/2022/11/07/setting-up-a-personal-mastodon-instance/)
+- [Scaling a Mastodon Server](https://www.digitalocean.com/community/tutorials/how-to-scale-your-mastodon-server)
+- [Mastodon GitHub Issues](https://github.com/mastodon/mastodon/issues/7958)
+
+## Credits
+
+Developed and maintained by **Kevin Veen-Birkenbach**.  
+Learn more at [veen.world](https://www.veen.world).
+
+Part of the [CyMaIS Project](https://github.com/kevinveenbirkenbach/cymais)  
+Licensed under [CyMaIS NonCommercial License (CNCL)](https://s.veen.world/cncl).
+
+
--- a/roles/web-app-mastodon/meta/main.yml
+++ b/roles/web-app-mastodon/meta/main.yml
@@ -0,0 +1,25 @@
+---
+galaxy_info:
+  author: "Kevin Veen-Birkenbach"
+  description: "Dive into a decentralized social experience with Mastodon, a federated platform designed for dynamic, community-driven networking. Enjoy real-time updates, robust moderation, and customizable user interfaces that empower communities to thrive."
+  license: "CyMaIS NonCommercial License (CNCL)"
+  license_url: "https://s.veen.world/cncl"
+  company: |
+    Kevin Veen-Birkenbach
+    Consulting & Coaching Solutions
+    https://www.veen.world
+  galaxy_tags:
+    - mastodon
+    - docker
+    - federated
+    - decentralized
+    - microblog 
+    - social network
+
+  repository: "https://s.veen.world/cymais"
+  issue_tracker_url: "https://s.veen.world/cymaisissues"
+  documentation: "https://s.veen.world/cymais"
+  logo:
+    class: "fa-solid fa-bullhorn"
+  run_after:
+    - web-app-keycloak
--- a/roles/web-app-mastodon/meta/schema.yml
+++ b/roles/web-app-mastodon/meta/schema.yml
@@ -0,0 +1,35 @@
+credentials:
+  secret_key_base:
+    description: "Main secret key used to verify the integrity of signed cookies and tokens"
+    algorithm: "sha256"
+    validation: "^[a-f0-9]{64}$"
+
+  otp_secret:
+    description: "OTP secret used for two-factor authentication"
+    algorithm: "sha256"
+    validation: "^[a-f0-9]{64}$"
+
+  vapid_private_key:
+    description: "Private VAPID key used for web push notifications"
+    algorithm: "plain"
+    validation: "^[-_a-zA-Z0-9]{30,}$"
+
+  vapid_public_key:
+    description: "Public VAPID key used for web push notifications"
+    algorithm: "plain"
+    validation: "^[-_a-zA-Z0-9]{30,}$"
+
+  active_record_encryption_deterministic_key:
+    description: "Deterministic encryption key for Active Record encryption"
+    algorithm: "sha256"
+    validation: "^[a-f0-9]{64}$"
+
+  active_record_encryption_key_derivation_salt:
+    description: "Key derivation salt for Active Record encryption"
+    algorithm: "sha256"
+    validation: "^[a-f0-9]{64}$"
+
+  active_record_encryption_primary_key:
+    description: "Primary encryption key for Active Record encrypted columns"
+    algorithm: "sha256"
+    validation: "^[a-f0-9]{64}$"
--- a/roles/web-app-mastodon/tasks/create-administrator.yml
+++ b/roles/web-app-mastodon/tasks/create-administrator.yml
@@ -0,0 +1,43 @@
+# Routines to create the administrator account
+# @see https://chatgpt.com/share/67b9b12c-064c-800f-9354-8e42e6459764
+
+- name: Check health status of {{ item }} container
+  shell: |
+    cid=$(docker compose ps -q {{ item }})
+    docker inspect \
+      --format '{{ "{{.State.Health.Status}}" }}' \
+      $cid
+  args:
+    chdir: "{{ docker_compose.directories.instance }}"
+  register: healthcheck
+  retries: 60
+  delay: 5
+  until: healthcheck.stdout == "healthy"
+  loop:
+    - web
+    - streaming
+    - sidekiq
+  loop_control:
+    label: "{{ item }}"
+
+- name: Remove line containing "- administrator" from config/settings.yml to allow creating administrator account
+  command: 
+    cmd:  "docker compose exec -u root web sed -i '/- administrator/d' config/settings.yml"
+    chdir: "{{docker_compose.directories.instance}}"
+  when: users.administrator.username == "administrator"
+
+- name: Create admin account via tootctl
+  command: 
+    cmd: 'docker compose exec -u root web bash -c "RAILS_ENV=production bin/tootctl accounts create {{users.administrator.username}} --email {{ users.administrator.email }} --confirmed --role Owner"'
+    chdir: "{{docker_compose.directories.instance}}"
+  register: tootctl_create
+  changed_when: tootctl_create.rc == 0
+  failed_when: >
+    tootctl_create.rc != 0
+    and
+    ("taken" not in tootctl_create.stderr | lower)
+    
+- name: Approve the administrator account in Mastodon
+  command: 
+    cmd: docker compose exec -u root web bash -c "RAILS_ENV=production bin/tootctl accounts modify {{users.administrator.username}} --approve"
+    chdir: "{{docker_compose.directories.instance}}"
--- a/roles/web-app-mastodon/tasks/main.yml
+++ b/roles/web-app-mastodon/tasks/main.yml
@@ -0,0 +1,30 @@
+---
+- name: "include service-rdbms-central"
+  include_role: 
+    name: service-rdbms-central
+
+- name: "Include setup for domain '{{ domain }}'"
+  include_role: 
+    name: webserver-proxy-domain
+  loop: "{{ domains.mastodon }}"
+  loop_control:
+    loop_var: domain
+  vars:
+    http_port:            "{{ ports.localhost.http[application_id] }}"
+    ws_path:              "/api/v1/streaming"
+    ws_port:              "{{ ports.localhost.websocket[application_id] }}"
+    client_max_body_size: "80m"
+    vhost_flavour:        "ws_generic"
+
+- name: flush docker service
+  meta: flush_handlers
+  when: applications.mastodon.setup |bool 
+
+- name: setup routine for mastodon
+  command:
+    cmd: "docker-compose run --rm web bundle exec rails db:migrate"
+    chdir: "{{docker_compose.directories.instance}}"
+  when: applications.mastodon.setup |bool 
+
+- name: "include create-administrator.yml for mastodon"
+  include_tasks: create-administrator.yml
--- a/roles/web-app-mastodon/templates/docker-compose.yml.j2
+++ b/roles/web-app-mastodon/templates/docker-compose.yml.j2
@@ -0,0 +1,44 @@
+{% include 'roles/docker-compose/templates/base.yml.j2' %}
+
+  web:
+{% set container_port = 3000 %}
+{% set container_healthcheck = 'health' %}
+    image: "{{ applications[application_id].images[application_id] }}"
+{% include 'roles/docker-container/templates/base.yml.j2' %}
+    command: bash -c "rm -f /mastodon/tmp/pids/server.pid; bundle exec rails s -p {{ container_port }}"
+{% include 'roles/docker-container/templates/healthcheck/wget.yml.j2' %}
+    ports:
+      - "127.0.0.1:{{ports.localhost.http[application_id]}}:{{ container_port }}"
+{% include 'roles/docker-container/templates/depends_on/dmbs_excl.yml.j2' %}
+    volumes:
+      - data:/mastodon/public/system
+{% include 'roles/docker-container/templates/networks.yml.j2' %}
+
+  streaming:
+{% set container_port = 4000 %}
+{% set container_healthcheck = 'api/v1/streaming/health' %}
+    image: "{{ applications[application_id].images.streaming }}"
+{% include 'roles/docker-container/templates/base.yml.j2' %}
+    command: node ./streaming
+{% include 'roles/docker-container/templates/healthcheck/wget.yml.j2' %}
+    ports:
+      - "127.0.0.1:{{ports.localhost.websocket[application_id]}}:{{ container_port }}"
+{% include 'roles/docker-container/templates/depends_on/dmbs_excl.yml.j2' %}
+{% include 'roles/docker-container/templates/networks.yml.j2' %}
+
+  sidekiq:
+    image: "{{ applications[application_id].images.mastodon }}"
+{% include 'roles/docker-container/templates/base.yml.j2' %}
+    command: bundle exec sidekiq
+{% include 'roles/docker-container/templates/depends_on/dmbs_excl.yml.j2' %}
+    volumes:
+      - data:/mastodon/public/system
+    healthcheck:
+      test: ['CMD-SHELL', "ps aux | grep '[s]idekiq\ 6' || false"]
+{% include 'roles/docker-container/templates/networks.yml.j2' %}
+
+{% include 'roles/docker-compose/templates/volumes.yml.j2' %}
+  redis:
+  data:
+
+{% include 'roles/docker-compose/templates/networks.yml.j2' %}
--- a/roles/web-app-mastodon/templates/env.j2
+++ b/roles/web-app-mastodon/templates/env.j2
@@ -0,0 +1,77 @@
+# Configuration file for mastodon
+# @see https://docs.joinmastodon.org/admin/config
+# @see https://github.com/mastodon/mastodon/blob/main/.env.production.sample
+
+
+LOCAL_DOMAIN={{domains | get_domain(application_id)}}
+ALTERNATE_DOMAINS="{{ domains.mastodon[1:] | join(',') }}"
+SINGLE_USER_MODE={{applications.mastodon.single_user_mode}}
+
+# Credentials
+
+# Secrets
+# -------
+# Make sure to use `bundle exec rails secret` to generate secrets
+# -------
+SECRET_KEY_BASE=    {{applications.mastodon.credentials.secret_key_base}}
+OTP_SECRET=         {{applications.mastodon.credentials.otp_secret}}
+
+# Web Push
+# --------
+# Generate with `bundle exec rails mastodon:webpush:generate_vapid_key`
+# --------
+VAPID_PRIVATE_KEY=  {{applications.mastodon.credentials.vapid_private_key}}
+VAPID_PUBLIC_KEY=   {{applications.mastodon.credentials.vapid_public_key}}
+
+# Encryption secrets
+# ------------------
+# Must be available (and set to same values) for all server processes
+# These are private/secret values, do not share outside hosting environment
+# Use `bin/rails db:encryption:init` to generate fresh secrets
+# Do NOT change these secrets once in use, as this would cause data loss and other issues
+ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY=     {{applications.mastodon.credentials.active_record_encryption_deterministic_key}}
+ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT=   {{applications.mastodon.credentials.active_record_encryption_key_derivation_salt}}
+ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY=           {{applications.mastodon.credentials.active_record_encryption_primary_key}}
+
+DB_HOST={{ database_host }}
+DB_PORT={{ database_port }}
+DB_NAME={{ database_name }}
+DB_USER={{ database_username }}
+DB_PASS={{ database_password }}
+
+REDIS_HOST=redis
+REDIS_PORT=6379
+REDIS_PASSWORD=
+
+SMTP_SERVER={{system_email.host}}
+SMTP_PORT={{system_email.port}}
+SMTP_LOGIN={{ users['no-reply'].email }}
+SMTP_PASSWORD={{ users['no-reply'].mailu_token }}
+SMTP_AUTH_METHOD=plain
+SMTP_OPENSSL_VERIFY_MODE=none
+SMTP_ENABLE_STARTTLS=auto
+SMTP_FROM_ADDRESS=Mastodon <{{ users['no-reply'].email }}>
+
+{% if applications | is_feature_enabled('oidc',application_id) %}
+################################### 
+# OpenID Connect settings
+###################################
+# @see https://github.com/mastodon/mastodon/pull/16221
+# @see https://stackoverflow.com/questions/72081776/how-mastodon-configured-login-using-sso
+
+OIDC_ENABLED={{ applications | is_feature_enabled('oidc',application_id) | string | lower }}
+OIDC_DISPLAY_NAME="{{oidc.button_text}}"
+OIDC_ISSUER={{oidc.client.issuer_url}}
+OIDC_DISCOVERY=true
+OIDC_SCOPE="openid,profile,email"
+# @see https://stackoverflow.com/questions/72108087/how-to-set-the-username-of-mastodon-by-log-in-via-keycloak
+OIDC_UID_FIELD={{oidc.attributes.username}}
+OIDC_CLIENT_ID={{oidc.client.id}}
+OIDC_REDIRECT_URI=https://{{domains | get_domain(application_id)}}/auth/auth/openid_connect/callback
+OIDC_SECURITY_ASSUME_EMAIL_IS_VERIFIED=true
+OIDC_CLIENT_SECRET={{oidc.client.secret}}
+# uncomment to only use OIDC for login / registration buttons
+OMNIAUTH_ONLY=true
+ALLOW_UNSAFE_AUTH_PROVIDER_REATTACH=true
+ONE_CLICK_SSO_LOGIN=true
+{% endif %}
--- a/roles/web-app-mastodon/vars/configuration.yml
+++ b/roles/web-app-mastodon/vars/configuration.yml
@@ -0,0 +1,24 @@
+images:
+  mastodon:           "ghcr.io/mastodon/mastodon:latest"
+  streaming:          "ghcr.io/mastodon/mastodon-streaming:latest"
+single_user_mode:     false                                           # Set true for initial setup
+setup:                false                                           # Set true in inventory file to execute the setup and initializing procedures                         
+features:
+  matomo:             true
+  css:                true
+  portfolio_iframe:   true
+  oidc:               true
+  central_database:   true
+domains:
+  canonical:
+    - "microblog.{{ primary_domain }}"
+csp:
+  whitelist:
+    frame-src:
+      - "*"
+docker:
+  services:
+    redis:
+      enabled: true
+    database: 
+      enabled: true
--- a/roles/web-app-mastodon/vars/main.yml
+++ b/roles/web-app-mastodon/vars/main.yml
@@ -0,0 +1,2 @@
+application_id:      "mastodon"
+database_type:      "postgres"