Added migration comment to mastodon

group_vars/all

@@ -190,6 +190,7 @@ bigbluebutton_enable_greenlight:  "true"
 
 #### Listmonk
 listmonk_admin_username:          "admin"
+listmonk_public_api_activated:    False # Security hole. Can be used for spaming
 
 #### Mastodon
 mastodon_version:                 "latest"

roles/docker-listmonk/files/deactivate-public-api.conf

@@ -0,0 +1,3 @@
+location /api/public/subscription {
+    return 403;
+}

roles/docker-listmonk/tasks/main.yml

@@ -2,6 +2,15 @@
 - name: "include docker/compose/database.yml"
   include_tasks: docker/compose/database.yml
 
+- name: Set nginx_docker_reverse_proxy_extra_configuration based on listmonk_public_api_activated
+  set_fact:
+    nginx_docker_reverse_proxy_extra_configuration: >-
+      {% if not listmonk_public_api_activated %}
+      {{ lookup('file', '{{ role_path }}/files/deactivate-public-api.conf') }}
+      {% else %}
+      ""
+      {% endif %}
+
 - name: "include tasks nginx-docker-proxy-domain.yml"
   include_tasks: nginx-docker-proxy-domain.yml
 

roles/docker-listmonk/templates/config.toml.j2

@@ -9,8 +9,9 @@ address = "0.0.0.0:9000"
 # be replaced with a better multi-user, role-based authentication system.
 # IMPORTANT: Leave both values empty to disable authentication on admin
 # only where an external authentication is already setup.
-admin_username = "{{listmonk_admin_username}}"
-admin_password = "{{listmonk_admin_password}}"
+# admin_username = "{{listmonk_admin_username}}"
+# admin_password = "{{listmonk_admin_password}}"
+# Deactivated for newer versions
 
 # Database.
 [db]

roles/docker-mailu/README.md

@@ -1,6 +1,6 @@
 # Mailu Server Docker Role
 
-This guide provides instructions for setting up, operating, and maintaining the Mailu server docker role. 
+This guide provides instructions for setting up, operating, and maintaining the Mailu server docker role.
 
 ## Table of Contents
 
@@ -14,19 +14,20 @@ This guide provides instructions for setting up, operating, and maintaining the
 - [Debugging](#debugging)
 - [Testing](#testing)
 - [Updates](#updates)
+- [Queue Management](#queue-management)
+- [Spam Issues](#spam-issues)
 - [To-Do](#to-do)
-- [Spam Issues][#spam-issues]
 - [References](#references)
 
 ## Setup
 
 ### Fetchmail Issues
 
-Fetchmail might not work properly with large amounts of data. For more information, refer to this [issue](https://github.com/Mailu/Mailu/issues/1719). 
+Fetchmail might not work properly with large amounts of data. For more information, refer to this [issue](https://github.com/Mailu/Mailu/issues/1719).
 
 #### Deactivating Fetchmail
 
-Before uninstalling Fetchmail, ensure to remove all fetched accounts from the administration panel.
+Before uninstalling Fetchmail, ensure you remove all fetched accounts from the administration panel.
 
 #### Fetchmail Security Concerns
 
@@ -114,8 +115,6 @@ docker-compose exec -it smtp postqueue -f
 
 Use the following tools for testing:
 
-
-
 - SSL-Tools Mailserver Test (URL: https://de.ssl-tools.net/mailservers/)
 - TestEmail.de (URL: http://testemail.de/)
 
@@ -123,24 +122,43 @@ Use the following tools for testing:
 
 For instructions on updating your Mailu setup, follow the official [Mailu maintenance guide](https://mailu.io/master/maintain.html).
 
+## Queue Management
+
+To manage the Postfix email queue in Mailu, you can use the following commands:
+
+- **Display the email queue**: This command shows all queued emails.
+  ```bash
+  docker compose exec -it smtp postqueue -p
+  ```
+
+- **Delete all emails in the queue**: To remove all queued emails permanently, use the command:
+  ```bash
+  docker compose exec -it smtp postsuper -d ALL
+  ```
+
+These commands can help control the email queue, especially for clearing out emails with delivery issues or delays.
+
 ## Spam Issues
 
 ### Inspect
 
-To inspect use:
-- [Google Postmaster](https://postmaster.google.com/)
-- [Yahoo Postmaster](https://postmaster.yahooinc.com)
+Use the following tools to monitor your domain and email deliverability:
 
-### Blacklisted
-It may be that your domain is blacklisted. In this case check out:
+- [Google Postmaster](https://postmaster.google.com/) - Analyzes deliverability and spam issues for Gmail.
+- [Yahoo Postmaster](https://postmaster.yahooinc.com) - Provides insights and delivery reports for Yahoo.
 
-- [Spamhaus](https://check.spamhaus.org/)
-- [Barracuda](https://www.barracudacentral.org/lookups)
+### Blacklist Check
 
-### Reset
-- [Cloudmark](https://csi.cloudmark.com/en/reset/)
+If your domain is blacklisted, you can check the status with these services and take steps to remove your domain if necessary:
 
+- [Spamhaus](https://check.spamhaus.org/) - One of the most widely used blacklists for spam.
+- [Barracuda](https://www.barracudacentral.org/lookups) - Checks if your IP is on the Barracuda blacklist.
 
+### Cloudmark Reset Request
+
+If your IP or domain is flagged by Cloudmark, you can submit a **reset request** to help restore deliverability:
+
+- [Cloudmark Reset](https://csi.cloudmark.com/en/reset/)
 
 ## To-Do
 

roles/docker-mastodon/README.md

@@ -32,9 +32,20 @@ After setting up mastodon you need to give the rights
 docker-compose exec -it -u root web chown -R 991:991 public
 ```
 
+### Running Database Migrations
+
+To ensure all required database tables and structures are in place after an update, you can run database migrations directly within the Mastodon web container. Use the following command:
+
+```bash
+docker compose exec -it web bash -c "RAILS_ENV=production bin/rails db:migrate"
+```
+
+This command enters the web container and runs migrations in production mode, updating the database structure to match the current version of the Mastodon software. If your Mastodon web container is named differently in your `docker-compose.yml`, replace `web` with the appropriate container name.
+
 ## further information
 - https://goneuland.de/mastodon-mit-docker-und-traefik-installieren/
 - https://gist.github.com/TrillCyborg/84939cd4013ace9960031b803a0590c4
 - https://www.2daygeek.com/linux-command-check-website-is-up-down-alive/
 - https://vitobotta.com/2022/11/07/setting-up-a-personal-mastodon-instance/
-- https://www.digitalocean.com/community/tutorials/how-to-scale-your-mastodon-server
+- https://www.digitalocean.com/community/tutorials/how-to-scale-your-mastodon-server
+

roles/docker-mastodon/templates/.env.production.j2

@@ -24,3 +24,7 @@ SMTP_AUTH_METHOD=plain
 SMTP_OPENSSL_VERIFY_MODE=none
 SMTP_ENABLE_STARTTLS=auto
 SMTP_FROM_ADDRESS=Mastodon <{{system_email_username}}>
+
+ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY= {{mastodon_active_record_encryption_deterministic_key}}
+ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT={{mastodon_active_record_encryption_key_derivation_salt}}
+ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY={{mastodon_active_record_encryption_primary_key}}

roles/docker-mastodon/templates/docker-compose.yml.j2

@@ -21,7 +21,7 @@ services:
 {% include 'templates/docker/container/networks.yml.j2' %}
 
   streaming:
-    image: ghcr.io/mastodon/mastodon:{{mastodon_version}}
+    image: ghcr.io/mastodon/mastodon-streaming:{{mastodon_version}}
     restart: {{docker_restart_policy}}
     env_file: .env.production
     command: node ./streaming

roles/update-docker/files/update-docker.py

@@ -25,12 +25,6 @@ def run_command(command):
 def git_pull():
     """
     Checks whether the Git repository in the specified directory is up to date and performs a git pull if necessary.
-
-    Args:
-        directory (str): The path to the directory of the Git repository.
-
-    Returns:
-        bool: True if a git pull was performed, otherwise False.
     """
     print(f"Checking if the git repository is up to date.")
     local = subprocess.check_output("git rev-parse @", shell=True).decode().strip()
@@ -39,12 +33,15 @@ def git_pull():
     if local != remote:
         print("Repository is not up to date. Performing git pull.")
         run_command("git pull")
-        return True;
+        return True
     
     print("Repository is already up to date.")
-    return False;
+    return False
         
 def get_image_digests(directory):
+    """
+    Retrieves the image digests for all images in the specified Docker Compose project.
+    """
     compose_project = os.path.basename(directory)
     try:
         images_output = subprocess.check_output(
@@ -59,16 +56,18 @@ def get_image_digests(directory):
             raise  # Other errors are still raised
 
 def is_any_service_up():
+    """
+    Checks if any Docker services are currently running.
+    """
     process = subprocess.Popen("docker-compose ps -q", shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
     output, _ = process.communicate()
     service_ids = output.decode().strip().splitlines()
-
-    # Check if there are any service containers up
-    if not service_ids:
-        return False  # No services are up
-    return True  # At least one service is up
+    return bool(service_ids)
 
 def pull_docker_images():
+    """
+    Pulls the latest Docker images for the project.
+    """
     print("Pulling docker images.")
     try:
         run_command("docker-compose pull")
@@ -82,13 +81,16 @@ def pull_docker_images():
     return False
 
 def update_docker(directory):
+    """
+    Checks for updates to Docker images and rebuilds containers if necessary.
+    """
     print(f"Checking for updates to Docker images in {directory}.")        
     before_digests = get_image_digests(directory)
     need_to_build = pull_docker_images()
     after_digests = get_image_digests(directory)
     if before_digests != after_digests:
         print("Changes detected in image digests. Rebuilding containers.")
-        need_to_build=True
+        need_to_build = True
         
     if need_to_build:
         run_command("docker-compose build")
@@ -96,7 +98,18 @@ def update_docker(directory):
     else:
         print("Docker images are up to date. No rebuild necessary.")
 
+def update_mastodon():
+    """
+    Runs the database migration for Mastodon to ensure all required tables are up to date.
+    """
+    print("Starting Mastodon database migration.")
+    run_command("docker compose exec -T web bash -c 'RAILS_ENV=production bin/rails db:migrate'")
+    print("Mastodon database migration complete.")
+
 def update_nextcloud():
+    """
+    Performs the necessary Nextcloud update procedures, including maintenance and app updates.
+    """
     print("Start Nextcloud update procedure.")
     update_procedure("docker-compose exec -T -u www-data application /var/www/html/occ upgrade")
     update_procedure("docker-compose exec -T -u www-data application /var/www/html/occ maintenance:repair")
@@ -104,12 +117,17 @@ def update_nextcloud():
     update_procedure("docker-compose exec -T -u www-data application /var/www/html/occ maintenance:mode --off")
     
 def update_discourse(directory):
+    """
+    Updates Discourse by running the rebuild command on the launcher script.
+    """
     os.chdir(directory)
     print("Start Discourse update procedure.")
     update_procedure("./launcher rebuild app")
-    
-# This procedure waits until the container is up
+
 def update_procedure(command):
+    """
+    Attempts to execute a command up to a maximum number of retries.
+    """
     max_attempts = 3
     for attempt in range(max_attempts):
         try:
@@ -120,10 +138,13 @@ def update_procedure(command):
                 print(f"Attempt {attempt + 1} failed, retrying in 60 seconds...")
                 time.sleep(60)  # Wait for 60 seconds before retrying
             else:
-                print("All attempts to update Nextcloud apps have failed.")
+                print("All attempts to update have failed.")
                 raise  # Re-raise the last exception after all attempts fail
 
 def start_docker(directory):
+    """
+    Starts or restarts Docker services in the specified directory.
+    """
     if is_any_service_up():
         print(f"Restarting containers in {directory}.")
         run_command("docker-compose up -d --force-recreate")
@@ -151,7 +172,7 @@ if __name__ == "__main__":
                     update_discourse(dir_path)
                 else:
                     print("Discourse update skipped. No changes in git repository.")
-            if os.path.basename(dir_path) == "matrix":
+            elif os.path.basename(dir_path) == "matrix":
                 # No autoupdate for matrix is possible atm, 
                 # due to the reason that the role has to be executed every time.
                 # The update has to be executed in the role
@@ -164,3 +185,5 @@ if __name__ == "__main__":
                 # Nextcloud needs additional update procedures
                 if os.path.basename(dir_path) == "nextcloud":
                     update_nextcloud()
+                elif os.path.basename(dir_path) == "mastodon":
+                    update_mastodon()