From 8d58aa2364ad9c23f9e1941bc3bd667b6d3c37c8 Mon Sep 17 00:00:00 2001 From: Kevin Veen-Birkenbach Date: Sun, 6 Apr 2025 05:01:09 +0200 Subject: [PATCH] Updated bigbluebutton role --- roles/backup-docker-to-local/tasks/main.yml | 2 +- roles/docker-bigbluebutton/Administration.md | 16 ---- roles/docker-bigbluebutton/README.md | 60 +++++++++---- roles/docker-bigbluebutton/handlers/main.yml | 70 ++++++++++----- roles/docker-bigbluebutton/meta/main.yml | 28 ++++++ roles/docker-bigbluebutton/tasks/main.yml | 49 +++++------ roles/docker-bigbluebutton/vars/main.yml | 18 ++-- roles/docker-compose/Administration.md | 11 +++ roles/docker-compose/README.md | 35 ++++++++ roles/docker-compose/meta/main.yml | 28 +++++- roles/docker-postgres/Administration.md | 2 + roles/docker-postgres/Upgrade.md | 92 ++++++++++++++++++++ 12 files changed, 321 insertions(+), 90 deletions(-) delete mode 100644 roles/docker-bigbluebutton/Administration.md create mode 100644 roles/docker-bigbluebutton/meta/main.yml create mode 100644 roles/docker-compose/Administration.md create mode 100644 roles/docker-compose/README.md create mode 100644 roles/docker-postgres/Upgrade.md diff --git a/roles/backup-docker-to-local/tasks/main.yml b/roles/backup-docker-to-local/tasks/main.yml index a2bb36b2..22d7bc5a 100644 --- a/roles/backup-docker-to-local/tasks/main.yml +++ b/roles/backup-docker-to-local/tasks/main.yml @@ -10,7 +10,7 @@ - name: Set fact for backup_docker_to_local_folder set_fact: - backup_docker_to_local_folder: "{{ pkgmgr_output.stdout }}" + backup_docker_to_local_folder: "{{ pkgmgr_output.stdout }}/" when: run_once_backup_docker_to_local is not defined - name: configure backup-docker-to-local-everything.cymais.service diff --git a/roles/docker-bigbluebutton/Administration.md b/roles/docker-bigbluebutton/Administration.md deleted file mode 100644 index a94e1076..00000000 --- a/roles/docker-bigbluebutton/Administration.md +++ /dev/null @@ -1,16 +0,0 @@ -## Administration - -## cleanup -```bash - docker compose down -v -``` - -## check container status -```bash -watch -n 2 "docker compose ps -a" -``` - -## database access -```bash - sudo docker-compose exec -it postgres psql -U postgres -``` \ No newline at end of file diff --git a/roles/docker-bigbluebutton/README.md b/roles/docker-bigbluebutton/README.md index 87a8cce9..4637d80e 100644 --- a/roles/docker-bigbluebutton/README.md +++ b/roles/docker-bigbluebutton/README.md @@ -1,19 +1,47 @@ -# docker bigbluebutton -@TODO Database needs to be decoupled +# Docker BigBlueButton πŸ“‘ -Role to deploy [BigBlueButton](https://bigbluebutton.org/). +## Description -## SSO -- https://docs.bigbluebutton.org/greenlight/v3/external-authentication/ +This Ansible role deploys [BigBlueButton](https://bigbluebutton.org/) using Docker Compose. It includes support for Greenlight, OIDC, LDAP, TURN/STUN, health checks, and a modular `.env` setup. This role is ideal for educational institutions and teams requiring a self-hosted video conferencing solution. -## Other Resources -- https://github.com/bigbluebutton/docker -- https://docs.bigbluebutton.org/greenlight/gl-install.html#setting-bigbluebutton-credentials -- https://goneuland.de/big-blue-button-mit-docker-und-traefik-installieren/ -- https://github.com/docker/compose/issues/4799 -- https://www.cyberciti.biz/faq/linux-command-to-remove-virtual-interfaces-or-network-aliases/ -- https://www.cyberciti.biz/faq/linux-restart-network-interface/ -- https://stackoverflow.com/questions/53347951/docker-network-not-found -- https://github.com/bigbluebutton/docker/issues/325 -- https://mattdyson.org/blog/2024/11/self-hosting-bluesky-pds/ -- https://atproto.com/specs/handle#handle-resolution @toto micoservice fΓΌr dynamische did auflΓΆsung implementieren +> πŸ”§ **Note**: The database layer should be decoupled in a future release to improve modularity and integration. + +## Overview + +This role provides a fully automated deployment of [BigBlueButton](https://bigbluebutton.org/) using Docker Compose on Arch Linux. It manages the entire lifecycle of the deployment, from cloning the upstream Docker repository and generating the `.env` configuration to customizing `docker-compose.yml` for volume usage, WebSocket proxying, and optional LDAP/OIDC integration. + +The setup includes conditional Greenlight activation, WebRTC support via TURN/STUN, and various fixes for known container orchestration issues. The role is modular and integrates seamlessly with the CyMaIS infrastructure, including reverse proxy configuration, domain management, and secrets templating. + +By default, BigBlueButton is deployed with best-practice hardening, modular secrets, and support for multiple authentication methods and scalable storage backends. + +## Features + +- 🐳 **Docker-based** deployment via official [bigbluebutton/docker](https://github.com/bigbluebutton/docker) +- βœ… **Greenlight** (v3) frontend support +- πŸ” **SSO with OIDC & LDAP** (optional) +- 🧱 Automatic `.env` templating and domain/Nginx integration +- πŸ›  Volume patching and Docker Compose customization +- πŸ“¬ SMTP integration and Greenlight admin creation +- πŸ§ͺ Workarounds for known Docker Compose or Etherpad issues + +## Single Sign-On (SSO) + +- Docs: [External Authentication](https://docs.bigbluebutton.org/greenlight/v3/external-authentication/) +- Supports: + - βœ… OpenID Connect (OIDC) + - βœ… LDAP (with custom DN and filters) + - 🧩 Custom OAuth2 flows via ENV vars + +## System Requirements + +- Arch Linux with Docker, Compose, and Nginx roles pre-installed +- DNS and reverse proxy configuration using `nginx-docker-reverse-proxy` +- Functional email system for Greenlight SMTP + +## Important Resources + +- [BigBlueButton Docker Docs](https://docs.bigbluebutton.org/greenlight/gl-install.html#setting-bigbluebutton-credentials) +- [Networking Fixes & Issues](https://stackoverflow.com/questions/53347951/docker-network-not-found) +- [Traefik + Docker Tutorial](https://goneuland.de/big-blue-button-mit-docker-und-traefik-installieren/) +- [Etherpad Healthcheck Bug](https://chatgpt.com/c/67a0fc7e-5104-800f-bb6b-3731e2f83b7b) +- [Virtual Interfaces Cleanup](https://www.cyberciti.biz/faq/linux-command-to-remove-virtual-interfaces-or-network-aliases/) \ No newline at end of file diff --git a/roles/docker-bigbluebutton/handlers/main.yml b/roles/docker-bigbluebutton/handlers/main.yml index bd2db04e..2852031e 100644 --- a/roles/docker-bigbluebutton/handlers/main.yml +++ b/roles/docker-bigbluebutton/handlers/main.yml @@ -2,29 +2,35 @@ - name: create docker-compose.yml for bigbluebutton command: cmd: bash ./scripts/generate-compose - chdir: "{{docker_compose.directories.instance}}" + chdir: "{{ bbb_repository_directory }}" environment: COMPOSE_HTTP_TIMEOUT: 600 DOCKER_CLIENT_TIMEOUT: 600 listen: setup bigbluebutton -- name: replace postgres bind mount by volume mount - replace: - path: "{{docker_compose_file}}" - regexp: '\./postgres-data:/var/lib/postgresql/data' - replace: 'database:/var/lib/postgresql/data' +- name: Copy docker-compose.yml from origin to final location + ansible.builtin.copy: + src: "{{ docker_compose_file_origine }}" + dest: "{{ docker_compose_file_final }}" + remote_src: yes listen: setup bigbluebutton -- name: replace greenlight bind mount by volume mount - replace: - path: "{{docker_compose_file}}" - regexp: '\./greenlight-data:/usr/src/app/storage' - replace: 'greenlight:/usr/src/app/storage' +- name: Replace bind mounts by named volume mounts + ansible.builtin.replace: + path: "{{ docker_compose_file_final }}" + regexp: "{{ item.regexp }}" + replace: "{{ item.replace }}" + loop: + - { regexp: '\./data/postgres:/var/lib/postgresql/data', replace: 'database:/var/lib/postgresql/data' } + - { regexp: '\./data/bigbluebutton:/var/bigbluebutton', replace: 'bigbluebutton:/var/bigbluebutton' } + - { regexp: '\./data/freeswitch-meetings:/var/freeswitch/meetings', replace: 'freeswitch:/var/freeswitch/meetings' } + - { regexp: '\./data/greenlight:/usr/src/app/storage', replace: 'greenlight:/usr/src/app/storage' } + - { regexp: '\./data/mediasoup:/var/mediasoup', replace: 'mediasoup:/var/mediasoup' } listen: setup bigbluebutton - name: add volume to redis lineinfile: - path: "{{ docker_compose_file }}" + path: "{{ docker_compose_file_final }}" insertafter: "^\\s*redis:" line: " volumes:\n - redis:/data" firstmatch: yes @@ -32,7 +38,7 @@ - name: add volume to coturn lineinfile: - path: "{{ docker_compose_file }}" + path: "{{ docker_compose_file_final }}" insertafter: "- ./mod/coturn/turnserver.conf:/etc/coturn/turnserver.conf" line: " - coturn:/var/lib/coturn" listen: setup bigbluebutton @@ -40,30 +46,46 @@ # Implemented due to etherpad health bug. # @todo Remove when health check is working fine # @see https://chatgpt.com/c/67a0fc7e-5104-800f-bb6b-3731e2f83b7b -- name: "Update docker-compose.yml for Etherpad health check" - lineinfile: - line: " healthcheck:\n test: [\"CMD\", \"curl\", \"-f\", \"http://127.0.0.1:9001\"]\n interval: 30s\n timeout: 10s\n retries: 5\n start_period: 10s" - path: "{{docker_compose_file}}" - insertafter: "etherpad:" - listen: setup bigbluebutton +#- name: "Update docker-compose.yml for Etherpad health check" +# lineinfile: +# line: " healthcheck:\n test: [\"CMD\", \"curl\", \"-f\", \"http://127.0.0.1:9001\"]\n interval: 30s\n timeout: 10s\n retries: 5\n start_period: 10s" +# path: "{{docker_compose_file_final}}" +# insertafter: "etherpad:" +# listen: setup bigbluebutton -- name: add volumes to docker compose +- name: Add volumes block after services in docker compose blockinfile: - path: "{{docker_compose_file}}" - block: |2 + path: "{{ docker_compose_file_final }}" + block: | + volumes: database: greenlight: redis: coturn: + #freeswitch: + bigbluebutton: marker: "# {mark} ANSIBLE MANAGED BLOCK FOR VOLUMES" - insertafter: "html5-static:" + insertbefore: "^services:" + listen: setup bigbluebutton + +- name: Replace all './' with '/services/' in docker-compose.yml + ansible.builtin.replace: + path: "{{ docker_compose_file_final }}" + regexp: '\./' + replace: './services/' + listen: setup bigbluebutton + +- name: docker compose pull bigbluebutton + command: + cmd: "docker-compose pull" + chdir: "{{ bbb_repository_directory }}" listen: setup bigbluebutton - name: docker compose up bigbluebutton command: cmd: "docker-compose -p bigbluebutton up -d --force-recreate{% if mode_cleanup | bool %} --remove-orphans{% endif %}" # Don't use the --build flag here. This leads to bugs - chdir: "{{docker_compose.directories.instance}}" + chdir: "{{ docker_compose.directories.instance }}" environment: COMPOSE_HTTP_TIMEOUT: 600 DOCKER_CLIENT_TIMEOUT: 600 diff --git a/roles/docker-bigbluebutton/meta/main.yml b/roles/docker-bigbluebutton/meta/main.yml new file mode 100644 index 00000000..4cec59e3 --- /dev/null +++ b/roles/docker-bigbluebutton/meta/main.yml @@ -0,0 +1,28 @@ +--- +galaxy_info: + author: "Kevin Veen-Birkenbach" + description: "Deploys BigBlueButton with Greenlight and SSO" + license: "CyMaIS NonCommercial License (CNCL)" + license_url: "https://s.veen.world/cncl" + company: | + Kevin Veen-Birkenbach + Consulting & Coaching Solutions + https://www.veen.world + min_ansible_version: "2.9" + platforms: + - name: Archlinux + versions: + - rolling + galaxy_tags: + - docker + - bigbluebutton + - conferencing + - education + - greenlight + - sso + - oidc + - ldap + - archlinux + repository: https://s.veen.world/cymais + issue_tracker_url: https://s.veen.world/cymaisissues + documentation: https://s.veen.world/cymais diff --git a/roles/docker-bigbluebutton/tasks/main.yml b/roles/docker-bigbluebutton/tasks/main.yml index 95484da7..9fc81837 100644 --- a/roles/docker-bigbluebutton/tasks/main.yml +++ b/roles/docker-bigbluebutton/tasks/main.yml @@ -3,24 +3,18 @@ include_role: name: docker-compose -# Leave this in the code until big blue button was working for a while. -# This is necessary due to the reason that big blue button wasn't fully tested after refactoring -# -#- name: "include task certbot-and-globals.yml" -# include_tasks: certbot-and-globals.yml -# -#- name: configure {{domains[application_id]}}.conf -# template: -# src: "nginx-proxy.conf.j2" -# dest: "{{nginx.directories.http.servers}}{{domains[application_id]}}.conf" -# notify: restart nginx - - name: "include role nginx-domain-setup for {{application_id}}" include_role: name: nginx-domain-setup - vars: - domain: "{{ domains[application_id] }}" - http_port: "{{ ports.localhost.http[application_id] }}" + +- name: pull docker repository + git: + repo: "https://github.com/bigbluebutton/docker.git" + dest: "{{ bbb_repository_directory }}" + update: yes + recursive: yes + version: main + notify: setup bigbluebutton - name: configure websocket_upgrade.conf copy: @@ -28,20 +22,23 @@ dest: "{{nginx.directories.http.maps}}websocket_upgrade.conf" notify: restart nginx -- name: pull docker repository - git: - repo: "https://github.com/bigbluebutton/docker.git" - dest: "{{docker_compose.directories.instance}}" - update: yes - recursive: yes - version: main - notify: setup bigbluebutton - ignore_errors: true +- name: "Remove directory {{ docker_compose.directories.env }}" + ansible.builtin.file: + path: "{{ docker_compose.directories.env }}" + state: absent - name: deploy .env - template: src=env.j2 dest={{docker_compose.directories.instance}}/.env + template: + src: env.j2 + dest: "{{ bbb_env_file_origine }}" notify: setup bigbluebutton +- name: Create symbolic link from .env file to target location + ansible.builtin.file: + src: "{{ bbb_env_file_origine }}" + dest: "{{ bbb_env_file_link }}" + state: link + - name: flush docker service meta: flush_handlers @@ -53,7 +50,7 @@ - name: create admin command: cmd: docker compose exec greenlight bundle exec rake admin:create - chdir: "{{docker_compose.directories.instance}}" + chdir: "{{ docker_compose.directories.instance }}" when: applications.bigbluebutton.setup | bool ignore_errors: true register: admin_creation_result diff --git a/roles/docker-bigbluebutton/vars/main.yml b/roles/docker-bigbluebutton/vars/main.yml index dce3cac9..f6d0f512 100644 --- a/roles/docker-bigbluebutton/vars/main.yml +++ b/roles/docker-bigbluebutton/vars/main.yml @@ -1,6 +1,12 @@ -application_id: "bigbluebutton" -docker_compose_file: "{{docker_compose.directories.instance}}docker-compose.yml" -database_instance: "bigbluebutton" -database_name: "greenlight-v3" -database_username: "postgres" -database_password: "{{bigbluebutton_postgresql_secret}}" \ No newline at end of file +application_id: "bigbluebutton" +bbb_repository_directory: "{{ docker_compose.directories.services }}" +docker_compose_file_origine: "{{ docker_compose.directories.services }}docker-compose.yml" +docker_compose_file_final: "{{ docker_compose.directories.instance }}docker-compose.yml" +database_instance: "bigbluebutton" +database_name: "greenlight-v3" +database_username: "postgres" +database_password: "{{bigbluebutton_postgresql_secret}}" +domain: "{{ domains[application_id] }}" +http_port: "{{ ports.localhost.http[application_id] }}" +bbb_env_file_link: "{{ docker_compose.directories.instance }}.env" +bbb_env_file_origine: "{{ bbb_repository_directory }}.env" \ No newline at end of file diff --git a/roles/docker-compose/Administration.md b/roles/docker-compose/Administration.md new file mode 100644 index 00000000..afb10781 --- /dev/null +++ b/roles/docker-compose/Administration.md @@ -0,0 +1,11 @@ +# Docker Compose + +## Delete all containers, networks and volumes +```bash + docker compose down -v +``` + +## Show the state of all containers +```bash +watch -n 2 "docker compose ps -a" +``` \ No newline at end of file diff --git a/roles/docker-compose/README.md b/roles/docker-compose/README.md new file mode 100644 index 00000000..7bf5352e --- /dev/null +++ b/roles/docker-compose/README.md @@ -0,0 +1,35 @@ +# Docker Compose 🧱 + +## Description + +This Ansible role manages Docker Compose project structures and workflows for applications on Arch Linux. It creates dedicated instance directories, manages `.env` and `docker-compose.yml` files, and provides automation logic for project reset, rebuild, and startup sequences. + +Refer to the [Docker Compose documentation](https://docs.docker.com/compose/), the [Arch Wiki – Docker](https://wiki.archlinux.org/title/Docker), and [Compose CLI reference](https://docs.docker.com/compose/cli-command/) for more details. + +## Overview + +This role creates a flexible directory layout for managing Docker Compose projects across environments. It ensures directories are initialized, optionally reset, and kept clean using internal flags like `mode_reset` or `mode_cleanup`. + +## Purpose + +To offer a centralized, extensible system for managing containerized applications using Docker Compose within the CyMaIS architecture. The role allows easy integration of services, secrets, configurations, and custom behaviors per application. + +## Features + +- **Dynamic Directory Structure:** Creates per-application instance folders for Compose setups. +- **Reset Logic:** Cleans previous Compose project files and data when `mode_reset` is enabled. +- **Handlers for Runtime Control:** Automatically builds, sets up, or restarts containers based on handlers. +- **Template-ready Service Files:** Predefined service base and health check templates. +- **Integration Support:** Compatible with `nginx-docker-reverse-proxy` and other CyMaIS service roles. + +## Administration Tips + +For administration tips checkout [this](Administration.md). + +## Credits πŸ“ + +Developed and maintained by **Kevin Veen-Birkenbach** +Learn more at [www.veen.world](https://www.veen.world) + +Part of the [CyMaIS Project](https://github.com/kevinveenbirkenbach/cymais) +License: [CyMaIS NonCommercial License (CNCL)](https://s.veen.world/cncl) \ No newline at end of file diff --git a/roles/docker-compose/meta/main.yml b/roles/docker-compose/meta/main.yml index d3606e63..88082909 100644 --- a/roles/docker-compose/meta/main.yml +++ b/roles/docker-compose/meta/main.yml @@ -1,2 +1,28 @@ +--- +galaxy_info: + author: "Kevin Veen-Birkenbach" + description: "Manages Docker Compose project structure and execution logic on Arch Linux." + license: "CyMaIS NonCommercial License (CNCL)" + license_url: "https://s.veen.world/cncl" + company: | + Kevin Veen-Birkenbach + Consulting & Coaching Solutions + https://www.veen.world + min_ansible_version: "2.9" + platforms: + - name: Archlinux + versions: + - rolling + galaxy_tags: + - docker + - compose + - container + - infrastructure + - devops + - automation + - archlinux + repository: https://s.veen.world/cymais + issue_tracker_url: https://s.veen.world/cymaisissues + documentation: https://s.veen.world/cymais dependencies: -- nginx-docker-reverse-proxy + - nginx-docker-reverse-proxy \ No newline at end of file diff --git a/roles/docker-postgres/Administration.md b/roles/docker-postgres/Administration.md index 8deb63cf..c1fa7d60 100644 --- a/roles/docker-postgres/Administration.md +++ b/roles/docker-postgres/Administration.md @@ -1,3 +1,5 @@ +# Administration + ## Root Access To access the database via the root account execute the following on the server: ```bash diff --git a/roles/docker-postgres/Upgrade.md b/roles/docker-postgres/Upgrade.md new file mode 100644 index 00000000..886a2b2a --- /dev/null +++ b/roles/docker-postgres/Upgrade.md @@ -0,0 +1,92 @@ +# PostgreSQL Docker Upgrade: Major Version Migration + +This guide explains how to safely upgrade a PostgreSQL Docker container from one major version to another (e.g., version 12 to 16) using a **dump and restore** method. This is the recommended approach in Docker environments. + +--- + +## ⚠️ Important +PostgreSQL data directories are **not compatible across major versions**. You cannot just point a newer version to the old data volume. You must export and re-import your data. + +--- + +## πŸ’Ύ Step 1: Start a temporary container with your current PostgreSQL version + +Replace `` with your current PostgreSQL version (e.g., `12`). + +```bash +docker run --rm -d \ + --name pg-old \ + -v pgdata:/var/lib/postgresql/data \ + postgres: +``` + +This container mounts your old data volume and runs the matching PostgreSQL version. + +--- + +## ⬇️ Step 2: Dump all databases + +```bash +docker exec pg-old pg_dumpall -U postgres > backup.sql +``` + +Stop the old container: + +```bash +docker stop pg-old +``` + +--- + +## πŸ’₯ Step 3: Remove the old data volume + +```bash +docker volume rm pgdata +``` + +⚠️ This will permanently delete your old PostgreSQL data files. Make sure you have a successful backup (`backup.sql`) before running this! + +--- + +## πŸ“¦ Step 4: Start a new container with your target PostgreSQL version + +Replace `` with the version you want to upgrade to (e.g., `16`). + +```bash +docker run --rm -d \ + --name pg-new \ + -v pgdata:/var/lib/postgresql/data \ + -e POSTGRES_PASSWORD=secret \ + postgres: +``` + +This creates a clean PostgreSQL instance with a fresh data directory. + +--- + +## ⬆️ Step 5: Restore your data + +```bash +cat backup.sql | docker exec -i pg-new psql -U postgres +``` + +This restores all roles, databases, and data into your new PostgreSQL instance. + +--- + +## βœ… Done! +You now have the target PostgreSQL version running with your old data successfully restored. + +--- + +## πŸ“ Tips +- Always test this procedure in a staging environment before running it in production. +- You can automate this with Ansible or a custom script. +- For large databases, consider using `pg_dump` per database and `pg_restore` with parallel jobs. + +--- + +## πŸ”— References +- [PostgreSQL Backup Documentation](https://www.postgresql.org/docs/current/backup-dump.html) +- [PostgreSQL Docker Image](https://hub.docker.com/_/postgres) +