Solved ldap_user nextcloud var import bug (hopefully) and optimized README.md's and metas

This commit is contained in:
Kevin Veen-Birkenbach 2025-03-21 18:01:11 +01:00
parent e4a0473f9e
commit c51f935eff
No known key found for this signature in database
GPG Key ID: 44D8F11FD62F878E
20 changed files with 352 additions and 257 deletions

View File

@ -1,25 +1,30 @@
# Automated USB Backup
# Docker 🐳
## Description
This Ansible role automates data backups to a swappable USB device. It triggers the backup process automatically when the USB is mounted, allowing for customizable source and destination paths and integrating with systemd for reliable execution.
This Ansible role installs and manages Docker on Arch Linux systems. It ensures that Docker and Docker Compose are available, configured, and ready to run containerized workloads, while enabling seamless integration with system roles and administrative tasks.
Checkout the [administration reference](./Administration.md) for volume cleanup, container resets, and Docker network recovery.
## Overview
Optimized for Archlinux, this role ensures that backups are performed consistently with minimal manual intervention. It leverages efficient synchronization methods and provides a seamless integration with systemd to manage the backup service.
Tailored for Arch Linux, this role handles the installation of Docker and Docker Compose using the systems package manager. It sets up a secure environment for managing Compose instances and ensures the Docker service is properly enabled and restarted. In addition, the role flags its state so that dependent roles can execute conditionally.
## Purpose
The primary purpose of this role is to simplify the backup process for systems that rely on removable USB devices. It is designed to reduce the risk of data loss by automating backups, ensuring that data is safely and consistently synchronized whenever the device is connected.
The purpose of this role is to automate the provisioning of Docker environments in a consistent and maintainable way. It reduces manual setup steps and enables clean integration with other infrastructure roles, making it ideal for production or homelab deployments.
## Features
- **Automatic Trigger:** Initiates the backup process immediately upon USB mount.
- **Customizable Paths:** Easily configure the source directory and backup destination.
- **Systemd Integration:** Manages the backup process via a dedicated systemd service.
- **Efficient Synchronization:** Utilizes rsync with incremental backup strategies for optimal performance.
- **Optimized for Archlinux:** Tailored for Archlinux systems using the rolling release model.
- **Installs Docker & Docker Compose:** Uses `pacman` to install necessary packages.
- **Service Management:** Automatically enables and restarts the Docker service.
- **Secure Directory Creation:** Creates a secure location for Docker Compose instance files.
- **Run-once Setup Logic:** Ensures idempotent execution by controlling task flow with internal flags.
- **System Role Integration:** Sets internal state (`docker_enabled`) for use by other CyMaIS roles.
## Credits 📝
Developed and maintained by **Kevin Veen-Birkenbach**. Special thanks to [OpenAI ChatGPT](https://chat.openai.com/share/a75ca771-d8a4-4b75-9912-c515ba371ae4) for its assistance in developing this role.
Developed and maintained by **Kevin Veen-Birkenbach**.
Learn more at [www.veen.world](https://www.veen.world)
Part of the [CyMaIS Project](https://github.com/kevinveenbirkenbach/cymais)
License: [CyMaIS NonCommercial License (CNCL)](https://s.veen.world/cncl)

View File

@ -124,7 +124,7 @@ run:
#- exec: rails r "User.find_by_email('{{users.administrator.email}}').update(username: '{{users.administrator.username}}')"
# The following code is just an inspiration, how to connect with the oidc account. as long as this is not set the admini account needs to be manually connected with oidc
# docker exec -it discourse_application rails runner "user = User.find_by_email('test@flock.town'); UserAuth.create(user_id: user.id, provider: 'oidc', uid: 'eindeutige_oidc_id', info: { name: user.username, email: user.email })"
# docker exec -it discourse_application rails runner "user = User.find_by_email('test@cymais.cloud'); UserAuth.create(user_id: user.id, provider: 'oidc', uid: 'eindeutige_oidc_id', info: { name: user.username, email: user.email })"
# OIDC Activation
- exec: rails r "SiteSetting.openid_connect_enabled = true"

View File

@ -1,4 +1,4 @@
# MariaDB Docker Ansible Role
# MariaDB
## Overview
This Ansible role facilitates the deployment of a MariaDB server using Docker. It is designed to ensure ease of installation and configuration, with the flexibility to adapt to different environments.
@ -25,3 +25,6 @@ Configure the role by setting the required variables. These can be set in the pl
## Contributing
Contributions to this project are welcome. Please submit issues and pull requests with your suggestions.
## Other Ressources
- [Reset Password for MariaDB/MySQL in Docker](https://wolfgang.gassler.org/reset-password-mariadb-mysql-docker/)

View File

@ -1,5 +1,7 @@
# Administration
Instructions for manual administrative operations like container login, config file edits, and post-update recovery actions.
## Modify Config 🔧
### Enter the Container
@ -12,180 +14,3 @@ Inside the container, install a text editor and edit the config:
```bash
apk add --no-cache nano && nano config/config.php
```
---
## Update 🔄
To update the Nextcloud container, execute the following commands on the server:
```bash
docker exec -it -u www-data nextcloud-application /var/www/html/occ maintenance:mode --on &&
export COMPOSE_HTTP_TIMEOUT=600 &&
export DOCKER_CLIENT_TIMEOUT=600 &&
docker-compose down
```
Afterwards, update the ***applications.nextcloud.version*** variable to the next version and run this repository with this Ansible role.
> **Note:**
> It is only possible to update from one to the next major version at a time.
> Wait for the update to finish.
Verify the update by checking the logs:
```bash
docker-compose logs application
```
and
```bash
docker-compose exec -it application top
```
If Nextcloud remains in maintenance mode after the update, try the following:
```bash
docker exec -it -u www-data nextcloud-application/var/www/html/occ maintenance:mode --on
docker exec -it -u www-data nextcloud-application /var/www/html/occ upgrade
docker exec -it -u www-data nextcloud-application /var/www/html/occ maintenance:mode --off
```
If the update process fails, execute:
```bash
docker exec -it -u www-data nextcloud-application /var/www/html/occ maintenance:repair --include-expensive
```
and disable any non-functioning apps.
---
## Recover Latest Backup 💾
```bash
cd {{path_docker_compose_instances}}nextcloud &&
docker-compose down &&
docker-compose exec -i database mysql -u nextcloud -pPASSWORT nextcloud < "/Backups/$(sha256sum /etc/machine-id | head -c 64)/backup-docker-to-local/latest/nextcloud_database/sql/backup.sql" &&
cd {{path_administrator_scripts}}backup-docker-to-local &&
bash ./recover-docker-from-local.sh "nextcloud_data" "$(sha256sum /etc/machine-id | head -c 64)"
```
---
## Database Management 🗄️
### Database Access
To access the database, execute:
```bash
docker-compose exec -it database mysql -u nextcloud -D nextcloud -p
```
### Recreate Database with New Volume
```bash
docker-compose run --detach --name database --env MYSQL_USER="nextcloud" --env MYSQL_PASSWORD=PASSWORD --env MYSQL_ROOT_PASSWORD=PASSWORD --env MYSQL_DATABASE="nextcloud" -v nextcloud_database:/var/lib/mysql
```
---
## OCC (Nextcloud Command Line) 🔧
To use OCC, run:
```bash
docker-compose exec -it -u www-data application /var/www/html/occ
```
### User Administration
#### List Users
```bash
docker compose exec -it -u www-data application php occ user:list
```
#### Sync Users
```bash
docker compose exec -it -u www-data application php occ user:sync
```
#### Create user via CLI
```bash
docker compose exec -it -u www-data application php occ user:add {{username}}
```
#### Make user admin via cli
```bash
docker compose exec -it -u www-data application php occ group:adduser admin {{username}}
```
#### Delete user via CLI
```bash
docker compose exec -it -u www-data application php occ user:delete {{username}}
```
---
### App Administration
```bash
docker compose exec -u www-data application php occ config:list {{app_name}}
```
### Initialize Duplicates
```bash
docker-compose exec -it -u www-data application /var/www/html/occ duplicates:find-all --output
```
### Unlock Files
```bash
docker-compose exec -it -u www-data application /var/www/html/occ maintenance:mode --on
docker-compose exec -it nextcloud_database_1 mysql -u nextcloud -pPASSWORD1234132 -D nextcloud -e "delete from oc_file_locks where 1"
docker-compose exec -it -u www-data application /var/www/html/occ maintenance:mode --off
```
---
## Apps
### App Relevant Tables 🗃️
- `oc_appconfig`
- `oc_migrations`
### Cospend
#### Relevant SQL Commands for Cospend
Debugguging Migrations:
https://github.com/julien-nc/cospend-nc/issues/325
```sql
-- Show all Cospend Tables
SHOW TABLES where Tables_in_nextcloud LIKE "%cospend%";
-- Show Cospend Configuration
SELECT * FROM `oc_appconfig` WHERE appid LIKE "%cospend%";
-- Show Cospend Database Migrations
SELECT * FROM `oc_migrations` WHERE app LIKE "%cospend%";
```
# IAM
IAM(Identity and Access Management) is setup via Keycloak and LDAP.
## OpenID Connect (OIDC) Support 🔐
OIDC is supported in this role—for example, via **Keycloak**. OIDC-specific tasks are included when enabled, allowing integration of external authentication providers seamlessly.
### Verify OIDC Configuration
```bash
docker compose exec -u www-data application /var/www/html/occ config:app:get sociallogin custom_providers
```
## LDAP
More information: https://docs.nextcloud.com/server/latest/admin_manual/configuration_user/user_auth_ldap.html
## Get all relevant entries except password
```sql
SELECT * FROM `oc_appconfig` WHERE appid LIKE "%ldap%" and configkey != "s01ldap_agent_password";
```
## Update User with LDAP values
```bash
docker compose exec -it -u www-data application php occ ldap:check-user --update {{username}}
```
## Federation
If users are just created via Keycloak and not via LDAP, they have a different username. Due to this reaso concider to use LDAP to guaranty that the username is valid.

View File

@ -0,0 +1,26 @@
# Nextcloud Applications
Details on specific apps like Cospend, including related SQL queries and debugging tips.
## App Relevant Tables 🗃️
- `oc_appconfig`
- `oc_migrations`
## LDAP
## Cospend
### Relevant SQL Commands for Cospend
Debugguging Migrations:
https://github.com/julien-nc/cospend-nc/issues/325
```sql
-- Show all Cospend Tables
SHOW TABLES where Tables_in_nextcloud LIKE "%cospend%";
-- Show Cospend Configuration
SELECT * FROM `oc_appconfig` WHERE appid LIKE "%cospend%";
-- Show Cospend Database Migrations
SELECT * FROM `oc_migrations` WHERE app LIKE "%cospend%";
```

View File

@ -0,0 +1,4 @@
# Administration
## Other Resources
- [Nextcloud Docker Example with Nginx Proxy, MariaDB, and FPM](https://github.com/nextcloud/docker/blob/master/.examples/docker-compose/with-nginx-proxy/mariadb/fpm/docker-compose.yml)

View File

@ -0,0 +1,15 @@
# Database Management (local)
To manage the database if you installed it locally use the following comments. If you have created the database via the central database option, look for the related documentation.
## Database Access
To access the database, execute:
```bash
docker-compose exec -it database mysql -u nextcloud -D nextcloud -p
```
### Recreate Database with New Volume
```bash
docker-compose run --detach --name database --env MYSQL_USER="nextcloud" --env MYSQL_PASSWORD=PASSWORD --env MYSQL_ROOT_PASSWORD=PASSWORD --env MYSQL_DATABASE="nextcloud" -v nextcloud_database:/var/lib/mysql
```

View File

@ -0,0 +1,32 @@
# Identity and Access Management
IAM(Identity and Access Management) is setup via Keycloak and LDAP.
## OpenID Connect (OIDC) Support 🔐
OIDC is supported in this role—for example, via **Keycloak**. OIDC-specific tasks are included when enabled, allowing integration of external authentication providers seamlessly.
### Verify OIDC Configuration
```bash
docker compose exec -u www-data application /var/www/html/occ config:app:get sociallogin custom_providers
```
## LDAP
More information: https://docs.nextcloud.com/server/latest/admin_manual/configuration_user/user_auth_ldap.html
## Get all relevant entries except password
```sql
SELECT * FROM `oc_appconfig` WHERE appid LIKE "%ldap%" and configkey != "s01ldap_agent_password";
```
## Update User with LDAP values
```bash
docker compose exec -it -u www-data application php occ ldap:check-user --update {{username}}
```
## Federation
If users are just created via Keycloak and not via LDAP, they have a different username. Due to this reaso concider to use LDAP to guaranty that the username is valid.

View File

@ -0,0 +1,55 @@
# OCC (Nextcloud Command Line) 🔧
Reference for frequently used OCC commands, including user and app management.
## General Use
To use OCC, run:
```bash
docker-compose exec -it -u www-data application /var/www/html/occ
```
## User Administration
### List Users
```bash
docker compose exec -it -u www-data application php occ user:list
```
### Sync Users
```bash
docker compose exec -it -u www-data application php occ user:sync
```
### Create user via CLI
```bash
docker compose exec -it -u www-data application php occ user:add {{username}}
```
### Make user admin via cli
```bash
docker compose exec -it -u www-data application php occ group:adduser admin {{username}}
```
### Delete user via CLI
```bash
docker compose exec -it -u www-data application php occ user:delete {{username}}
```
---
## App Administration
```bash
docker compose exec -u www-data application php occ config:list {{app_name}}
```
## Initialize Duplicates
```bash
docker-compose exec -it -u www-data application /var/www/html/occ duplicates:find-all --output
```
## Unlock Files
```bash
docker-compose exec -it -u www-data application /var/www/html/occ maintenance:mode --on
docker-compose exec -it nextcloud_database_1 mysql -u nextcloud -pPASSWORD1234132 -D nextcloud -e "delete from oc_file_locks where 1"
docker-compose exec -it -u www-data application /var/www/html/occ maintenance:mode --off
```

View File

@ -1,24 +1,45 @@
# Nextcloud Server
# Nextcloud
This repository contains an Ansible role for deploying and managing [Nextcloud](https://nextcloud.com/) using [Docker](https://www.docker.com/). It covers configuration modifications, updates, backups, database management, and more. Additionally, OIDC (OpenID Connect) is supported (for example, via **Keycloak**).
---
## Description
## Other Resources
This Ansible role provisions a production-grade **Nextcloud** deployment using Docker Compose. It includes support for LDAP and OIDC authentication, Redis caching, secure configuration management, and declarative plugin control via `occ`. The setup is modular and integrates cleanly into larger infrastructure environments.
- [Nextcloud Docker Example with Nginx Proxy, MariaDB, and FPM](https://github.com/nextcloud/docker/blob/master/.examples/docker-compose/with-nginx-proxy/mariadb/fpm/docker-compose.yml)
- [Nextcloud Upgrade via Docker by Goneuland](https://goneuland.de/nextcloud-upgrade-auf-neue-versionen-mittels-docker/)
- [Nextcloud Data Version Issue](https://help.nextcloud.com/t/cant-start-nextcloud-because-the-version-of-the-data-is-higher-than-the-docker-image-version-and-downgrading-is-not-supported/109438)
- [Nextcloud Docker Issue #1302](https://github.com/nextcloud/docker/issues/1302)
- [Update to Nextcloud 22 Failed Database Error](https://help.nextcloud.com/t/update-to-22-failed-with-database-error-updated/120682)
- [Nextcloud 21.0.0-beta1 Database Error](https://help.nextcloud.com/t/nc-update-to-21-0-0-beta1-exception-database-error/101124/4)
- [Reset Password for MariaDB/MySQL in Docker](https://wolfgang.gassler.org/reset-password-mariadb-mysql-docker/)
- [Ansible Docker Container and depends_on Issue](https://unix.stackexchange.com/questions/478855/ansible-docker/container/and-depends-on)
- [Docker Convenience Scripts by gdiepen](https://github.com/gdiepen/docker-convenience-scripts)
- [Issues After Upgrading to Nextcloud 21](https://help.nextcloud.com/t/several-issues-after-upgrading-to-nextcloud-21/113118/3)
- [Nextcloud Talk Plugin and Turnserver in Docker](https://forum.openmediavault.org/index.php?thread/31782-docker-nextcloud-talk-plugin-and-turnserver/)
- [Nextcloud Talk on Docker: Turn Server Issues](https://help.nextcloud.com/t/nextcloud-talk-im-docker/container/turn-server-auf-docker-host-kein-video/84133/10)
## Overview
---
*Enjoy and happy containerizing! 😄*
The role ensures consistent deployments, full automation, and secure configuration injection into `config.php` using additive includes. Authentication is handled through LDAP and OIDC (e.g., via Keycloak), and backup/recovery operations are fully supported.
## Purpose
To automate the deployment of **secure, extensible, and production-ready Nextcloud instances** using Docker and Ansible.
## Features
- Dockerized Nextcloud (PHP-FPM, Nginx, Cron, Redis)
- Templated Nginx configuration (internal and external)
- Automated certificate and HTTPS proxy integration
- Healthcheck support
- Backup & recovery integration
- IAM & SOO
- Hundreds of integrated plugins
## Related Documentation
Here are all supporting documentation files within this role:
| Topic | Description |
|-------|-------------|
| [Applications](Applications.md) | SQL examples and debug notes for plugins like **Cospend** |
| [Architecture](Architecture.md) | Overview of architectural integrations |
| [Administration](Administration.md) | Manual operations like update, config edits, recovery |
| [Update](Update.md) | Step-by-step update and restore instructions |
| [OCC](OCC.md) | Nextcloud CLI usage guide (user management, config, maintenance) |
| [Database](Database.md) | Managing the database (local mode) |
| [IAM](IAM.md) | LDAP & OIDC Identity and Access Management |
## External Resources
- [Nextcloud Docker Documentation](https://github.com/nextcloud/docker)
- [Nextcloud Admin Manual](https://docs.nextcloud.com/server/latest/admin_manual/)
- [LDAP Integration Guide](https://docs.nextcloud.com/server/latest/admin_manual/configuration_user/user_auth_ldap.html)
- [OIDC Login Plugin (pulsejet)](https://github.com/pulsejet/nextcloud-oidc-login)
- [Sociallogin Plugin (Official)](https://apps.nextcloud.com/apps/sociallogin)

View File

@ -0,0 +1,53 @@
# Update 🔄
To update the Nextcloud container, execute the following commands on the server:
```bash
docker exec -it -u www-data nextcloud-application /var/www/html/occ maintenance:mode --on &&
export COMPOSE_HTTP_TIMEOUT=600 &&
export DOCKER_CLIENT_TIMEOUT=600 &&
docker-compose down
```
Afterwards, update the ***applications.nextcloud.version*** variable to the next version and run this repository with this Ansible role.
> **Note:**
> It is only possible to update from one to the next major version at a time.
> Wait for the update to finish.
Verify the update by checking the logs:
```bash
docker-compose logs application
```
and
```bash
docker-compose exec -it application top
```
If Nextcloud remains in maintenance mode after the update, try the following:
```bash
docker exec -it -u www-data nextcloud-application/var/www/html/occ maintenance:mode --on
docker exec -it -u www-data nextcloud-application /var/www/html/occ upgrade
docker exec -it -u www-data nextcloud-application /var/www/html/occ maintenance:mode --off
```
If the update process fails, execute:
```bash
docker exec -it -u www-data nextcloud-application /var/www/html/occ maintenance:repair --include-expensive
```
and disable any non-functioning apps.
---
## Recover Latest Backup 💾
```bash
cd {{path_docker_compose_instances}}nextcloud &&
docker-compose down &&
docker-compose exec -i database mysql -u nextcloud -pPASSWORT nextcloud < "/Backups/$(sha256sum /etc/machine-id | head -c 64)/backup-docker-to-local/latest/nextcloud_database/sql/backup.sql" &&
cd {{path_administrator_scripts}}backup-docker-to-local &&
bash ./recover-docker-from-local.sh "nextcloud_data" "$(sha256sum /etc/machine-id | head -c 64)"
```
## Other Resources
- [Nextcloud Upgrade via Docker by Goneuland](https://goneuland.de/nextcloud-upgrade-auf-neue-versionen-mittels-docker/)

View File

@ -0,0 +1,30 @@
---
galaxy_info:
author: "Kevin Veen-Birkenbach"
description: >
Provisions a secure and production-ready Nextcloud instance,
with support for LDAP, OIDC, SSO, and automated plugin configuration.
license: "CyMaIS NonCommercial License (CNCL)"
license_url: "https://s.veen.world/cncl"
company: |
Kevin Veen-Birkenbach
Consulting & Coaching Solutions
https://www.veen.world
min_ansible_version: "2.9"
platforms:
- name: Linux
versions:
- all
galaxy_tags:
- nextcloud
- docker
- nginx
- oidc
- ldap
- automation
- cloud
- web
- php
repository: "https://s.veen.world/cymais"
issue_tracker_url: "https://s.veen.world/cymaisissues"
documentation: "https://s.veen.world/cymais/docker-nextcloud"

View File

@ -0,0 +1,2 @@
# Variables
This folder contains the Nextcloud Variables

View File

@ -11,7 +11,7 @@ domain: "{{domains[application_id]}}
http_port: "{{ ports.localhost.http[application_id] }}" # Port at which nextcloud is reachable in the local network
# Localhost
nextcloud_localhost_plugin_configuration_directory: "{{role_path}}/plugins/" # Folder in which the files for the plugin configuration are stored
nextcloud_localhost_plugin_configuration_directory: "{{role_path}}/vars/plugins/" # Folder in which the files for the plugin configuration are stored
# Host

View File

@ -1 +1,2 @@
This folder contains the plugin specific configurations which willö be applied
# Plugins
This folder contains the plugin specific configurations which will be applied

View File

@ -0,0 +1,43 @@
# Administration 🛠️
## List Unused Volumes
```bash
docker volume ls -q -f "dangling=true"
```
## Remove All Unused Volumes
```bash
docker volume rm $(docker volume ls -q -f "dangling=true")
```
## Network Issues Fixes
```bash
docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)
docker network prune -f
systemctl stop docker
rm -fv /var/lib/docker/network/files/local-kv.db
systemctl start docker
```
---
# Warning ⚠️
**Caution:** The following instructions will delete **all Docker containers and volumes** on your server.
Make sure you have backups or that you're certain you want to clean your Docker environment completely.
---
## Cleaning Up Docker Containers and Volumes 🧹
### Delete All Docker Containers
```bash
docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)
```
### Delete All Docker Volumes
```bash
docker volume rm $(docker volume ls -q)
```

View File

@ -2,52 +2,4 @@
This role is part of the [CyMaIS Project](https://github.com/kevinveenbirkenbach/cymais), maintained and developed by [Kevin Veen-Birkenbach](https://www.veen.world/).
---
## Maintenance 🛠️
### List Unused Volumes
```bash
docker volume ls -q -f "dangling=true"
```
### Remove All Unused Volumes
```bash
docker volume rm $(docker volume ls -q -f "dangling=true")
```
### Network Issues Fixes
```bash
docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)
docker network prune -f
systemctl stop docker
rm -fv /var/lib/docker/network/files/local-kv.db
systemctl start docker
```
---
## Warning ⚠️
**Caution:** The following instructions will delete **all Docker containers and volumes** on your server.
Make sure you have backups or that you're certain you want to clean your Docker environment completely.
---
## Cleaning Up Docker Containers and Volumes 🧹
### Delete All Docker Containers
```bash
docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)
```
### Delete All Docker Volumes
```bash
docker volume rm $(docker volume ls -q)
```
---
Enjoy using this role and happy containerizing! 🎉

View File

@ -1,3 +1,31 @@
---
galaxy_info:
author: "Kevin Veen-Birkenbach"
description: >
Installs and maintains Docker.
license: "CyMaIS NonCommercial License (CNCL)"
license_url: "https://s.veen.world/cncl"
company: |
Kevin Veen-Birkenbach
Consulting & Coaching Solutions
https://www.veen.world
min_ansible_version: "2.9"
platforms:
- name: Linux
versions:
- all
galaxy_tags:
- docker
- container
- infrastructure
- automation
- cleanup
- linux
- system
repository: "https://s.veen.world/cymais"
issue_tracker_url: "https://s.veen.world/cymaisissues"
documentation: "https://s.veen.world/cymais/docker"
dependencies:
- backup-docker-to-local
- user-administrator