Commit Graph
260 Commits
Author SHA1 Message Date
kevinveenbirkenbach 5f54382496 fix(svc-db-openldap): add libldap to Arch deps so python-ldap can load libldap.so.2
Include the libldap runtime package for Arch (dind runner) and keep openldap for
CLI tools, ensuring python-ldap imports successfully in the Ansible interpreter.

https://chatgpt.com/share/6964335a-29cc-800f-ae65-6a35c7cd2559
2026-01-12 12:29:33 +01:00
kevinveenbirkenbach a80ebf9f18 feat(svc-db-openldap): install OS build and runtime deps for python-ldap
Install distro-specific OpenLDAP, SASL, SSL, libffi and build toolchain packages
(Arch, Debian/Ubuntu, Fedora/CentOS) before installing python-ldap to ensure the
module can be built and imported in the Ansible runtime.

https://chatgpt.com/share/6964335a-29cc-800f-ae65-6a35c7cd2559
2026-01-12 11:45:15 +01:00
kevinveenbirkenbach d8d6349c0b debug(svc-db-openldap): add verbose python-ldap import sanity check
Run python-ldap import test in the Ansible interpreter and print version, module
path, and full traceback when MODE_DEBUG is enabled to simplify debugging.

https://chatgpt.com/share/6964335a-29cc-800f-ae65-6a35c7cd2559
2026-01-12 10:48:50 +01:00
kevinveenbirkenbach c27ab7c413 Set application_id: svc-db-openldap to solve reference bug 2026-01-12 01:10:32 +01:00
kevinveenbirkenbach 9e993c338e feat(pip): add global_install flag and fix python-ldap installation for Ansible interpreter
- Add global_install flag to sys-pip-install role (default: false)
- Allow optional system-wide installs via pip3 with --break-system-packages
- Install python-ldap via pip in Ansible interpreter for OpenLDAP role
- Refactor svc-db-openldap tasks into 01_core.yml with run-once gating
- Fix environment handling for conditional global pip installs
- Enable global pip installs for CLI tools (gitcon, ldapsm, backup tools, cleanup tools)

https://chatgpt.com/share/6963dd5d-cae0-800f-b8d7-82a3808be634
2026-01-11 18:26:49 +01:00
kevinveenbirkenbach ac48b01d06 Added sahred for ldap und oidc:
If oidc shared then keyclaok is used
if ldap shared tehn openldap is used

https://chatgpt.com/share/6962d2e6-7fdc-800f-ba83-9e150b19a5f0
2026-01-10 23:29:10 +01:00
kevinveenbirkenbach 04c9490745 Solved unit and integration tests 2026-01-10 13:07:12 +01:00
kevinveenbirkenbach e2bf636ba9 Changed features structure 2026-01-10 11:13:29 +01:00
kevinveenbirkenbach e7c464f988 refactor(openldap): install ldapsm via sys-pip-install
- Replace pkgmgr-based ldapsm installation with system pip
- Install ldapsm once before applying schema tasks
- Simplify schema task files by removing duplicate installs

https://chatgpt.com/share/695d2615-d664-800f-b821-5705c631bfe8
2026-01-06 17:30:06 +01:00
kevinveenbirkenbach dd729fd35d Introduced lifecycle into meta/main.yml roles 2026-01-06 14:42:52 +01:00
kevinveenbirkenbach 6e5516c3d7 Replaced 127.0.0.1 by DOCKER_REACH_HOST 2026-01-02 04:56:39 +01:00
kevinveenbirkenbach 4dcffa8601 Changed 127.0.0.1 to DOCKER_BIND_HOST 2026-01-02 04:26:41 +01:00
kevinveenbirkenbach 17cba11c99 Optimized administrator integration 2025-12-24 16:47:51 +01:00
kevinveenbirkenbach 1104d4710d Normalize docker service reachability via DOCKER_REACH_HOST
Introduce DOCKER_REACH_HOST as a single source of truth for service
reachability checks (wait_for), decoupled from DOCKER_BIND_HOST.

- Default reach host is 127.0.0.1 for host-based execution
- Automatically override to Docker bridge gateway when running
  inside a container (Docker socket mounted)
- Add custom docker_bridge_gateway module for reliable detection
- Update all wait_for tasks to use DOCKER_REACH_HOST
- Remove ad-hoc debug TCP probing logic

This ensures consistent behavior across host runs and
container-based control nodes without breaking existing setups.

https://chatgpt.com/share/694aaf8b-1fe4-800f-88ac-b5f6e508a2e3
2025-12-23 16:04:36 +01:00
kevinveenbirkenbach fd54faff65 refactor(ansible): replace pacman installs with package
Migrate most roles from community.general.pacman to the generic package module for better portability. Keep pacman where it is required (system upgrades and Arch-specific tasks). Split VirtualBox setup into an Arch-only task file and add a warning when skipped on non-Arch systems. Update integration test examples accordingly.

https://chatgpt.com/share/6947ce9f-3228-800f-8365-2253b5ed9451
2025-12-21 11:40:10 +01:00
kevinveenbirkenbach 7123841fb4 style: apply ruff autoformat 2025-12-17 16:37:21 +01:00
kevinveenbirkenbach 716ebef33b Refactor task includes and update variable handling for Ansible 2.20 migration
This commit updates multiple roles to ensure compatibility with Ansible 2.20.
Several include paths and task-loading mechanisms required adjustments,
as Ansible 2.20 applies stricter evaluation rules for complex Jinja expressions
and no longer resolves certain relative include paths the way Ansible 2.18 did.

Key changes:
- Replaced legacy once_finalize.yml and once_flag.yml with the new structure
  under tasks/utils/once/finalize.yml and tasks/utils/once/flag.yml.
- Updated all include_tasks statements to use 'path_join' with playbook_dir,
  ensuring deterministic and absolute file resolution across roles.
- Fixed all network helper includes by converting direct relative paths such as
  'roles/docker-compose/tasks/utils/network.yml' to proper Jinja-evaluated paths.
- Normalized MATOMO_* variable names for consistency with the updated variable
  scope behavior in Ansible 2.20.
- Removed deprecated patterns that were implicitly supported in Ansible 2.18
  but break under the more strict variable and path resolution model in 2.20.

These changes are part of the full migration step required to ensure the
infinito-nexus roles remain stable, deterministic, and forward-compatible with
Ansible 2.20.

Details of the discussion and reasoning can be found in this conversation:
https://chatgpt.com/share/69300a8d-24d4-800f-bec0-e895a695618a
2025-12-03 11:02:34 +01:00
kevinveenbirkenbach 8008afe0de Refactor user variable name from OPERNLDAP_USERS to OPENLDAP_USERS and add dynamic state handling for objectClass cleanup.
See conversation: https://chatgpt.com/share/692cab28-1ce0-800f-81da-712c8ea08e5c
2025-11-30 21:38:16 +01:00
kevinveenbirkenbach 26dfab147d Implement reserved username handling for users, LDAP and Keycloak
Add end-to-end support for reserved usernames and tighten CAPTCHA / Keycloak logic.

Changes:

- Makefile: rename EXTRA_USERS → RESERVED_USERNAMES and pass it as --reserved-usernames to the users defaults generator.

- cli/build/defaults/users.py: propagate  flag into generated users, add --reserved-usernames CLI option and mark listed accounts as reserved.

- Add reserved_users filter plugin with  and  helpers for Ansible templates and tasks.

- Add unit tests for reserved_users filters and the new reserved-usernames behaviour in the users defaults generator.

- group_vars/all/00_general.yml: harden RECAPTCHA_ENABLED / HCAPTCHA_ENABLED checks with default('') and explicit > 0 length checks.

- svc-db-openldap: introduce OPENLDAP_PROVISION_* flags, add OPENLDAP_PROVISION_RESERVED and OPERNLDAP_USERS to optionally exclude reserved users from provisioning.

- svc-db-openldap templates/tasks: switch role/group LDIF and user import loops to use OPERNLDAP_USERS instead of the full users dict.

- networks: assign dedicated subnet for web-app-roulette-wheel.

- web-app-keycloak vars: compute KEYCLOAK_RESERVED_USERNAMES_LIST and KEYCLOAK_RESERVED_USERNAMES_REGEX from users | reserved_usernames.

- web-app-keycloak user profile template: inject reserved-username regex into username validation pattern and improve error message, fix SSH public key attribute usage and add component name field.

- web-app-keycloak update/_update.yml: strip subComponents from component payloads before update and disable async/poll for easier debugging.

- web-app-keycloak tasks/main.yml: guard cleanup include with MODE_CLEANUP and keep reCAPTCHA update behind KEYCLOAK_RECAPTCHA_ENABLED.

- user/users defaults: mark system/service accounts (root, daemon, mail, admin, webmaster, etc.) as reserved so they cannot be chosen as login names.

- svc-prx-openresty vars: simplify OPENRESTY_CONTAINER lookup by dropping unused default parameter.

- sys-ctl-rpr-btrfs-balancer: simplify main.yml by removing the extra block wrapper.

- sys-daemon handlers: quote handler name for consistency.

Context: change set discussed and refined in ChatGPT on 2025-11-29 (Infinito.Nexus reserved usernames & Keycloak user profile flow). See conversation: https://chatgpt.com/share/692b21f5-5d98-800f-8e15-1ded49deddc9
2025-11-29 17:40:45 +01:00
kevinveenbirkenbach 61e138c1a6 Optimize OpenLDAP container resources for up to 5k users (1.25 CPU / 1.5GB RAM / 1024 PIDs). See https://chatgpt.com/share/68ef7228-4028-800f-8986-54206a51b9c1 2025-10-15 12:06:51 +02:00
kevinveenbirkenbach 6824e444b0 Changed bitnami images to legacy. See https://github.com/bitnami/containers/issues/83267. 2025-10-02 07:31:20 +02:00
kevinveenbirkenbach 9d8e06015f Added whitespaces 2025-09-23 00:59:55 +02:00
kevinveenbirkenbach 208848579d svc-db-openldap: make LDIF import idempotent, unify container var, and tidy role
- Add handlers/main.yml to load memberof/refint modules and import groups via docker exec
- Use OPENLDAP_CONTAINER consistently (replace OPENLDAP_NAME)
- Rename tasks/ldifs_creation.yml -> tasks/_ldifs_creation.yml and update includes
- Drop default param from get_app_conf calls; add explicit meta: flush_handlers
- docker-compose: honor OPENLDAP_NETWORK_EXPOSE_LOCAL | bool; minor formatting
- env template: formatting/comments consistency
- Remove unused 01_rbac_group.ldif.j2; rename 02_rbac_roles -> 01_rbac_roles and fix filter to LDAP
- vars: rename OPENLDAP_NAME -> OPENLDAP_CONTAINER; prune LDIF schema type

Conversation: https://chatgpt.com/share/68d1d25d-e788-800f-bfb6-13b1f5bc6121
2025-09-23 00:49:57 +02:00
kevinveenbirkenbach 7ca3a73f21 Normalized OpenLDAP variables 2025-09-22 21:02:24 +02:00
kevinveenbirkenbach 97e2d440b2 Normalized OpenLDAP constants 2025-09-22 19:08:11 +02:00
kevinveenbirkenbach fa6bb67a66 Removed whitespaces in templates: 2025-09-22 16:28:57 +02:00
kevinveenbirkenbach e6803e5614 refactor(ansible): normalize include_role syntax and unify host config paths via path_join
- Remove stray spaces after include_role: across many roles to ensure clean YAML and
  consistent linting/formatting.
- Listmonk:
  - Introduce LISTMONK_CONFIG_HOST = [ docker_compose.directories.config, 'config.toml' ] | path_join
  - Use that var in the template task (dest) and the docker-compose volume mount
- Matrix:
  - Build MATRIX_SYNAPSE_CONFIG_PATH_HOST, MATRIX_SYNAPSE_LOG_PATH_HOST, and
    MATRIX_ELEMENT_CONFIG_PATH_HOST via path_join
- Mobilizon:
  - Build mobilizon_host_conf_exs_file via path_join
  - Keep get_app_conf strictness unchanged (defaults to True in our filter), so behavior
    remains strict even though the explicit third arg was dropped
- Simpleicons:
  - Build server.js and package.json host paths via path_join
- Numerous web-app roles (Confluence, Discourse, EspoCRM, Friendica, Funkwhale, Gitea,
  GitLab, Jenkins, Joomla, Listmonk, Mailu, Mastodon, Matomo, Matrix, MediaWiki,
  Mobilizon, Moodle, Nextcloud, OpenProject, Peertube, Pixelfed, Pretix, Roulette Wheel,
  Snipe-IT, Syncope, Taiga, WordPress, XWiki, Yourls) and web-svc roles (coturn,
  libretranslate, simpleicons) updated for consistent include_role formatting

Why:
- path_join avoids double slashes and missing separators across different config roots
- Consistent include_role: formatting improves readability and prevents linter noise

Ref:
- Conversation: https://chatgpt.com/share/68d14711-727c-800f-b454-7dc4c3c1f4cb
2025-09-22 14:55:25 +02:00
kevinveenbirkenbach 4f8ce598a9 Mastodon: allow internal chess host & refactor var names; OpenLDAP: safer get_app_conf
- Add ALLOWED_PRIVATE_ADDRESSES to .env (from svc-db-postgres) to handle 422 Mastodon::PrivateNetworkAddressError
- Switch docker-compose to MASTODON_* variables and align vars/main.yml
- Always run 01_setup.yml during deployment (removed conditional flag)
- OpenLDAP: remove implicit True default on network.local to avoid unintended truthy behavior

Context: chess.infinito.nexus resolved to 192.168.200.30 (private IP) from Mastodon; targeted allowlist unblocks federation lookups.

Ref: https://chat.openai.com/share/REPLACE_WITH_THIS_CONVERSATION_LINK
2025-09-03 21:44:47 +02:00
kevinveenbirkenbach a8c88634b5 cleanup: remove unused handlers and add integration test for unused handlers
Removed obsolete handlers from roles (VirtualBox, backup-to-USB, OpenLDAP)
and introduced an integration test under tests/integration/test_handlers_invoked.py
that ensures all handlers defined in roles/*/handlers are actually notified
somewhere in the code base. This keeps the repository clean by preventing
unused or forgotten handlers from accumulating.

Ref: https://chatgpt.com/share/68b6b28e-4388-800f-87d2-34dfb34b8d36
2025-09-02 11:02:30 +02:00
kevinveenbirkenbach b02d88adc0 Refactored server roles for better readability 2025-09-01 18:08:35 +02:00
kevinveenbirkenbach 7791bd8c04 Implement filter checks: ensure all defined filters are used and remove dead code
Integration tests added/updated:
- tests/integration/test_filters_usage.py: AST-based detection of filter definitions (FilterModule.filters), robust Jinja detection ({{ ... }}, {% ... %}, {% filter ... %}), plus Python call tracking; fails if a filter is used only under tests/.
- tests/integration/test_filters_are_defined.py: inverse check — every filter used in .yml/.yaml/.j2/.jinja2/.tmpl must be defined locally. Scans only inside Jinja blocks and ignores pipes inside strings (e.g., lookup('pipe', "... | grep ... | awk ...")) to avoid false positives like trusted_hosts, woff/woff2, etc.

Bug fixes & robustness:
- Build regexes without %-string formatting to avoid ValueError from literal '%' in Jinja tags.
- Strip quoted strings in usage analysis so sed/grep/awk pipes are not miscounted as filters.
- Prevent self-matches in the defining file.

Cleanup / removal of dead code:
- Removed unused filter plugins and related unit tests:
  * filter_plugins/alias_domains_map.py
  * filter_plugins/get_application_id.py
  * filter_plugins/load_configuration.py
  * filter_plugins/safe.py
  * filter_plugins/safe_join.py
  * roles/svc-db-openldap/filter_plugins/build_ldap_nested_group_entries.py
  * roles/sys-ctl-bkp-docker-2-loc/filter_plugins/dict_to_cli_args.py
  * corresponding tests under tests/unit/*
- roles/svc-db-postgres/filter_plugins/split_postgres_connections.py: dropped no-longer-needed list_postgres_roles API; adjusted tests.

Misc:
- sys-stk-front-proxy/defaults/main.yml: clarified valid vhost_flavour values (comma-separated).

Ref: https://chatgpt.com/share/68b56bac-c4f8-800f-aeef-6708dbb44199
2025-09-01 11:47:51 +02:00
kevinveenbirkenbach cb66fb2978 Refactor LDAP variable schema to use top-level constant LDAP and nested ALL-CAPS keys.
- Converted group_vars/all/13_ldap.yml from lower-case to ALL-CAPS nested keys.
- Updated all roles, tasks, templates, and filter_plugins to reference LDAP.* instead of ldap.*.
- Fixed Keycloak JSON templates to properly quote Jinja variables.
- Adjusted svc-db-openldap filter plugins and unit tests to handle new LDAP structure.
- Updated integration test to only check uniqueness of TOP-LEVEL ALL-CAPS constants, ignoring nested keys.

See: https://chatgpt.com/share/68b01017-efe0-800f-a508-7d7e2f1c8c8d
2025-08-28 10:15:48 +02:00
kevinveenbirkenbach c182ecf516 Refactor and cleanup OIDC, desktop, and web-app roles
- Improved OIDC variable definitions (12_oidc.yml)
- Added account/security/profile URLs
- Restructured web-app-desktop tasks and JS handling
- Introduced oidc.js and iframe.js with runtime loader
- Fixed nginx.conf, LDAP, and healthcheck templates spacing
- Improved Lua injection for CSP and snippets
- Fixed typos (WordPress, receive, etc.)
- Added silent-check-sso nginx location

Conversation: https://chatgpt.com/share/68ae0060-4fac-800f-9f02-22592a4087d3
2025-08-26 20:44:05 +02:00
kevinveenbirkenbach a4f39ac732 Renamed webserver roles to more speakable names 2025-08-20 08:54:17 +02:00
kevinveenbirkenbach a5941763ff refactor: normalize Jinja2 spacing in volume paths and add async support in backup task
- Standardized spacing in {{ docker_compose.directories.volumes }} across multiple roles
- Added async and poll support to sys-bkp-docker-2-loc database seeding and file permission tasks
- Moved Installation.md for web-app-matrix into docs/ for better structure
2025-08-18 01:05:01 +02:00
kevinveenbirkenbach 5c9ca20e04 Optimized keycloak variables 2025-08-17 11:40:15 +02:00
kevinveenbirkenbach 022800425d THE HUGE REFACTORING CALENDER WEEK 33; Optimized Matrix and during this updated variables, and implemented better reset and cleanup mode handling, also solved some initial setup bugs 2025-08-15 15:15:48 +02:00
kevinveenbirkenbach 0228014d34 Replaced .infinito.service and .infinito.timer by SOFTWARE_NAME suffix, optimized LICENSE link and update OIDC Realm and ID conf 2025-08-14 14:39:18 +02:00
kevinveenbirkenbach c729edb525 Refactor async task handling
- Standardize async/poll usage with 'ASYNC_ENABLED | bool'
- Add async/poll parameters to Cloudflare, Nginx, Mailu, MIG, Nextcloud, and OpenLDAP tasks
- Update async configuration in 'group_vars/all/00_general.yml' to ensure boolean evaluation
- Allow CAA, cache, and DNS tasks to run asynchronously when enabled

https://chatgpt.com/share/689cd8cc-7fbc-800f-bd06-a667561573bf
2025-08-13 21:56:26 +02:00
kevinveenbirkenbach 597e9d5222 Refactor async execution handling across LDAP and Nextcloud roles
- Introduce global async configuration in group_vars/all/00_general.yml:
  - ASYNC_ENABLED (disabled in debug mode)
  - ASYNC_TIME (default 300s, omitted if async disabled)
  - ASYNC_POLL (0 for async fire-and-forget, 10 for sync mode)
- Replace hardcoded async/poll values with global vars in:
  - svc-db-openldap (03_users.yml, 04_update.yml)
  - web-app-mig (02_build_data.yml)
  - web-app-nextcloud (03_admin.yml, 04_system_config.yml, 05_plugin.yml,
    06_plugin_routines.yml, 07_plugin_enable_and_configure.yml)
- Guard changed_when and failed_when conditions to only evaluate in synchronous
  mode to avoid accessing undefined rc/stdout/stderr in async runs

  https://chatgpt.com/share/689cd8cc-7fbc-800f-bd06-a667561573bf
2025-08-13 20:26:40 +02:00
kevinveenbirkenbach 4fa1c6cfbd ansible: quote file modes; keycloak: robust LDAP bind update + config cleanup
Highlights
- Quote all file modes as strings ("0755"/"0770") across multiple roles to avoid YAML octal quirks and improve portability.
- Keycloak: introduce actions.{import_realm,update_ldap_bind} feature flags and wire them via vars/config.
- Implement idempotent LDAP bind updater (tasks/03_update-ldap-bind.yml):
  * kcadm login with no_log protection,
  * fetch LDAP UserStorage component by name,
  * compare current bindDn/bindCredential and update only when changed.
- Keycloak realm import template: keep providerId="ldap" and set name from keycloak_ldap_component_name.
- Centralize Keycloak readiness check in tasks/main.yml; remove duplicate waits from 02_update_client_redirects.yml and 04_ssh_public_key.yml.
- 01_import.yml: fix typo (keycloak), quote modes, tidy spacing, and replace Jinja-in-Jinja fileglob with concatenation.
- 02_update_client_redirects.yml: correct assert fail_msg filename; keep login-first flow.
- Minor template/vars tidy-ups (spacing, comments, consistent variable usage).

Files touched (excerpt)
- roles/*/*: replace 0755/0770 → "0755"/"0770"
- roles/web-app-keycloak/config/main.yml: add actions map
- roles/web-app-keycloak/vars/main.yml: unify Keycloak vars and feature flags
- roles/web-app-keycloak/tasks/{01_import,02_update_client_redirects,03_update-ldap-bind,04_ssh_public_key,main}.yml
- roles/web-app-keycloak/templates/{docker-compose.yml.j2,import/realm.json.j2}

https://chatgpt.com/share/689bda16-b138-800f-8258-e13f6d7d8239
2025-08-13 02:20:38 +02:00
kevinveenbirkenbach f31565e4c5 Optimized URLS 2025-08-13 00:33:47 +02:00
kevinveenbirkenbach f72ac30884 Replaced redirects by origine to raise performance 2025-08-11 19:44:14 +02:00
kevinveenbirkenbach 44e0fea0b2 Renamed cymais to infinito and did some other optimations and logout implementations 2025-07-29 16:35:42 +02:00
kevinveenbirkenbach f62355e490 Replaced nginx native with openresty for logout injection. Right now still buggy on nextcloud and espocrm 2025-07-24 03:19:16 +02:00
kevinveenbirkenbach 3bc64023af Added logout pages to some applications 2025-07-22 18:49:23 +02:00
kevinveenbirkenbach ff18c7cd73 Expect this to solve openldap import bug 2025-07-22 14:18:33 +02:00
kevinveenbirkenbach e78974b469 Solved openldap folder naming bug 2025-07-21 17:41:18 +02:00
kevinveenbirkenbach 1882fcfef5 Changed lam to web-app-lam 2025-07-20 09:59:31 +02:00
kevinveenbirkenbach fac8971982 Solved typo 2025-07-17 07:49:05 +02:00