Commit Graph

78 Commits

Author SHA1 Message Date
a526d1adc4 Solved Keycloak Master Email Configuration Update settings 2025-10-11 16:57:36 +02:00
ca95079111 Added Email Configuration for Keycloak Master Realm 2025-10-11 16:45:50 +02:00
e410d66cb4 Add health check for Keycloak container and grant global 'admin' realm role to permanent admin user
This update waits for the Keycloak container to become healthy before attempting login and replaces the old realm-management based role assignment with the global 'admin' realm role.
See: https://chatgpt.com/share/68e99953-e988-800f-8b82-9ffb14c11910
2025-10-11 01:40:48 +02:00
ab48cf522f fix(keycloak): make permanent admin creation idempotent and fix password command
- prevent task failure when 'User exists with same username'
- remove invalid '--temporary false' flag from set-password command
- ensure realm-admin role grant remains idempotent

See: https://chatgpt.com/share/68e99271-fdb0-800f-a8ad-11c15d02a670
2025-10-11 01:10:57 +02:00
41c12bdc12 Refactor Keycloak permanent admin creation:
- Remove jq dependency to avoid container command errors
- Use username-based operations instead of user ID lookups
- Improve idempotency and portability across environments

See: https://chatgpt.com/share/68e98e77-9b3c-800f-8393-71b0be22cb46
2025-10-11 00:54:04 +02:00
aae463b602 Optimized keycloak setup procedures 2025-10-11 00:20:27 +02:00
bb50551533 Restrucutred keycloak accounts 2025-10-10 23:40:58 +02:00
098099b41e Refactored web-app-keycloak 2025-10-10 22:45:26 +02:00
426ba32c11 feat(services): add CPU/RAM/PIDs defaults for heavy roles and align service names
Add per-service resource overrides (cpus, mem_reservation, mem_limit, pids_limit) for ollama, mariadb, postgres, confluence, gitlab, jira, keycloak, nextcloud; light formatting fixes in wordpress.

Rename service keys from generic 'application/web' to concrete names (jira, confluence, gitlab, keycloak) and update compose templates accordingly.

Jira: introduce JIRA_STORAGE_PATH and switch mounts/README accordingly.

https://chatgpt.com/share/68d2d96c-9bf4-800f-bbec-d4f2c0051c06
2025-09-23 21:43:50 +02:00
5d1210d651 feat(ai): introduce dedicated AI roles and wiring; clean up legacy AI stack
• Add svc-ai category under roles and load it in constructor stage

• Create new 'svc-ai-ollama' role (vars, tasks, compose, meta, README) and dedicated network

• Refactor former AI stack into separate app roles: web-app-flowise and web-app-openwebui

• Add web-app-minio role; adjust config (no central DB), meta (fa-database, run_after), compose networks include, volume key

• Provide user-focused READMEs for Flowise, OpenWebUI, MinIO, Ollama

• Networks: add subnets for web-app-openwebui, web-app-flowise, web-app-minio; rename web-app-ai → svc-ai-ollama

• Ports: rename ai_* keys to web-app-openwebui / web-app-flowise; keep minio_api/minio_console

• Add group_vars/all/17_ai.yml (OLLAMA_BASE_LOCAL_URL, OLLAMA_LOCAL_ENABLED)

• Replace hardcoded include paths with path_join in multiple roles (svc-db-postgres, sys-service, sys-stk-front-proxy, sys-stk-full-stateful, sys-svc-webserver, web-svc-cdn, web-app-keycloak)

• Remove obsolete web-app-ai templates/vars/env; split Flowise into its own role

• Minor config cleanups (CSP flags to {}, central_database=false)

https://chatgpt.com/share/68d15cb8-cf18-800f-b853-78962f751f81
2025-09-22 18:40:20 +02:00
5e616d3962 web: general domain cleanup (canonical/aliases normalization)
- Normalize domain blocks across apps:
  - Add explicit 'aliases: []' everywhere (no implicit aliases)
  - Standardize canonical subdomains for consistency:
    * Bluesky: web/api under *.bluesky.<PRIMARY_DOMAIN>
    * EspoCRM: espo.crm.<PRIMARY_DOMAIN>
    * Gitea:   tea.git.<PRIMARY_DOMAIN>
    * GitLab:  lab.git.<PRIMARY_DOMAIN>
    * Joomla:  joomla.cms.<PRIMARY_DOMAIN>
    * Magento: magento.shop.<PRIMARY_DOMAIN>
    * OpenProject: open.project.<PRIMARY_DOMAIN>
    * Pretix:  ticket.shop.<PRIMARY_DOMAIN>
    * Taiga:   kanban.project.<PRIMARY_DOMAIN>
  - Remove legacy/duplicate aliases and use empty list instead
  - Fix 'alias' -> 'aliases' where applicable

Context: preparing for AUTO_BUILD_ALIASES=False and deterministic redirect mapping.

Ref: conversation https://chatgpt.com/share/68cd512c-c878-800f-bdf2-81737adf7e0e
2025-09-19 14:51:56 +02:00
cbfb096cdb Refactor web health checker & domain expectations (filter-based)
- Move all domain→expected-status mapping to filter `web_health_expectations`.
- Require explicit app selection via non-empty `group_names`; only those apps are included.
- Add `www_enabled` flag (wired via `WWW_REDIRECT_ENABLED`) to generate/force www.* → 301.
- Support `redirect_maps` to include manual redirects (sources forced to 301), independent of app selection.
- Aliases always 301; canonicals use per-key override or `server.status_codes.default`, else [200,302,301].
- Remove legacy fallbacks (`server.status_codes.home` / `landingpage`).
- Wire filter output into systemd ExecStart script as JSON expectations.
- Normalize various templates to use `to_json` and minor spacing fixes.
- Update app configs (e.g., YOURLS default=301; Confluence default=302; Bluesky web=405; MediaWiki/Confluence canonical/aliases).
- Constructor now uses `WWW_REDIRECT_ENABLED` for domain generation.

Tests:
- Add comprehensive unit tests for filter: selection by group, keyed/default codes, aliases, www handling, redirect_maps, input sanitization.
- Add unit tests for the standalone checker script (JSON parsing, OK/mismatch counting, sanitization).

See conversation: https://chatgpt.com/share/68c2b93e-de58-800f-8c16-ea05755ba776
2025-09-11 13:58:16 +02:00
90843726de keycloak: update realm mail settings to use smtp_server.json.j2 (SPOT); merge via kc_merge_path; fix display name and SSL handling
See: https://chatgpt.com/share/68bb0b25-96bc-800f-8ff7-9ca8d7c7af11
2025-09-05 18:09:33 +02:00
7ca8b7c71d feat(nextcloud): integrate Talk & Whiteboard; refactor to NEXTCLOUD_* vars; full-stack setup
config(ports): add Nextcloud websocket port (4003); canonical domains (nextcloud/talk/whiteboard)

refactor: unify get_app_conf usage & Jinja spacing; migrate paths/handlers to new NEXTCLOUD_* vars

feat(plugins): split plugin routines; configure Whiteboard via occ (URL + JWT)

fix(oidc): use NEXTCLOUD_URL for logout; correct LDAP attribute mappings; add OIDC flavor switch

feat: Whiteboard container & reverse-proxy location; Talk STUN/WS ports; Redis URL for Whiteboard

chore: drop obsolete TODO; minor cleanups in oauth2-proxy, matrix, peertube, pgadmin, phpldapadmin, pixelfed, phpmyadmin

security(schema): Bluesky jwt_secret now base64_prefixed_32; add Nextcloud whiteboard_jwt_secret

db: normalize postgres image tag templating; central DB host checks spacing fixes

ops: add full-stack bootstrap (certs, proxy, volumes); internal nginx config reload handler update

refs: https://chatgpt.com/share/68b5f5b7-8d64-800f-b001-1241f818dc0e
2025-09-01 21:37:02 +02:00
231fd567b3 feat(frontend): rename inj roles to sys-front-*, add sys-svc-cdn, cache-busting lookup
Introduce sys-svc-cdn (cdn_paths/cdn_urls/cdn_dirs) and ensure CDN directories + latest symlink.

Rename sys-srv-web-inj-* → sys-front-inj-*; update includes/templates; serve shared/per-app CSS & JS via CDN.

Add lookup_plugins/local_mtime_qs.py for mtime-based cache busting; split CSS into default.css/bootstrap.css + optional per-app style.css.

CSP: use style-src-elem; drop unsafe-inline for styles. Services: fix SYS_SERVICE_ALL_ENABLED bool and controlled flush.

BREAKING CHANGE: role names changed; replace includes and references accordingly.

Conversation: https://chatgpt.com/share/68b55494-9ec4-800f-b559-44707029141d
2025-09-01 10:10:23 +02:00
2fccebbd1f Enforce uppercase README.md and TODO.md filenames
- Renamed all Readme.md → README.md
- Renamed all Todo.md → TODO.md
- Added integration test (tests/integration/test_filename_conventions.py) to automatically check naming convention.

Background:
Consistency in file naming (uppercase README.md and TODO.md) avoids issues with case-sensitive filesystems and ensures desktop cards (e.g. Pretix) are properly included.
Ref: https://chatgpt.com/share/68b1d135-c688-800f-9441-46a3cbfee175
2025-08-29 18:11:53 +02:00
6ea8301364 Refactor: migrate cmp/* and srv/* roles into sys-stk/* and sys-svc/* namespaces
- Removed obsolete 'cmp' category, introduced 'stk' category (fa-bars-staggered icon).
- Renamed roles:
  * cmp-db-docker → sys-stk-back-stateful
  * cmp-docker-oauth2 → sys-stk-back-stateless
  * srv-domain-provision → sys-stk-front
  * cmp-db-docker-proxy → sys-stk-full-stateful
  * cmp-docker-proxy → sys-stk-full-stateless
  * cmp-rdbms → sys-svc-rdbms
- Updated all include_role references, vars, templates and README.md files.
- Adjusted run_once comments and variable paths accordingly.
- Updated all web-app roles to use new sys-stk/* and sys-svc/* roles.

Conversation: https://chatgpt.com/share/68b0ba66-09f8-800f-86fc-76c47009d431
2025-08-28 22:23:09 +02:00
cb66fb2978 Refactor LDAP variable schema to use top-level constant LDAP and nested ALL-CAPS keys.
- Converted group_vars/all/13_ldap.yml from lower-case to ALL-CAPS nested keys.
- Updated all roles, tasks, templates, and filter_plugins to reference LDAP.* instead of ldap.*.
- Fixed Keycloak JSON templates to properly quote Jinja variables.
- Adjusted svc-db-openldap filter plugins and unit tests to handle new LDAP structure.
- Updated integration test to only check uniqueness of TOP-LEVEL ALL-CAPS constants, ignoring nested keys.

See: https://chatgpt.com/share/68b01017-efe0-800f-a508-7d7e2f1c8c8d
2025-08-28 10:15:48 +02:00
b9da6908ec keycloak(role): add realm support to generic updater
- Allow kc_object_kind='realm'
- Map endpoint to 'realms' and default lookup_field to 'id'
- Use realm-specific kcadm GET/UPDATE (no -r flag)
- Preserve immutables: id, realm
- Guard query-based ID resolution to non-realm objects

Context: fixing failure in 'Update REALM mail settings' task.
See: https://chatgpt.com/share/68affdb8-3d28-800f-8480-aa6a74000bf8
2025-08-28 08:57:29 +02:00
5aaf2d28dc Refactor path handling, service conditions and dependencies
- Fixed incorrect filter usage in docker-compose handler (proper use of | path_join).
- Improved LetsEncrypt template by joining paths with filenames instead of appending manually.
- Enhanced sys-svc-msmtp task with an additional condition to only run if no-reply mailu_token exists.
- Updated Keycloak meta to depend on Mailu (ensuring token generation before setup).
- Refactored Keycloak import path variables to use path_join consistently.
- Adjusted Mailu meta dependency to run after Matomo instead of Keycloak.

See: https://chatgpt.com/share/68af13e6-edc0-800f-b76a-a5f427837173
2025-08-27 16:19:57 +02:00
97f4045c68 Keycloak: align client attributes with realm dictionary
- Extended kc_force_attrs in tasks/main.yml to source 'publicClient',
  'serviceAccountsEnabled' and 'frontchannelLogout' directly from
  KEYCLOAK_DICTIONARY_REALM for consistency with import definitions.
- Updated default.json.j2 import template to set 'publicClient' to true.
- Public client mode is required so the frontend API of role 'web-app-desktop'
  can handle login/logout flows without client secret.

Ref: https://chatgpt.com/share/68ae0060-4fac-800f-9f02-22592a4087d3
2025-08-26 21:22:27 +02:00
c182ecf516 Refactor and cleanup OIDC, desktop, and web-app roles
- Improved OIDC variable definitions (12_oidc.yml)
- Added account/security/profile URLs
- Restructured web-app-desktop tasks and JS handling
- Introduced oidc.js and iframe.js with runtime loader
- Fixed nginx.conf, LDAP, and healthcheck templates spacing
- Improved Lua injection for CSP and snippets
- Fixed typos (WordPress, receive, etc.)
- Added silent-check-sso nginx location

Conversation: https://chatgpt.com/share/68ae0060-4fac-800f-9f02-22592a4087d3
2025-08-26 20:44:05 +02:00
eb781dbf8b fix(keycloak/ldap): make userObjectClasses JSON-safe and exclude posixAccount
- Render userObjectClasses via `tojson` (and trim) to avoid invalid control
  characters and ensure valid realm import parsing.
- Introduce KEYCLOAK_LDAP_USER_OBJECT_CLASSES in vars; exclude `posixAccount`
  for Keycloak’s LDAP config while keeping it for Ansible-managed UNIX users.
- Update UserStorageProvider template to use the new variable.

Rationale:
Keycloak must not require `posixAccount` on every LDAP user. We keep
`posixAccount` structural for Ansible provisioning, but filter it out for
Keycloak to prevent sync/import errors on entries without POSIX attributes.

Touched:
- roles/web-app-keycloak/templates/import/components/org.keycloak.storage.UserStorageProvider.json.j2
- roles/web-app-keycloak/vars/main.yml

Refs: conversation https://chatgpt.com/share/68aa1ef0-3658-800f-bdf4-5b57131d03b4
2025-08-23 22:05:26 +02:00
6016da6f1f Optimized bbb variables 2025-08-23 19:21:07 +02:00
9180182d5b Optimized variables 2025-08-21 16:27:10 +02:00
c27d16322b Optimized variables 2025-08-20 18:17:13 +02:00
384beae7c1 Added task to update default email settings 2025-08-20 16:41:53 +02:00
a10dd402b8 refactor: improve service handling and introduce MODE_ASSERT
- Improved get_service_name filter plugin (clearer suffix handling, consistent var names).
- Added MODE_ASSERT flag to optionally execute validation/assertion tasks.
- Fixed systemd unit handling: consistent use of %I instead of %i, correct escaping of instance names.
- Unified on_failure behavior and alarm composer scripts.
- Cleaned up redundant logging, handlers, and debug config.
- Strengthened sys-service template resolution with assert (only active when MODE_ASSERT).
- Simplified timer and suffix handling with get_service_name filter.
- Hardened sensitive tasks with no_log.
- Added conditional asserts across roles (Keycloak, DNS, Mailu, Discourse, etc.).

These changes improve consistency, safety, and validation across the automation stack.

Conversation: https://chatgpt.com/share/68a4ae28-483c-800f-b2f7-f64c7124c274
2025-08-19 19:02:52 +02:00
d3cc187c3b Made System Email Variables UPPER 2025-08-19 09:34:18 +02:00
29df95ed82 Optimized RBAC variables and async in keycloak 2025-08-18 00:15:41 +02:00
d1cd87c843 Fix RBAC groups handling and refactor Keycloak role
- Fixed incorrect handling of RBAC group configuration (moved from OIDC claims into dedicated RBAC variable set).
- Unified RBAC group usage across applications (LAM, pgAdmin, phpLDAPadmin, phpMyAdmin, YOURLS).
- Replaced old 'KEYCLOAK_OIDC_RBAC_SCOPE_NAME' with dedicated 'KEYCLOAK_RBAC_GROUP_*' variables.
- Updated OAuth2 Proxy configuration to use 'RBAC.GROUP.CLAIM'.
- Refactored Keycloak role task structure:
  * Renamed and reorganized task files for clarity ('_update.yml', '02_cleanup.yml', etc.).
  * Introduced meta and dependency handling separation.
- Cleaned up Keycloak config defaults and recaptcha placeholders.
2025-08-17 23:27:01 +02:00
5f0762e4f6 Finished implementation of oauth2 import 2025-08-17 21:59:58 +02:00
5642793f4a Added parameter to skipp dependency loading to speed up debugging 2025-08-17 21:44:15 +02:00
7d0502ebc5 feat(keycloak): implement SPOT with Realm
Replace 01_import.yml with 01_initialize.yml (KEYCLOAK_HOST_IMPORT_DIR)
Add generic 02_update.yml (kcadm updater for clients/components)
- Resolve ID → read current → merge (kc_merge_path optional)
- Preserve immutable fields; support kc_force_attrs
Update tasks/main.yml:
- Readiness via KEYCLOAK_MASTER_REALM_URL; kcadm login
- Merge LDAP component config from Realm when KEYCLOAK_LDAP_ENABLED
- Update client settings incl. frontchannel.logout.url
realm.json.j2: include ldap.json in UserStorageProvider
ldap.json.j2: use KEYCLOAK_LDAP_* vars for bindDn/credential/connectionUrl
vars/main.yml: add KEYCLOAK_* URLs/dirs and KEYCLOAK_DICTIONARY_REALM(_RAW)
docker-compose.yml.j2: mount KEYCLOAK_HOST_IMPORT_DIR
Cleanup: remove 02_update_client_redirects.yml, 03_update-ldap-bind.yml, 04_ssh_public_key.yml; drop obsolete config flag; formatting

Note: redirectUris/webOrigins ordering may still cause changed=true; consider sorting for stability in a follow-up.
2025-08-17 14:27:33 +02:00
20c8d46f54 Keycloak import templates cleanup
- Removed all static 'id' fields from realm.json.j2, ldap.json.j2, and client.json.j2
- Replaced 'desktop-secret' with correct 'client-secret' authenticator type
- Standardized Jinja filters to use 'to_json' consistently
- Corrected defaultClientScopes entry from 'web-app-origins' to built-in 'web-origins'
- Verified LDAP mapper definitions and optional realm role mapping
- Ensured realm.json.j2 contains only required scopes

References: Chat with ChatGPT (2025-08-17)
https://chatgpt.com/share/68a1aaae-1b04-800f-aa8d-8a0ef6d33cba
2025-08-17 12:11:14 +02:00
a524c52f89 Created own ldap.json.j2 for better readability in keycloak 2025-08-17 11:49:50 +02:00
5c9ca20e04 Optimized keycloak variables 2025-08-17 11:40:15 +02:00
bfe18dd83c Refactor Keycloak role:
- Replace KEYCLOAK_KCADM_PATH with KEYCLOAK_EXEC_KCADM consistently
- Externalize client.json to separate Jinja2 template and include it in realm.json
- Simplify LDAP bind update to use explicit KEYCLOAK_LDAP_* vars
- Add async/poll support for long-running kcadm updates
- Restructure vars/main.yml: clearer grouping (General, Docker, Server, Update, LDAP, API)
- Compute redirectUris/webOrigins centrally in vars
- Align post.logout.redirect.uris handling with playbook

Conversation: https://chatgpt.com/share/68a1a11f-f8ac-800f-bada-cdc99a4fa1bf
2025-08-17 11:30:33 +02:00
0a83f3159a Updated keycloak variables 2025-08-17 10:47:40 +02:00
fb7b3a3c8e Added setting of frontchannel.logout.url for keycloak 2025-08-17 10:38:25 +02:00
838a55ea94 Solved realm bug which appeared due to refactoring 2025-08-16 18:38:22 +02:00
0de26fa6c7 Solved bug existed due to difference between mailu domain and hostname difference. also refactored during this to find the bug 2025-08-16 14:29:07 +02:00
eadcb62f2a Added web-svc-logout as dependency for keycloak 2025-08-16 00:05:33 +02:00
3ac9bd9f90 Optimized variable typos 2025-08-15 18:43:42 +02:00
022800425d THE HUGE REFACTORING CALENDER WEEK 33; Optimized Matrix and during this updated variables, and implemented better reset and cleanup mode handling, also solved some initial setup bugs 2025-08-15 15:15:48 +02:00
0228014d34 Replaced .infinito.service and .infinito.timer by SOFTWARE_NAME suffix, optimized LICENSE link and update OIDC Realm and ID conf 2025-08-14 14:39:18 +02:00
4a65a254ae replaced port-ui-desktop with desktop to make it more speakable 2025-08-14 11:45:08 +02:00
db0e030900 Renamed general and mode constants and implemented a check to verify that constants are just defined ones over the whole repository 2025-08-13 19:11:14 +02:00
e497c001d6 keycloak: robust LDAP bind and connectionUrl update via kcadm (argv + JSON); strict ldap.*; idempotent
Switch to command:argv to avoid shell quoting and argument splitting issues.

Pass -s config values as JSON arrays via to_json, fixing previous errors: Cannot parse the JSON / failed at splitting arguments.

Also reconcile config.connectionUrl from ldap.server.uri.

Source desired values strictly from ldap.* (no computed defaults) and assert their presence.

Keep operation idempotent by reading current values and updating only on change.

Minor refactor: build reusable kcadm_argv_base and expand client state extraction.

Touch: roles/web-app-keycloak/tasks/03_update-ldap-bind.yml

https://chatgpt.com/share/689bea84-7188-800f-ba51-830a0735f24c
2025-08-13 03:30:14 +02:00
4fa1c6cfbd ansible: quote file modes; keycloak: robust LDAP bind update + config cleanup
Highlights
- Quote all file modes as strings ("0755"/"0770") across multiple roles to avoid YAML octal quirks and improve portability.
- Keycloak: introduce actions.{import_realm,update_ldap_bind} feature flags and wire them via vars/config.
- Implement idempotent LDAP bind updater (tasks/03_update-ldap-bind.yml):
  * kcadm login with no_log protection,
  * fetch LDAP UserStorage component by name,
  * compare current bindDn/bindCredential and update only when changed.
- Keycloak realm import template: keep providerId="ldap" and set name from keycloak_ldap_component_name.
- Centralize Keycloak readiness check in tasks/main.yml; remove duplicate waits from 02_update_client_redirects.yml and 04_ssh_public_key.yml.
- 01_import.yml: fix typo (keycloak), quote modes, tidy spacing, and replace Jinja-in-Jinja fileglob with concatenation.
- 02_update_client_redirects.yml: correct assert fail_msg filename; keep login-first flow.
- Minor template/vars tidy-ups (spacing, comments, consistent variable usage).

Files touched (excerpt)
- roles/*/*: replace 0755/0770 → "0755"/"0770"
- roles/web-app-keycloak/config/main.yml: add actions map
- roles/web-app-keycloak/vars/main.yml: unify Keycloak vars and feature flags
- roles/web-app-keycloak/tasks/{01_import,02_update_client_redirects,03_update-ldap-bind,04_ssh_public_key,main}.yml
- roles/web-app-keycloak/templates/{docker-compose.yml.j2,import/realm.json.j2}

https://chatgpt.com/share/689bda16-b138-800f-8258-e13f6d7d8239
2025-08-13 02:20:38 +02:00