Keycloak persists only attributes the declarative user profile declares, and unmanagedAttributePolicy defaults to DISABLED in 26.x. ensure_realm_user.sh:81 writes attributes.uidNumber and kcadm returns rc 0, but the value never lands, so the mattermost-id mapper (web-app-mattermost/tasks/02_keycloak.yml:118) has nothing to emit, userinfo answers 200 without an id claim, and the Mattermost GitLab provider rejects the callback with "Could not parse auth data out of gitlab user object ... user id can be 0". Run 33828309745 fails three Mattermost jobs on it, in compose and swarm alike, and all three Playwright specs are that one failure seen at three downstream points. b0ca8784e2 added the mirroring for exactly this failure and it could not take. LDAP realms are unaffected: the federation writes the attribute, and 08_users/realm.yml runs only when LDAP is off.
The profile push must now precede the seeding that depends on it, so it is synchronous. It measured 0.9 to 1.5s, and its reap sat after the seeding, so with ASYNC_ENABLED true the write would still have hit the old profile. Its register drops out of the reap list with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A membership written in LAM crosses two systems before any consumer sees it:
Keycloak reads it from LDAP through this federation provider, then the consumer
syncs its groups claim on the next login. With cachePolicy DEFAULT the first hop
serves a cached federated user, so a grant is invisible to Keycloak -- and
therefore to every OIDC consumer -- until the entry expires. The user
re-authenticates and still does not hold the role they were just given.
NO_CACHE makes every lookup hit LDAP, so the membership lands at the next
authentication. This is the other half of 54aced186: that change had to make its
spec poll rather than read once precisely because of this staleness.
The cost is explicit: this removes a cache from the authentication path, so every
login and group check now performs an LDAP round trip. On this platform
svc-db-openldap sits on the container network and the round trip is cheap, but on
a deployment with a slow or remote directory it would be felt at every sign-in.
Choosing NO_CACHE over a bounded EVICT_DAILY or MAX_LIFESPAN is a judgement, not
a measurement: a grant that silently fails to apply is worse than a slower login.
The better design is an explicit cache invalidation when a grant is written,
which would keep both freshness and caching; it is not built here.
Not verified: no deploy has run with this, and the login-latency cost was not
measured before or after. I also did not confirm this template is the only place
a cache policy reaches the federation provider -- a second component or a runtime
override could still cache.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The healthy replica threw `StorageUnavailableException: LDAP connection has been
closed` while the other two were down for an unrelated reason. slapd sets no
idletimeout, so it never closes a connection itself; what cuts them is the swarm
path, where an idle flow through a service VIP loses its conntrack entry without
either side seeing a FIN. The pool then hands the dead connection to the next
caller.
Capping the wait was the obvious remedy and is not available: reading the class
files of the pinned keycloak-ldap-federation jar, the provider knows
connectionPooling, connectionTimeout and readTimeout, and passes only
com.sun.jndi.ldap.connect.pool and .pool.protocol to JNDI. There is no
connectionPoolingTimeout - setting one would have parked a dead key in the
component config. Disabling the pool removes the failure class instead of
shortening the wait on a connection that is already gone; with importEnabled the
directory is not on a hot path, so the extra bind per operation costs little.
The flag lives in meta/services.yml and is read without a fallback, so an
inventory can put the pool back and a missing key fails the deploy rather than
resolving to a guess.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Resolve every 3+-file same-prefix group flagged by the new prefix lint
by moving the files into a subfolder named after the shared prefix and
dropping the prefix from the file names, updating all references:
- utils/env/handlers/infinito_*.py -> infinito/ (dir/, package_cache/
nested a second level); registry imports and the four handler-dir
globs switched to iter_project_files
- utils/install/lint/ansible_*.py -> ansible/ (aliased imports)
- roles/sys-svc-compose/files/swarm_*.{py,sh} -> files/swarm/
- web-app-erpnext/web-app-zammad files/scripts/apply_*.{py,rb} -> apply/
- web-app-odoo files/sql/{select,update}-* and sql/oidc/{grant,insert,
select,update}-* -> matching subfolders
- web-app-keycloak tasks/update/05_ldap*.yml -> 05_ldap/ (internal
includes made role_path-absolute)
- plugins/filter/get_*.py -> plugins/filter/get/ (filter names are the
filters() dict keys, so Jinja usage is unaffected; ansible discovers
the one-level-deep subdir)
Ansible lookup dirs, addon dirs and .github/workflows keep their flat
layout (the filename is the public id / GitHub only discovers flat).
Unit tests mirror the new source layout.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Removes 67 live references to docs/requirements/<NNN>-*.md files across
43 contributor docs, role READMEs, code comments, task files, and tests.
Requirement docs are initial-creation-only artefacts; live references
freeze a snapshot that rots once the per-feature SPOT (a contributor
doc, a code module, a lint test) takes over the live behaviour.
In the same pass, fixes documentation.md violations on the touched
files: drops em-dashes used as thought breaks, removes "SPOT for ..."
self-labels, strips historical phrasing ("the legacy", "previously",
"used to", "is no longer used", "matching the historical"), and
reflows soft-wrapped sentences so each sentence stays on one logical
line.
The `test_inventory_vars_file_spot.py` whitelist for the literal
inventory path drops the docs/requirements/003 entry and instead
skips docs/requirements/ entirely; future requirement docs that
mention the literal path are then ignored automatically.
Verified by tests/lint/repository/test_no_stale_requirement_refs.py
(green tree-wide).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements requirements 005, 006, and 007.
005: per-application Keycloak `group-ldap-mapper`s anchor each app's
role groups under `/roles/<application_id>/<role_name>` (and, for
tenant-aware apps, `/roles/<application_id>/<tenant>/<role_name>`).
The flat-CN LDAP layout keeps each CN globally unique so Keycloak's
group-tree resolver never collides; per-app mappers use the LDAP
`description` as the visible Keycloak group name to keep the path
free of redundant prefixes. Migration purges legacy pre-005 entries
and forces a Keycloak `logout-all` so stale tokens drop pre-005
group paths.
006: per-service `<SERVICE>_SERVICE_ENABLED` env flags and the
`service-gating.js` helper let role specs `skipUnlessServiceEnabled`
cleanly when a shared service is missing or `SERVICES_DISABLED` is
set; `MODE_CI` markers (`INFINITO_MAKE_DEPLOY`, `GITHUB_ACTIONS`,
`ACT`) propagate through the deploy CLI into the test stage.
007: WordPress->Discourse round-trip Playwright scenario gated on the
`web-app-discourse` deployment.
WordPress Playwright runs 5/5 green on the single-site flow; the
multisite + Discourse scenarios are correctly skipped when their
service prerequisites are absent. The `rbac_group_path` lookup is the
single sanctioned producer for OIDC group paths and is wired into
every consumer (yourls, lam, minio, openwebui, pgadmin, phpldapadmin,
phpmyadmin, prometheus, xwiki).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add configurable LDAP sync periods to the Keycloak role and wire them into
the LDAP UserStorageProvider component.
This enables automatic changed-user sync every 15 minutes and a full sync
once per day via:
docker.services.ldap.sync.period.changes
docker.services.ldap.sync.period.full
Defaults can now be centrally managed via application config.
https://chatgpt.com/share/696d02b4-1418-800f-8fe5-864f4e7494e8
- Add LDAP.RBAC.EMPTY_MEMBER_DN placeholder for empty role groups (schema-compliant groupOfNames)
- Ensure RBAC role entries always contain at least one member
- Align users OU attribute with DN (ou=users)
- Switch Keycloak LDAP group/role mappers to DN-based membership
- Fix missing newline in Keycloak schema file
https://chatgpt.com/share/696b7226-7c18-800f-a0b0-59068bb69d9f
Add end-to-end support for reserved usernames and tighten CAPTCHA / Keycloak logic.
Changes:
- Makefile: rename EXTRA_USERS → RESERVED_USERNAMES and pass it as --reserved-usernames to the users defaults generator.
- cli/build/defaults/users.py: propagate flag into generated users, add --reserved-usernames CLI option and mark listed accounts as reserved.
- Add reserved_users filter plugin with and helpers for Ansible templates and tasks.
- Add unit tests for reserved_users filters and the new reserved-usernames behaviour in the users defaults generator.
- group_vars/all/00_general.yml: harden RECAPTCHA_ENABLED / HCAPTCHA_ENABLED checks with default('') and explicit > 0 length checks.
- svc-db-openldap: introduce OPENLDAP_PROVISION_* flags, add OPENLDAP_PROVISION_RESERVED and OPERNLDAP_USERS to optionally exclude reserved users from provisioning.
- svc-db-openldap templates/tasks: switch role/group LDIF and user import loops to use OPERNLDAP_USERS instead of the full users dict.
- networks: assign dedicated subnet for web-app-roulette-wheel.
- web-app-keycloak vars: compute KEYCLOAK_RESERVED_USERNAMES_LIST and KEYCLOAK_RESERVED_USERNAMES_REGEX from users | reserved_usernames.
- web-app-keycloak user profile template: inject reserved-username regex into username validation pattern and improve error message, fix SSH public key attribute usage and add component name field.
- web-app-keycloak update/_update.yml: strip subComponents from component payloads before update and disable async/poll for easier debugging.
- web-app-keycloak tasks/main.yml: guard cleanup include with MODE_CLEANUP and keep reCAPTCHA update behind KEYCLOAK_RECAPTCHA_ENABLED.
- user/users defaults: mark system/service accounts (root, daemon, mail, admin, webmaster, etc.) as reserved so they cannot be chosen as login names.
- svc-prx-openresty vars: simplify OPENRESTY_CONTAINER lookup by dropping unused default parameter.
- sys-ctl-rpr-btrfs-balancer: simplify main.yml by removing the extra block wrapper.
- sys-daemon handlers: quote handler name for consistency.
Context: change set discussed and refined in ChatGPT on 2025-11-29 (Infinito.Nexus reserved usernames & Keycloak user profile flow). See conversation: https://chatgpt.com/share/692b21f5-5d98-800f-8e15-1ded49deddc9
- Move all domain→expected-status mapping to filter `web_health_expectations`.
- Require explicit app selection via non-empty `group_names`; only those apps are included.
- Add `www_enabled` flag (wired via `WWW_REDIRECT_ENABLED`) to generate/force www.* → 301.
- Support `redirect_maps` to include manual redirects (sources forced to 301), independent of app selection.
- Aliases always 301; canonicals use per-key override or `server.status_codes.default`, else [200,302,301].
- Remove legacy fallbacks (`server.status_codes.home` / `landingpage`).
- Wire filter output into systemd ExecStart script as JSON expectations.
- Normalize various templates to use `to_json` and minor spacing fixes.
- Update app configs (e.g., YOURLS default=301; Confluence default=302; Bluesky web=405; MediaWiki/Confluence canonical/aliases).
- Constructor now uses `WWW_REDIRECT_ENABLED` for domain generation.
Tests:
- Add comprehensive unit tests for filter: selection by group, keyed/default codes, aliases, www handling, redirect_maps, input sanitization.
- Add unit tests for the standalone checker script (JSON parsing, OK/mismatch counting, sanitization).
See conversation: https://chatgpt.com/share/68c2b93e-de58-800f-8c16-ea05755ba776
- Converted group_vars/all/13_ldap.yml from lower-case to ALL-CAPS nested keys.
- Updated all roles, tasks, templates, and filter_plugins to reference LDAP.* instead of ldap.*.
- Fixed Keycloak JSON templates to properly quote Jinja variables.
- Adjusted svc-db-openldap filter plugins and unit tests to handle new LDAP structure.
- Updated integration test to only check uniqueness of TOP-LEVEL ALL-CAPS constants, ignoring nested keys.
See: https://chatgpt.com/share/68b01017-efe0-800f-a508-7d7e2f1c8c8d
- Render userObjectClasses via `tojson` (and trim) to avoid invalid control
characters and ensure valid realm import parsing.
- Introduce KEYCLOAK_LDAP_USER_OBJECT_CLASSES in vars; exclude `posixAccount`
for Keycloak’s LDAP config while keeping it for Ansible-managed UNIX users.
- Update UserStorageProvider template to use the new variable.
Rationale:
Keycloak must not require `posixAccount` on every LDAP user. We keep
`posixAccount` structural for Ansible provisioning, but filter it out for
Keycloak to prevent sync/import errors on entries without POSIX attributes.
Touched:
- roles/web-app-keycloak/templates/import/components/org.keycloak.storage.UserStorageProvider.json.j2
- roles/web-app-keycloak/vars/main.yml
Refs: conversation https://chatgpt.com/share/68aa1ef0-3658-800f-bdf4-5b57131d03b4
- Fixed incorrect handling of RBAC group configuration (moved from OIDC claims into dedicated RBAC variable set).
- Unified RBAC group usage across applications (LAM, pgAdmin, phpLDAPadmin, phpMyAdmin, YOURLS).
- Replaced old 'KEYCLOAK_OIDC_RBAC_SCOPE_NAME' with dedicated 'KEYCLOAK_RBAC_GROUP_*' variables.
- Updated OAuth2 Proxy configuration to use 'RBAC.GROUP.CLAIM'.
- Refactored Keycloak role task structure:
* Renamed and reorganized task files for clarity ('_update.yml', '02_cleanup.yml', etc.).
* Introduced meta and dependency handling separation.
- Cleaned up Keycloak config defaults and recaptcha placeholders.
