tasks/00_core.yml built webserver_extra_configuration by concatenating the
helper iframes, the admin allowlist and the logout panel template lookups
with `~`. With the shipped empty `services.keycloak.admin.allowed_networks`
the allowlist template rendered no text, Ansible's template lookup returned
None for it, and `~` wrote the word "None" into the auth vhost. OpenResty
then refused `unknown directive "Nonelocation"` in front of the logout
panel location and every deploy with Keycloak went red.
The role now resolves the declared networks once in KEYCLOAK_ADMIN_NETWORKS
and the task appends the allowlist template only when that list is not
empty, the same guard the logout panel uses. The template renders the
restricted /admin location unconditionally from KEYCLOAK_ADMIN_NETWORKS.
With a list that did not contain the deploy host, the deploy still went
red: Keycloak answers `/` with a redirect to `/admin/`, so the CSP health
check and the front proxy probe, which request `/` from the host, got 403
and hlth-csp failed. While the allowlist is active, `location = /` now
redirects to the account console of the platform realm instead, and the
README no longer claims the deploy needs no entry for any reason.
test_admin_allowlist.py rendered the template with plain Jinja, which
yields '' for an empty template, so it could not see the None. It now
evaluates the task's own webserver_extra_configuration expression through
Ansible's Templar and real template lookup, which reproduces
"HELPER_IFRAMES\nNoneLOGOUT_PANEL" on the old task, and covers the root
redirect.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Keycloak vhost proxied every path to Keycloak, so the admin console
and the admin REST API under /admin/realms answered any client on the
internet (observed on auth.veen.world and auth.infinito.nexus), and
sys-svc-proxy had no way to limit a path by source address.
services.keycloak.admin.allowed_networks now takes the networks the
operator declares in the inventory. When the list is non-empty, the
vhost gets its own `location ^~ /admin`, rendered through the shared
html.conf.j2 like `/`, with one `allow` per entry and `deny all`. The
`^~` modifier keeps a regex location from taking an admin path over.
The OIDC endpoints under /realms/ and the assets under /resources/ stay
public. The role ships the list empty and adds no network of its own,
so an inventory that declares nothing renders today's vhost unchanged.
kc_admin_networks rejects anything but a list of addresses or CIDR
blocks without host bits, so a typo fails the render instead of the
nginx reload.
The deploy's own admin calls do not cross the vhost: every kcadm.sh
login, including web-app-opentalk's and web-app-mattermost's, runs
inside the Keycloak container against a loopback URL, and a unit test
now pins that. The README names the callers that do cross it (the
OpenTalk controller's user search, onion visitors, the Playwright
suites) and so need an entry. The TODO.md item is dropped.
Closes#1121
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In the async pass the role runs kcadm jobs side by side. Every kcadm call in
the Keycloak container shares one config file. kcadm rewrites that file when
it refreshes the access token, about every 55 seconds, takes the file lock
with tryLock() and gives up at once when another kcadm process holds it. One
of two overlapping declared-client jobs then failed with "Failed to get lock
on /opt/keycloak/.keycloak/kcadm.config", and the deploy with it.
keycloak_kcadm_update now repeats a call that lost the lock, up to ten times
one second apart. The call fails while it refreshes its token, before it
sends its own request, so repeating it is safe. Every other error still ends
the call at once.
That covers the module only, so the async jobs are reduced to the ones that
go through it:
- The default-scope assignment was an async shell task. It ran beside the
shell tasks that follow it, and neither side repeats a lost lock. It runs
in line now.
- The two mail-settings jobs were async without a register, so nothing
reaped them and a failure was never seen. They run in line now.
The client job and the declared-client jobs stay async. A test fails when an
async kcadm task does not use the module or is not registered for the reap.
A config file of its own per call was the other option. It does not work by
itself, because the admin session lives only in the shared file.
Closes#805
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The per-app mapper loop included three tasks that resolve the groups.path root
group id: a container exec, an assert and a set_fact. Their environment is
KC_CONTAINER, KC_REALM and KC_RBAC_ROOT_PATH, none of which mentions kc_app_id,
so every iteration re-fetched the same id.
Move them above the loop, right after the purge that clears orphan groups under
the root. The purge works below the root and leaves it standing, so the id
resolves against the same state the first iteration sees today.
per_app_root_id was read only by the assert and the set_fact, and both moved
with it. What the loop still reads is kc_root_group_id, now a host fact set
four lines before the include, and it feeds the same ROOT_GROUP_ID env as
before. One include site, one reader, both checked across the repo.
The assert stays and now runs once. It checks a loop-invariant value, so
repeating it per item proved nothing the single evaluation does not.
Worth 168 seconds per CI job.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Keycloak persists only attributes the declarative user profile declares, and unmanagedAttributePolicy defaults to DISABLED in 26.x. ensure_realm_user.sh:81 writes attributes.uidNumber and kcadm returns rc 0, but the value never lands, so the mattermost-id mapper (web-app-mattermost/tasks/02_keycloak.yml:118) has nothing to emit, userinfo answers 200 without an id claim, and the Mattermost GitLab provider rejects the callback with "Could not parse auth data out of gitlab user object ... user id can be 0". Run 33828309745 fails three Mattermost jobs on it, in compose and swarm alike, and all three Playwright specs are that one failure seen at three downstream points. b0ca8784e2 added the mirroring for exactly this failure and it could not take. LDAP realms are unaffected: the federation writes the attribute, and 08_users/realm.yml runs only when LDAP is off.
The profile push must now precede the seeding that depends on it, so it is synchronous. It measured 0.9 to 1.5s, and its reap sat after the seeding, so with ASYNC_ENABLED true the write would still have hit the old profile. Its register drops out of the reap list with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The OIDC admin wiring is gated on SSO alone, and SSO is 'web-app-keycloak' in group_names. Round 1 takes keycloak variant 1, which pins ldap off, so svc-db-openldap is not in the closure; the block still resolved its swarm service and the waiter spent 600 attempts on a service that does not exist (run 33698844864, Mattermost#1, 25 minutes). In compose the same block would have died on a missing container, so the defect is mode-independent.
The number it fetched is already in the registry: svc-db-openldap/tasks/04_users.yml:118 writes uidNumber from the same lookup('users'), and an earlier run shows inventory uid 1001 against ldapsearch 1001. Reading it directly removes the cross-stack exec entirely.
Keycloak's mattermost-id mapper reads the uidNumber attribute, which until now only the LDAP federation wrote, so the same round would have produced no id claim to match. ensure_realm_user.sh now mirrors the uid into that attribute whenever it is passed one, leaving callers without a uid unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The web-origins scope mapper listed every registered origin in each access token. openresty forwards that token to SSO-gated upstreams, so on a Tor deployment the claim reached 5457 of the token's 8650 characters and LAM's Apache refused the request with 400 (Moodle#0, run 33689701895).
The realm template carries the change, so a fresh import and every CI round get it directly. The reconcile task exists because 04_realm_import.yml only imports when the realm is missing, which would leave the old mapper in place on running installations.
Keycloak resolves CORS from client.webOrigins in DefaultCors.checkAllowedOrigins whenever the token carries no origins, so enforcement is unchanged; that fallback exists since 26.7 and the role pins 26.7.3. Introspection keeps the claim, and nothing in this repository reads it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The sync sent kcadm's rc into `|| true` and the task discarded what was
left with `failed_when: false`, so a sync that never ran reported
`changed` -- in CI run 32119014832 every sync in the outage window failed
server-side with 'Database operation failed' while the deploy stayed
green, and the first visible symptom was an unrelated group-id assert 40
seconds later.
Let the rc through and gate the task on it. Three attempts over ~220s
cover the 135s outage observed there; `ignore_errors` keeps a persistent
failure non-fatal like the sibling handler in handlers/main.yml, but it
now shows up as a failed task instead of a green one. The 60s timeout
bounds a call whose OS-level kill is SYS_TIMEOUT_CONTAINER_EXEC (7d) and
leaves 22x headroom over the slowest healthy sync measured across 111
sync tasks in both deploy modes (max 2.7s).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reset the cross-node recovery budget after a successful update pass so
later updates.yml invocations in the same deploy start with 5 fresh
attempts, and make resolve_root_group_id.sh exit non-zero when the
kcadm groups listing cannot be fetched instead of feeding a silent
empty string to the root-group assert (both surfaced by CI run
32119014832: swarm healthcheck kill mid-update).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
kcadm keeps its session inside the container, so a swarm task that gets rescheduled comes back unauthenticated. The cross-node rescue re-resolved the node but never logged in again, so all five recovery attempts died with "No server specified" -- the recovery path could not recover, by construction. Observed in CI run 32119014832 on FusionDirectory#0 and Fediwall#2 with an identical signature; a planned replacement survives today only because the realm import re-includes login.yml right after it.
Reuse utils/login.yml for the recovery leg: it already performs the same resolve and additionally re-authenticates and waits for health.
Drop the health=healthy filter on the login resolve. It contradicted the wait-for-healthy task right after it: with no healthy container the lookup returned an empty id, and inspecting an empty id can never report healthy, turning that wait into a 300s dead end whose empty-but-defined fact also defeats the default() in vars/main.yml. It also let login target a different container than the one updates.yml resolves unfiltered a moment later. aa6e156a6 relaxed the sibling resolver to status=running for the same reason.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A handler name was doing two jobs at once: the line an operator reads in the
play output, and the identifier a `notify:` binds to. The two pull in opposite
directions -- the log wants "restart the SSH service, with diagnostics", the
binding wants a token that survives quoting, YAML flow style and a shell
round-trip -- and a name that changes for readability silently changes which
handler runs.
Every handler that a notify reached by name now carries a `listen:` with that
exact wording, so the binding is explicit before anything is renamed. The
topics themselves become identifiers: `compose up` reads `compose-up`,
`Restart NFS ganesha` reads `restart-nfs-ganesha`, 22 of them across both
sides of every notify. Where the topic is assembled at runtime, both branches
moved together: the conditional in sys-service carries two literals, and the
LDAP renderer interpolates `import-{{ folder }}-ldif-files`, which is what
makes those handlers reachable at all.
Freed from the binding, the names take the emoji prefix every other task line
already has.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
08_users/realm.yml seeded the realm from `lookup('users') | non_reserved_users`
while 08_rbac_groups_realm.yml built group memberships from the raw
`lookup('users')`. The membership builder has its own guards, but both are dead
against real data: across all 269 default users, none carries a `reserved` key
and none has an empty password. So membership was effectively unfiltered while
seeding filtered on `accounts`.
helpdesk and no-reply (accounts: [mailbox], roles: [bot]) were therefore
demanded as members of roles/web-app-mailu/bot but never created, and
ensure_group_path_members.sh exits 1 on the first miss. Four jobs -- Mattermost 1,
Friendica 1, Matrix 1, OpenProject 1 -- each burned five cross-node recovery
attempts on a data failure no reschedule could fix.
Rather than adding the filter at the site that forgot it, remove the ability to
forget: lookup('users_non_reserved') resolves users and applies the predicate in
one place, and all four consumers now read it -- the two Keycloak user tasks, the
RBAC group builder, and OPENLDAP_USERS, which had it right all along. The filter
stays as the pure predicate the lookup calls.
Reproduced offline against the real user model, byte-identical to the CI member
list: before, bot demanded helpdesk and no-reply; after, members are exactly the
four seeded accounts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The repo's async convention is fire and forget: ASYNC_POLL is 0 whenever
ASYNC_ENABLED is true, and the swarm test lane forces it true through
extra vars. A task dispatched that way returns a stub, so a changed_when
or failed_when reading its own register evaluates against nothing and can
never report changed and never fail. Several tasks carried a
"not ASYNC_ENABLED and" prefix in front of those conditions, added to
avoid touching undefined rc and stdout, which turned the guard off
entirely rather than adapting it.
That is how the nextcloud plugin configuration stayed broken unseen: the
batch reported ok after 0.8s while its delegated commands kept running,
the next task read the half written configuration, and no failure in any
of them could surface.
Every async in the repo was decided individually rather than swept. Where
nothing consumes the result, the async stays and a one line rationale now
records what does not read it. Where a later task, a handler or a browser
does consume it, either the async goes or the result is collected.
Collecting is what the new reaper does. It waits for a whole batch, then
reports every broken job at once instead of aborting on the first, so one
run shows whether a single item or the whole set is broken. Deciding what
counts as broken lives in a filter plugin, because the distinction is
subtle enough to deserve tests: a module reports failure in msg, a command
in rc and stdout, an idempotent API answers "already exists" and means
success, and a job that cannot be read at all comes back marked finished
with no outcome, which would otherwise read as green.
Timing cost is roughly one to two minutes on a deploy measured in hours.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An untyped inspect searches every docker object kind, and a role's overlay network carries the same entity name as its main container, so the call can resolve the network and report a missing State key instead of naming the container. That is how the openresty assert died on every swarm job of run 30709070050.
Extend container-exec-resolver to cover inspect, stripping raw markers and quoted spans first so the flag walker survives Go template format strings. Add container-inspect-type, scoped to roles/ because the collision follows the entity naming convention and the outer act namespace carries no role networks. Sweep 40 call sites; suppress the delegate_to-resolved CIDs and the compose co-location probes with a stated reason.
Move the five container rules into tests/lint/ansible/shell/container/ and drop the container_ prefix from their filenames. Register both markers in the suppression docs, where the resolver marker was missing entirely.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replaces the raw package modules at 76 call sites with package_install, moves
each role's distribution mapping into its own meta/packages.yml, and turns the
inventory PACKAGES variable from source-keyed literal names into a list of
declared package ids. The lint that forbids the raw modules is enabled here,
because the migration burns it down from 76 offenders to zero.
Removes sys-aur, sys-aur-install and sys-package. The AUR build user, the
passwordless pacman rule and the inventory-driven install are part of the plan
the action plugin executes, so the roles no longer carry them.
Declaring the mappings surfaced breakage the raw modules had hidden: on Debian
the `which` binary comes from debianutils, jrnl is in no Debian or Ubuntu
suite, chromium-browser is absent from Debian stable, JDK 11 is unobtainable on
RedHat, wireguard had no RedHat branch at all, qbittorrent was built from the
AUR although Arch ships it in extra, and desk-docker installed a Docker variant
that conflicts with the one sys-svc-container installs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.
Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The install and realm-import waits accepted Shutdown as success, so a crashed job passed silently: akaunting went green without ever installing, and a keycloak import crash left synapse against a realm-less provider, surfacing as matrix Playwright OIDC timeouts. The state probe moves to a shared sys-svc-compose script where only Complete passes, Shutdown fails loudly with the task error, and Failed/Rejected count as pending because the jobs now restart on-failure with three bounded attempts, turning the observed import race from a coin flip into self-healing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Same class as the mattermost bump: the 60x5s override lost against slow first boots under CI load (keycloak gated on health=healthy); 120x5s stays well under the resolve util's 600x2s default and still fails hard after the budget.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.
Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Resolve every 3+-file same-prefix group flagged by the new prefix lint
by moving the files into a subfolder named after the shared prefix and
dropping the prefix from the file names, updating all references:
- utils/env/handlers/infinito_*.py -> infinito/ (dir/, package_cache/
nested a second level); registry imports and the four handler-dir
globs switched to iter_project_files
- utils/install/lint/ansible_*.py -> ansible/ (aliased imports)
- roles/sys-svc-compose/files/swarm_*.{py,sh} -> files/swarm/
- web-app-erpnext/web-app-zammad files/scripts/apply_*.{py,rb} -> apply/
- web-app-odoo files/sql/{select,update}-* and sql/oidc/{grant,insert,
select,update}-* -> matching subfolders
- web-app-keycloak tasks/update/05_ldap*.yml -> 05_ldap/ (internal
includes made role_path-absolute)
- plugins/filter/get_*.py -> plugins/filter/get/ (filter names are the
filters() dict keys, so Jinja usage is unaffected; ansible discovers
the one-level-deep subdir)
Ansible lookup dirs, addon dirs and .github/workflows keep their flat
layout (the filename is the public id / GitHub only discovers flat).
Unit tests mirror the new source layout.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace scripts/system/diagnostics/rescue.sh with utils/diagnostics/
container.py: one workflow call on failure captures the current level
(containers with logs+inspect, swarm services, journal, host resources,
in-play role dumps) and recurses itself through every DiD boundary via
self-copy, from the runner down to the innermost runtime (RESCUE_MAX_DEPTH).
compose and swarm now share an identical failure step; the swarm per-node
collector wrapper is gone. All 17 script-only Ansible rescue: blocks are
deleted and their bare block: wrappers inlined (plus a wordpress leftover);
the three justified blocks (postgres pg_hba, xwiki extensions, keycloak
cross-node recovery) keep their evidence tasks and now end in a fail task.
Two lints enforce the state: rescue: is forbidden without a
'# nocheck: rescue-<reason>' first-task marker, and a block without
rescue/always/when or another scoping key must be inlined.
sys-service-terminator hoists become to the include apply. Also move
routine/08_cleanup.sh to utils/clean/teardown.sh and topology/ to
utils/topology/ so routine/ holds only the sequenced flow.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every Ansible rescue: block used to carry ad-hoc per-role failure dumps that
drifted apart and captured different subsets. Replace them all with one
diagnostics script (scripts/system/diagnostics/rescue.sh) that snapshots
containers (full logs plus inspect per container), swarm services (ps plus
logs per service), journal, host resources, runtime info and stats into a
per-run folder and always exits 1 so the deploy still fails. A new lint
enforces the shape: every rescue ends with the script, at most one
nocheck-justified extra task before it (postgres pg_hba dump, xwiki
extension-repo probe), and a nocheck rescue-recovery exemption for the one
block that must genuinely recover: keycloak object updates, where a
cross-node reschedule needs a swarm-wide node re-resolve plus a delegate_to
change that only a rescue can express; its recovery is kept and now also
captures diagnostics before the terminal failure.
The snapshot folder path is a group_vars SPOT: DIR_RESCUE_DIAGNOSTICS and
DIR_VAR_LIB live in group_vars/all/05_paths.yml as plain strings; new env
handlers derive INFINITO_RESCUE_DIAGNOSTICS_DIR and INFINITO_DIR_VAR_LIB
from it for the generated .env, replacing the default.env entries, and the
play-level environment forwards the value to every task host so rescue.sh
needs no second default. CI uploads the folder like the inventory and
playwright artifacts: compose copies it out of the deploy container in
dedicated.sh, swarm captures per node (manager and both workers) into
node-named subfolders.
Deleting the ad-hoc dumps orphaned helper files (postgres inspect_container,
terminator docker_diagnostics, peertube diagnostics, matomo bootstrap debug,
xwiki and openresty rescue includes) and 14 role vars; all removed. Generic
pieces of those dumps (runtime version and info, stats, dmesg oom evidence,
swapon) moved into the SPOT script instead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two stacked defects broke keycloak object updates on the swarm-delegated
cross-node path:
- get_merged_users cached the templar-less raw_users payload (fetched by
get_merged_applications) under the same key as templar-backed lookups, so
every later lookup('users') in the process returned UNRENDERED values. The
literal '{{ 42 | strong_password }}' / '{{ ansible_become_password }}' then
reached kcadm and swarmkit, failing the realm password policy and the stack
env validation. Key the cache on templar presence so rendered and raw
payloads never share an entry.
- The cross-node recovery block re-included itself with a self-referential
'kc_update_attempt' template var, which Ansible 2.19 expands recursively
('Recursive loop detected: maximum recursion depth exceeded') on the second
attempt. Advance the counter with an eager set_fact so each level reads a
concrete int.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
resolve_host_cid/wait_converged/guard_instance_dir took a run_host parameter every caller had to pass; STACK_HOST already encodes the correct host in both modes (first swarm manager, degrading to inventory_hostname in compose), so the manager-only queries now delegate to it directly and the parameter plus all caller declarations are gone. Also repairs the swarm guard include: apply/delegate_to are invalid on a HandlerTaskInclude and the previous commit carried duplicate run_once/delegate_to keys, which failed the ansible syntax-check. The keycloak handler drops its masking default; the 02b conditional collapses into the SPOT.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Without LDAP the fallback only seeded the administrator user and no RBAC groups, so oauth2-proxy's allowed_groups check 403'd every group-gated app (prometheus/seaweedfs, matomo/css downstream) and non-admin logins hit user_not_found (biber). 08_users_realm now seeds every non-reserved user with a usable password (parity with the LDAP path, which federates the same set); new 08b_rbac_groups_realm materialises the /roles/<app>/<role> group tree + memberships directly in the realm: build_realm_rbac_groups mirrors build_ldap_role_entries and restricts members to the users 08 actually seeds (by username attribute); ensure_group_path_members.sh walks the path via kcadm (children endpoint, exact user lookup, idempotent membership PUT) and hard-fails on any missed group or membership instead of masking. Both routines are gated to KEYCLOAK_LDAP_ENABLED false; LDAP-enabled deploys are untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Swarm: KC_CACHE=ispn + KC_CACHE_STACK=jdbc-ping so replicas share an Infinispan session/realm cache via JDBC_PING2 over the existing Postgres (no multicast, built-in + auto-table in 26.6.3). Compose: KC_CACHE=local (single node). Realm-import scale-back uses KEYCLOAK_REPLICAS/_WANT constants; DB network attached only when the DB is shared (KEYCLOAK_DB_NETWORK_ARG). Does NOT by itself fix the admin-console static-chunk 404 (that needs a shared gzip cache / sticky routing).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert the two resolve_host_cid.yml includes in 02b_updates.yml from the
[playbook_dir, ...] | path_join form to lookup('path_absolute', ...), matching
the repo-wide sweep.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce plugins/lookup/path_absolute.py as the single source for the
repo-root anchor: lookup('path_absolute', 'roles/...') returns an absolute
path under playbook_dir, joining every term. Replace all hand-built
playbook_dir path constructions across .yml/.yaml and .j2 templates with it:
[playbook_dir, '<rel>'] | path_join -> lookup('path_absolute', '<rel>')
playbook_dir ~ '/<rel>' -> lookup('path_absolute', '<rel>')
{{ playbook_dir }}/<rel> -> lookup('path_absolute', '<rel>')
Unrelated repo_path symbols in cli/meta/roles/applications/resolution and the
single bare discover_playwright_roles(playbook_dir) call are left untouched.
Add tests/lint/ansible/test_playbook_dir_via_lookup.py: fails the build if any
of the three construction forms reappears in .yml/.yaml/.j2, pointing callers
at the lookup. Unit test covers the plugin.
Touching these files pulled their pre-existing narration comments into
test_comments_valid scope; clean them (delete restatement/banners, keep real
trip-wires as Workaround/Upstream/Invariant markers). Drop caution, important
and pitfall from the comment-lint marker vocabulary and remove the comments
that used them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In swarm the object updates run delegated to KEYCLOAK_HOST_NODE, but that fact
was resolved early in the login step. Over a long deploy swarm can reschedule
keycloak to another node, leaving the update delegated to a node that no longer
runs keycloak, so kcadm fails with "Failed to create Keycloak object". The
cross-node rescue recovered, but with any_errors_fatal the rescued failure still
aborted the play (exit 2) after a slow recovery loop.
Re-resolve the live hosting node at the start of the update block and run the
update routines with a fresh delegate_to binding, so the first attempt targets
the node currently running keycloak. The rescue stays as a fallback for a
reschedule during the update itself.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.
- emit emoji names at the source: cli/build/include generator now writes
"🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The realm-import scales keycloak 0->N (fresh reschedule across nodes), then the kcadm update block runs delegated to a fixed KEYCLOAK_HOST_NODE; the in-module recovery resolves only a node-local container id, so when a replica reschedules to another node it can never be found -> 'No such container: <dead cid>'. Wrap the update routines (extracted to 02b_update_routines.yml) in a rescue that re-resolves the hosting node + live id and re-delegates the updates, bounded to 5 attempts; compose re-raises immediately. Relax the recovery resolver to status=running so it can find the just-rescheduled, not-yet-healthy replica.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
02a_manager_ops_on_host.yml set 'delegate_to: KEYCLOAK_HOST_NODE' directly on 'include_tasks: 02b_updates.yml', which Ansible rejects ("'delegate_to' is not a valid attribute for a TaskInclude"), so the playbook failed to parse (exit 4) for every SSO-consuming app, since keycloak is preloaded for them. The delegation is already supplied by the caller 02_manager_ops.yml, which includes 02a via apply: { delegate_to: KEYCLOAK_HOST_NODE }, so the include-level delegate_to was both redundant and invalid. Remove it; the sibling include 05_realm_import.yml already relies on the caller's apply, which is why it had no delegate_to.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 'Resolve hosting node + container id' block was gated 'when: KEYCLOAK_HOST_NODE is not defined or none'. A block-level when propagates to the included resolve_host_cid tasks and is re-evaluated per task; resolve_host_cid sets <prefix>_HOST_NODE in its first task, so once KEYCLOAK_HOST_NODE was set the later container-id resolution and the KEYCLOAK_LOCAL_CID set_fact self-skipped, leaving KEYCLOAK_LOCAL_CID undefined. The 'Wait for keycloak container to be healthy' shell then failed finalizing _raw_params with 'KEYCLOAK_LOCAL_CID is undefined' across all 16 SSO-dependent swarm apps. Flatten the block and drop the self-mutating when so resolve_host_cid always runs and sets both KEYCLOAK_HOST_NODE and KEYCLOAK_LOCAL_CID; this also keeps both facts fresh after the realm-import scale reschedules keycloak.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the ten Keycloak update routines (client, mail, RBAC, LDAP, reCAPTCHA, userprofile, users) out of 02a_manager_ops_on_host.yml into 02b_updates.yml so 02a stays within the block-size limit; 02a now includes 02b. Delegation is preserved by the caller 02_manager_ops, which includes 02a with apply: delegate_to KEYCLOAK_HOST_NODE; that propagates to the nested update tasks. The include conditions (LDAP shared/disabled, reCAPTCHA, emailVerified, non-LDAP user provisioning) are moved verbatim.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the per-role swarm host-node/container-id resolve duplicated across fider, matomo, decidim, keycloak (login + realm-import), collabora and listmonk with two shared task utils under roles/sys-svc-compose/tasks/utils/swarm/. resolve_host_cid sets <prefix>_HOST_NODE / <prefix>_LOCAL_CID (swarm: converge + node + cid delegated to the node; compose: inventory host + given cid); wait_converged is the bare convergence gate.
Teach test_variable_definitions to read templated set_fact keys like {{ prefix }}_HOST_NODE as a *_HOST_NODE name pattern (guarded so a pure {{ x }} key cannot become a catch-all), and add the playbook_dir-relative include load-failure exemptions to .ansible-lint-ignore.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The compose-flavour realm-import file runs `compose stop/run/up` and chdirs into
directories.instance, but it is only ever included via the DEPLOYMENT_MODE-driven
`05_realm_import/{{ DEPLOYMENT_MODE }}.yml` switch, so those compose-only calls
never reach a swarm host. Add the
`# include-gated: when: DEPLOYMENT_MODE == "compose"` file-header marker so the
compose-verb / compose-chdir lints treat the file as exempt; it was missing since
the 05_realm_import flavour split.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
DEPLOYMENT_MODE is binary (swarm / compose), so a negated `!= 'swarm'` gate
always means `== 'compose'`. Replace every negated gate across roles/ and
group_vars with the positive form for readability, and add a lint
(tests/lint/ansible/variables/test_no_deployment_mode_negation.py) that forbids
`DEPLOYMENT_MODE !=` in .yml/.yaml/.j2 with a `# nocheck:
deployment-mode-negation` escape.
The compose-verb / chdir / jinja-gate lint guidance messages now suggest the
positive form too; the gate matcher (utils/annotations/task_gate.py) and its
test inputs keep recognising the legacy `!= 'swarm'` so a nocheck-marked
exception is still detected as compose-only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
