Commit Graph
14682 Commits
Author SHA1 Message Date
evangelostsak 1500470c9d feat(web-app-openbao): add idempotent identity group-alias helper 2026-09-07 00:31:51 +02:00
evangelostsak ad98e188ef feat(web-app-openbao): mount the optional internal PKI behind a service flag 2026-09-07 00:31:31 +02:00
evangelostsak 8c68430c0c feat(web-app-openbao): configure OIDC and LDAP auth with group-to-policy binding 2026-09-07 00:31:17 +02:00
evangelostsak 47df1e8a11 feat(web-app-openbao): mount KV v2 and reconcile the RBAC policies 2026-09-07 00:30:59 +02:00
evangelostsak 50adf4456d feat(web-app-openbao): pin AppRole ids from inventory and revoke the root token 2026-09-07 00:30:45 +02:00
evangelostsak 9c7979f0ec feat(web-app-openbao): initialise once and wait for the static seal to auto-unseal 2026-09-07 00:30:30 +02:00
evangelostsak f2862c2868 feat(web-app-openbao): render config after the backend stack, before the handler flush 2026-09-07 00:30:06 +02:00
evangelostsak dfa2350d6f feat(web-app-openbao): add run-once role entry point 2026-09-07 00:29:52 +02:00
evangelostsak 1019e17b71 feat(web-app-openbao): resolve OIDC, LDAP and seal values from the config SPOT 2026-09-07 00:29:37 +02:00
evangelostsak 3c29f8bb61 feat(web-app-openbao): add sso, ldap and pki variant matrix 2026-09-07 00:27:51 +02:00
evangelostsak 970ae41b48 feat(web-app-openbao): add descriptive role metadata 2026-09-07 00:27:05 +02:00
evangelostsak af59d181ee feat(web-app-openbao): add role-local user map 2026-09-07 00:26:47 +02:00
evangelostsak ad097bf598 feat(web-app-openbao): add seal key and AppRole credential schema 2026-09-07 00:25:03 +02:00
evangelostsak 299dd1955c feat(web-app-openbao): declare administrator, operator and reader RBAC roles 2026-09-07 00:24:48 +02:00
evangelostsak e82e9e1326 fix(web-app-openbao): accept the 307 redirect in the webserver health check 2026-09-07 00:24:32 +02:00
evangelostsak 1f9fc80437 feat(web-app-openbao): add CSP flags for the Ember UI 2026-09-07 00:24:16 +02:00
evangelostsak 1963fbd6d0 feat(web-app-openbao): allocate role-local subnet 2026-09-07 00:23:45 +02:00
evangelostsak f172f59741 feat(web-app-openbao): add canonical domain 2026-09-07 00:23:31 +02:00
evangelostsak fd8fdea14f feat(web-app-openbao): declare raft, logs and policy volumes 2026-09-07 00:23:16 +02:00
evangelostsak bec1de3692 feat(web-app-openbao): declare service entity, image pin and deploy modes 2026-09-07 00:23:01 +02:00
evangelostsak ddabb3c9db feat(web-app-openbao): add galaxy metadata 2026-09-07 00:22:45 +02:00
evangelostsak 60859ca65f docs(requirements): add 032 OpenBao integration requirement 2026-09-07 00:20:17 +02:00
Alejandro Roman 9e05e04573 fix(stalwart): CI issues related to Stalwart +temporal test to cehck issues locally not reproduceble 2026-09-06 13:58:01 +04:00
infinito-nexus-updater[bot] 6b6fd8c451 update(images): update Docker image versions (test-e2e-playwright) 2026-09-05 03:08:34 +00:00
infinito-nexus-updater[bot] 4eac7c0812 update(images): update Docker image versions (svc-db-elasticsearch) 2026-09-05 03:08:30 +00:00
kevinveenbirkenbachandClaude Opus 5 8b9733b425 fix(mattermost): let the DM page load once the team join has landed
The biber DM spec opened /main/messages/@administrator as its first navigation after SSO, so a single page load both joined the open team and opened the DM. The Agents plugin fetches team-scoped data during that load and errors with "Unable to find an existing account matching your username for this team", after which the typed message stays in the composer and Enter sends nothing.

The spec was flaky rather than broken: the same job recorded it both green and red, and the DM channel still held a message from an earlier successful attempt, so the URL does join the team. Reload it once, so the plugin initialises against a membership that already exists.

Evidence from the run artifacts: the trace records two pageError entries from the mattermost-ai bundle during that navigation, and the failure snapshot shows the message still sitting in the textbox. The causal step from the plugin error to the lost keypress is inferred, not proven, so a single green run will not confirm this fix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 23:50:20 +02:00
Alejandro Roman 42f267407a fix(stalwart): fix fediwall, mailu and nexcloud issues in CI, all green now 2026-09-04 23:02:01 +04:00
Alejandro Roman d3c3caa54e Merge remote-tracking branch 'upstream/main' into feature/role-stalwart-mail-swarm 2026-09-04 22:54:11 +04:00
kevinveenbirkenbachandClaude Opus 5 e0048f157b fix(litellm): gate consumers on a served model, not on the gateway being deployed
The gateway publishes only models it has a backend for and renders model_list: [] when neither ollama nor lmstudio is deployed and no OpenRouter key is set. LITELLM_CHAT_MODEL fell back to openrouter/auto without checking that key, so consumers were configured for a model the gateway does not carry and every prompt failed with a 400.

Observed in the Mattermost#1 CI job: litellm.log carries 12 occurrences of "Invalid model name passed in model=openrouter/auto" next to 400 Bad Request, and tests/test-litellm-agent.js failed on every attempt while the rest of the suite passed.

Make the fallback conditional on the key so the name is empty when nothing is served, promote LITELLM_CHAT_MODEL_SERVED from the role vars to group_vars where consumers can see it (it was the third copy of this logic and the only correct one), and switch the 14 consumers from "litellm is deployed" to "litellm serves a model". The Playwright service flag reads the same lookup, so the spec skips together with the Ansible gate. The bool filter carries weight: the folded scalar yields the string False, which Jinja treats as truthy.

A new unit test renders both expressions against Jinja and pins the invariant that a model name exists exactly when a model is served, including the case where ollama preloads embedding models only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 19:48:32 +02:00
kevinveenbirkenbachandClaude Opus 5 b41c61556e fix(svc-db-mariadb): stop the root password probe from waiting on a settled answer
The probe exists only to decide whether 02_reset_root_password.yml runs, but it carried until: rc == 0 with retries: 30 and delay: 5. A wrong password is a settled answer, so the loop waited its full budget and then took the reset branch anyway. It waited longest in exactly the case it exists to detect.

Measured at 169.8s in one Nextcloud job against 0.6s upstream, which matches 30 retries at 5s plus 31 attempts. Exit on Access denied as well, the way the gitlab and homeassistant token probes already exit on REJECTED. A server that is still starting reports a connection failure rather than Access denied, so the budget for genuine unreadiness is untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 18:56:21 +02:00
kevinveenbirkenbachandClaude Opus 5 e9dbcd90ae fix(front-proxy): probe the vhost being deployed, not the primary domain
lookup(tls, <application_id>) resolves through resolve_primary_domain_from_app, so every iteration of a multi-vhost loop probed the same primary domain. For seaweedfs that meant checking s3. three times and filer. and master. never, right after writing their vhosts.

Passing the domain instead returns it unchanged (utils/tls_common.py:221-227). Resolve the probe URL and the accepted status codes from that one term in a second set_fact, so the URL and the codes cannot disagree even when the lookup falls back to the primary domain.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 17:36:03 +02:00
kevinveenbirkenbachandClaude Opus 5 c88e50ef17 fix(front-proxy): accept the status codes an app declares, not just the triple
The front proxy hardcoded [200, 301, 302] and escalated to a full OpenResty redeploy on anything else. Apps that answer 403, 404 or 405 by design tripped it on every run: 19 escalations in one Nextcloud job at a measured median of 40.5s each.

roles/<app>/meta/server.yml already declares status_codes per canonical key, and sys-ctl-hlth-webserver already reads it. Move the resolution into utils/roles/applications/status_codes.py so both consumers share one definition of DEFAULT_OK, and expose it to templates as the accepted_status_codes filter.

The proxy gate takes the union of the default and the declaration rather than replacing it. Most declarations are narrower than the triple, so replacing would escalate vhosts that answer perfectly well. Contract checking stays with sys-ctl-hlth-webserver, which keeps replacement semantics.

_normalize_codes and _valid_http_code became dead in the health filter and are removed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 17:30:34 +02:00
kevinveenbirkenbachandClaude Opus 5 bbfd82f893 fix(profiling): let the task recap reach past the twenty slowest
ansible.cfg enables profile_tasks, whose output_limit defaults to 20. On a deploy round of this size the recap therefore cut off at 66.07s and printed nothing below, so a role whose cost is spread over many cheap tasks could not be attributed at all: in run 33828309745 that left 22.7 of sys-svc-compose 51.9 minutes unexplained, and the same blind spot covers web-app-nextcloud and web-app-keycloak almost entirely.

The callback accepts all, which disables the cap, but the recap prints one line per task execution and pass 1 alone ran over 7500 of them. Since the Actions worker dies silently on ENOSPC while writing its own logs, which is why utils/tests/swarm/matrix.py aborts a step at DISK_FLOOR_MB, the limit stays bounded at 300: about 600 extra lines across both passes.

Measured with a 30-task play: 30 recap entries with this config, 20 with the upstream default forced through PROFILE_TASKS_TASK_OUTPUT_LIMIT. The lint pins the setting so removing the section cannot silently restore the truncation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 16:45:44 +02:00
kevinveenbirkenbachandClaude Opus 5 988d7ef11c style: apply make autoformat
isort moved unittest ahead of pathlib and dropped the blank line between them; ruff wrapped the assertTrue message, the re.compile concatenation and the join generator at the 88 column boundary. No regex, condition or assertion changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 16:35:14 +02:00
kevinveenbirkenbachandClaude Opus 5 75fe7ccafb fix(swarm-matrix): make every phase report what it cost
The step banner announced a start and nothing else, so a phase duration had to be read off the next banner, and a phase with no output of its own was invisible. In the budget-killed Nextcloud#0 of run 33828309745 the credential rotation spent 17 minutes with nothing but the 30-second sampler between its banner and the next one, and reconstructing the split between provisioning, both passes, the DR drill and the rotation meant subtracting timestamps by hand.

The watch loop moves unchanged into _run_watched. _run now brackets it and prints the elapsed time and the exit code, so this holds for every phase, not only the rotation.

The test stubs _run_watched and pins the three things that matter: the exit code passes through, the opening banner is printed, and the closing line carries duration and rc. It is the first test over this module, whose absence was named earlier as the reason a wrong change to the pass sequence would turn nothing red.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 16:31:17 +02:00
kevinveenbirkenbachandClaude Opus 5 942d6eb110 fix(front-proxy): say which status forced the OpenResty redeploy
Both escalation tasks decide on a registered probe that carries failed_when: false, so the status never reaches the log and every probe prints ok:. Measured in run 33828309745 job 100890535287: the site-down branch fired 4 times and the post-reload fallback 15 times across 30 domains, which is 19 of the jobs 27 full redeploys at roughly 38s each. Nothing in the artifact says what any of them answered.

Ansible renders a task name before it evaluates when:, so carrying the status in the name reports it for all 30 domains while the skipping or changed line below says which one escalated. That costs no extra task.

The names stay under the 80 characters test_name_emoji_and_length allows by dropping the domain, which the neighbouring verify task already names.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 16:30:06 +02:00
kevinveenbirkenbachandClaude Opus 5 f5c0b2e572 fix(nextcloud): stop doubling the slash on the onlyoffice document-server URL
plugins/lookup/tls.py:102 builds url.base as "{protocol}://{domain}/", pinned by test_tls.py:59. meta/addons/onlyoffice.yml appended another one, so DocumentServerInternalUrl was stored as https://host//.

The connector treats its two URLs differently: getDocumentServerUrl normalises with rtrim, getDocumentServerInternalUrl returns the stored string raw and concatenates the endpoint onto it. Every server-side call therefore went to //healthcheck and //converter. A swarm vhost forwards $request_uri verbatim, so the document server answered 404; a compose vhost uses proxy_pass with a URI part and nginx collapses the double slash. The 404 makes the connector cron write settings_error, and FilesListener then stops adding the onlyoffice script to the Files view, which is why iframe#onlyofficeFrame never appears and the addon spec times out. Observed in run 33828309745 job 100890535389: "GET //healthcheck HTTP/2.0" 404 in the openresty log, and the docx row offering Download as its default action.

The lint generalises it. This class has been fixed one site at a time at least five times before (efee415b21, 035c1efbf3, b0c7c9601e, a96bdbba38, 83636408c9), so the rule now holds repo-wide: a variable carrying an unstripped tls url.base may never be followed by a slash. Definitions that strip it again are excluded, which is what keeps KEYCLOAK_URL and OPENTALK_KEYCLOAK_BASE_URL out of the finding set; measured, no name is defined both ways.

Not fixed here: settings_error is sticky, so an instance already poisoned stays broken after a redeploy. The obvious remedy, gating the deploy on occ onlyoffice:documentserver --check, is a four-step round trip whose steps never succeeded in either artifact, so it needs its own design.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 16:07:43 +02:00
kevinveenbirkenbachandClaude Opus 5 f1c3dd47b5 fix(tests): read the jellyfin auth script through the cache
test_no_uncached_file_reads flagged SCRIPT.read_text() in the extraction test. The suite shares one decoded read per file through utils.cache.files, and a bare pathlib read opts out of that.

The full gate surfaced it; the targeted run that accompanied a8469e064 executed the test in isolation and could not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 14:10:06 +02:00
kevinveenbirkenbachandClaude Opus 5 a57a42b4f3 style: apply make autoformat
ruff wrapped the set comprehension in the jellyfin auth extraction test at the 88 column boundary. No assertion changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 14:06:36 +02:00
kevinveenbirkenbachandClaude Opus 5 b2dca7cf65 fix(keycloak): declare uidNumber, or the realm silently drops it
Keycloak persists only attributes the declarative user profile declares, and unmanagedAttributePolicy defaults to DISABLED in 26.x. ensure_realm_user.sh:81 writes attributes.uidNumber and kcadm returns rc 0, but the value never lands, so the mattermost-id mapper (web-app-mattermost/tasks/02_keycloak.yml:118) has nothing to emit, userinfo answers 200 without an id claim, and the Mattermost GitLab provider rejects the callback with "Could not parse auth data out of gitlab user object ... user id can be 0". Run 33828309745 fails three Mattermost jobs on it, in compose and swarm alike, and all three Playwright specs are that one failure seen at three downstream points. b0ca8784e2 added the mirroring for exactly this failure and it could not take. LDAP realms are unaffected: the federation writes the attribute, and 08_users/realm.yml runs only when LDAP is off.

The profile push must now precede the seeding that depends on it, so it is synchronous. It measured 0.9 to 1.5s, and its reap sat after the seeding, so with ASYNC_ENABLED true the write would still have hit the old profile. Its register drops out of the reap list with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 14:04:51 +02:00
kevinveenbirkenbachandClaude Opus 5 a8469e0642 fix(jellyfin): authenticate once, so the token survives to the password change
configure-auth.sh pins DeviceId="infinito-deploy" in CLIENT_HDR, and realign_admin_password authenticated twice with it: once for the access token, once for the user id. Jellyfin replaces the session bound to a device id when a new one is created, so the first token was already revoked when POST /Users/{id}/Password presented it. Run 33828309745 job 100890535194 shows the result as fifteen times "curl: (22) The requested URL returned error: 401", with the deploy otherwise complete (ok=2830 failed=1). get_user_id arrived with 1bf28bb2bf and carried the second call in from the start.

One AuthenticateByName response now yields both fields, read through token_of and user_id_of so neither expression is spelled twice.

Redeeming the reset pin clears the stored password: in that job forty authentications with the deployed password were denied, and the next one, with the pin string as the password, succeeded. Every exit after the redemption therefore states that the administrator is currently without one, and the two silent returns around ForgotPassword now log why they gave up.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 13:54:31 +02:00
kevinveenbirkenbachandClaude Opus 5 c750247e33 fix(joomla,yourls,penpot): exec on the node that holds the container
container exec is node-local: sys-svc-container/files/python/container.py dispatches docker exec with no swarm routing. These roles passed *_CONTAINER_ADDRESS, which names the service cluster-wide, and ran from the stack host, so a call landed on the manager and answered "No such container" whenever the task sat on a worker.

Each branch now resolves the hosting node and its node-local id once through sys-svc-compose/tasks/utils/swarm/resolve_host_cid.yml, the way web-app-baserow already does, and every exec uses *_LOCAL_CID with delegate_to on *_HOST_NODE. Compose is untouched: resolve_host_cid sets *_HOST_NODE to inventory_hostname there, so the delegation is to self and resolve_compose_cid passes the old address through.

penpot resolves only when native login is enabled, because the resolver waits up to 600x2s for convergence and both of its exec tasks are already gated on that flag.

The *_LOCAL_CID names come from a runtime set_fact, which the resolver lint cannot approve, so each scanned call site carries the nocheck the migrated mastodon sites already use. Measured with the lint parser itself: 19 exec lines carry a target, 19 would be flagged, 19 are suppressed. No nocheck reaches a rendered command or a task name, and the one comment that does land inside a bash body passes bash -n.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 13:47:35 +02:00
Alejandro Roman ee6c555f88 fix(stalwart): fix mailu issues, add more playwrithg with full cehcks, fix lint issues 2026-09-04 15:45:16 +04:00
kevinveenbirkenbachandClaude Opus 5 c2d737d609 fix(hlth-volumes): probe the NFS port the server actually serves
The reachability check called showmount, which speaks the MOUNT protocol. The ganesha export sets NFS_CORE_PARAM Protocols = 4 and Enable_NLM = false, and every EXPORT block repeats Protocols = 4, so MOUNT is never registered and showmount cannot succeed against a healthy server. The repo already records this elsewhere: scripts/tests/deploy/swarm/utils/collect/diagnostics.sh:125 prints "showmount -e failed; NFSv4-only servers do not answer MOUNT".

Every failure fed status, and a nonzero exit reaches OnFailure, which starts sys-ctl-cln-anon-volumes (dockreap --no-confirmation). So on any host holding both anonymous and NFS volumes the check has been failing on every run and reaping volumes for a reason that was never true. Probing tcp/2049 is what the role does for its own readiness gate in files/nfs-ganesha-restart.conf:9, and it is correct for the kernel flavor too.

The two tests drive a whitelisted anonymous volume so the anonymous section contributes nothing and the exit status reflects only the NFS check. One asserts a listening port passes, the other that a refused one fails; the first is the regression guard, the second keeps the check from being one that always fails.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 13:43:37 +02:00
Alejandro Roman 1351c7ae96 fix(stalwart): fix mailu issues, add more playwrithg with full cehcks, fix some SPOT 2026-09-04 14:57:08 +04:00
kevinveenbirkenbachandClaude Opus 5 74afec37c3 fix(health): exit with a status, not a failure count
Four probes returned the number of failures as the process exit code. An exit status is 8 bits wide, so exactly 256 or 512 failures arrive as 0 and the probe reports success. sys-service installs these as Type=oneshot units (roles/sys-service/vars/main.yml:28) and sys-service-terminator starts them with until: result is not failed, so that exit code is the whole health verdict.

The count is not lost: each of the four prints it to stdout a line earlier. Only the exit status stops carrying it, because it cannot.

This makes no CI job green and is not meant to. For any count other than 256 or 512 nothing changes; at those two it turns a false success into an honest failure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 04:41:04 +02:00
kevinveenbirkenbachandClaude Opus 5 f1e86eef58 fix(ci-trigger): retire a green suite on any input
The retrigger turned a suite off only when the source run had been
dispatched with that suite explicitly on. b1ca39e9dd introduced that
guard together with the feature itself, and it contradicts the same
commit's message and docstring, both of which state that the job record
is the only evidence.

The guard made the feature a no-op in its most common case. An input
left on its default renders no title segment and resolves to empty, so
config.get('workspace') was never 'true' for a run that reached the
suite on the 'auto' default. call-orchestrator.yml reads 'auto' as
global scope off the whitelist alone, and a --failed retrigger puts its
selection on the priority line while leaving the whitelist empty, so
every retrigger in the chain redeployed a suite that had already gone
green. Measured on run 33785198875: six workspace jobs green, and the
dispatch still went out carrying workspace=auto.

Drop the guard. suite_passed already refuses to retire anything that
failed, was cancelled, timed out, was skipped or never ran at all, so
the job record decides on its own.

Three tests cover the paths 8600f1d9aa left open, all of which pinned
the explicit 'true': green on 'auto', green with no recorded input, and
red on 'auto' staying on.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 04:06:04 +02:00
kevinveenbirkenbach 974f593146 Merge branch 'main' into feature/ai-agent-employees 2026-09-04 03:46:42 +02:00
kevinveenbirkenbachandClaude Opus 5 bf20562cec style: apply make autoformat
Line-length reflow in three test modules and a stray blank line in zammad variants. In router.yml the formatter also moved the nsswitch lineinfile regex from single to double quotes with the backslashes doubled to match; single-quoted YAML already passes a backslash through literally, so the string reaching lineinfile is byte-identical and the rule still changes only NOTFOUND.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 03:40:17 +02:00
kevinveenbirkenbachandClaude Opus 5 7b4ea2eaf2 test(svc-net-tor): follow the resolv task into the block that now holds it
The resolv.conf assertion looked its task up with a flat scan over router.yml top level. Encapsulating the two TOR_DNSMASQ_OWNS_LISTENER tasks in a block moved that task one level down, so the generator ran dry and StopIteration failed the test. A recursive walk over block/rescue/always restores the lookup while leaving what the test asserts untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 03:39:06 +02:00