Commit Graph
23 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 54aced1868 fix(openwebui): stop the first MCP grant from locking a user out
ENABLE_OAUTH_ROLE_MANAGEMENT does two things and only one is obvious. Besides
mapping the administrator group onto the Open WebUI admin role, it turns
OAUTH_ALLOWED_ROLES into a sign-in gate: a user whose groups claim is non-empty
and matches neither the allowed list nor the admin list is refused outright.

Upstream defaults that list to the literals `user,admin`, and Keycloak emits
claims shaped /roles/<app>/<role>, which can never match them. A user with no
groups therefore signed in fine, and the moment they received their first `mcp`
grant their claim became non-empty, matched nothing, and they were refused. The
grant meant to give access was what removed it.

The env sets OAUTH_ALLOWED_ROLES=*. The grant itself stays the real gate, since
BYPASS_ADMIN_ACCESS_CONTROL=false keeps it binding on administrators too. Stated
plainly, this does widen a control: every authenticated platform user now passes
Open WebUI's role gate. A list matching the actual role paths would be stricter
and is not built here.

The LAM grant spec is rewritten to drive the real path rather than assert around
it. Three behaviours it now handles: LAM renders per-value add and clear controls
as anchors with an empty alt, so they cannot be found by accessible name and are
located structurally; a membership written in LAM crosses Keycloak federation and
then an Open WebUI login sync before it is observable, so the check polls instead
of reading once; and the sign-out step clears localStorage rather than only
cookies, because Open WebUI keeps its session token there and a cookie-only reset
left the old session live while the assertion passed.

Not verified: openwebui was not in the last deploy's role set, so neither the env
change nor the rewritten spec has run. The spec is also now genuinely
re-authenticating, which is more correct and will surface auth flakiness the
previous version masked.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 14:20:25 +02:00
kevinveenbirkenbachandClaude Opus 5 d7a0f19bea fix(web-app-openwebui): let the gateway's models be chatted with
Keeping the admin bypass off makes check_model_access run for every user,
and a model discovered through a connection has no row in the model table,
so the lookup raised and the chat answered 400 Model not found while the
same model was still listed. The flag restores the upstream default and is
read nowhere near the MCP tool-server gating, which stays on the admin
bypass alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 14:44:24 +02:00
kevinveenbirkenbach 857b1b0964 feat(web-app-openwebui): register the discovered MCP servers and scope them
Four defects in one chain kept the tool servers from ever being served.

The connections never arrived. The provisioner read them back from the config
API, but ENABLE_PERSISTENT_CONFIG=false means the environment feeds the running
process and is never written to the table that API serves, so a fresh
deployment answered with an empty list however long the deploy waited. It now
falls back to the declared list it was handed.

The groups stayed empty. Group resolution returned only the id and dropped the
group itself, so membership could not be written. It returns both, and the
member list is reconciled explicitly, because the OIDC claim cannot revoke on
its own: Open WebUI drops a stale membership only while the claim is non-empty,
so a user who loses their last group keeps it until some later login carries
another.

Success was not checkable. The script printed a count and exited zero even when
nothing was granted. It now refuses to report success when servers are declared
and none was granted, and emits the resolved group ids, which are persisted and
rendered back into the environment. Without that second render a bare container
restart drops the grants, since the environment is authoritative on every start
and the ids did not exist when it was first written.

The signing key is pinned because Open WebUI otherwise generates it into the
process environment only, where a container exec sees a different value than
the server.
2026-08-06 21:09:57 +02:00
kevinveenbirkenbachandClaude Opus 5 e355bbdb8b feat(mcp): gate Open WebUI tool servers on the serving role's mcp group
Eight server roles declare an `mcp` RBAC role, so the platform mints
roles/<app>/mcp and the administrator joins it. Open WebUI now honours those
groups instead of flattening them to user/admin, and each MCP tool server is
restricted to the group of the role that serves it.

The grant cannot be rendered: Open WebUI matches it against a group id that
insert_new_group mints as a fresh UUID, so tasks/01_mcp.yml resolves the group
after the container is up and writes the grant through the admin API, the way
every instance-minted credential in this repository is provisioned.

Entries render disabled. An empty access_grants is not 'nobody', Open WebUI
reads it as every administrator, and ENABLE_PERSISTENT_CONFIG=false makes the
env authoritative again on every start. Enabling only together with the grant
keeps the fallback at no access rather than too much.

The provisioning account is the role-internal openwebui-api-bot, never a
platform identity: OAUTH_MERGE_ACCOUNTS_BY_EMAIL ships disabled and an OIDC
login whose email a local account already holds is refused, so seeding a
person's address would lock them out.

Known limits are documented rather than claimed away: a bare restart leaves MCP
off until the next deploy, a user's roles list carries no application context so
roles: [mcp] grants every deployed mcp group at once, and Open WebUI drops a
stale membership only while the groups claim is non-empty.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 22:06:47 +02:00
kevinveenbirkenbachandClaude Opus 5 ab444915be feat(openwebui,flowise): wire the web UIs to the discovered MCP servers
Register every discovered server as an Open WebUI tool server, skipping
the auth schemes its ToolServerConnection cannot present so none is
stored with an unusable credential. Serialise that value with compact
separators: a space makes dotenv_quote wrap it, swarm keeps the quotes
inside the value, and Open WebUI then parses a string instead of a list
and crash-loops on startup.

Prepare the Flowise instance instead of registering servers in it.
Flowise 3.1.3 exposes no instance-level MCP registry; MCP lives in a
per-flow Custom MCP node. The role pins the transport to URL-based
servers so no flow can spawn a local command, and relaxes the host deny
list far enough to reach a container origin while loopback and the
cloud-metadata addresses stay denied. Selecting a server inside a flow
stays an operator step, and the README says so.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 22:27:46 +02:00
kevinveenbirkenbachandClaude Opus 5 92c45e2471 refactor(openwebui): serve models through the gateway, not the model server
Open WebUI talked to Ollama directly. It now speaks the OpenAI-compatible surface of svc-ai-litellm with a per-consumer virtual key, so a second backend or an external provider becomes a gateway config change instead of a role change. A lint keeps future roles off the model servers.

The gateway advertised a hardcoded model list (ollama/llama3, ollama/nomic-embed-text) while the matrix variant preloads smollm2:135m, so it offered only models the backend did not have and hid the one it did. The list is derived from the ollama preload SPOT now.

The chat spec followed the rename: it gates on litellm instead of the removed ollama service flag, and picks the first non-embedding model from /api/models rather than pinning a variant-specific name, so it exercises the chain rather than a fixture.

Routing openwebui through the gateway pulls both model backends into the closure of every role that consumes it, which pushed nextcloud variant 2 over the memory budget. The budget stays at 32/64 GB; instead the matrix variants of svc-ai-ollama and svc-ai-lmstudio now declare the size their test model needs (1g/2g) while the base keeps the 6g/8g that llama3 and mistral require. The budget guard only saw half of that: it replaced the evaluated role with its variant but counted every dependency from meta/services.yml, a combination no matrix round deploys. It now evaluates every variant-carrying role at the same round index, falling back to variant 0, the way the deploy planner does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 15:49:59 +02:00
kevinveenbirkenbachandClaude Opus 4.8 15e7647d2f refactor(env): central TZ+LANG base included by every role env.j2
New roles/sys-svc-container/templates/env.j2 sets TZ={{ HOST_TIMEZONE }} and LANG=C.UTF-8. Every role env.j2 includes it at the top; it renders via stack_host_template, the same engine that resolves the includes in compose.yml.j2, so it works in both compose and swarm. TZ/LANG are removed from the individual role env.j2 so each value stays single-source.

New lint tests/lint/ansible/test_env_base_include.py verifies every role env.j2 includes the base and never re-sets a base-owned key (keys derived from the base file).

mattermost and gitea meta/volumes.yml drop the /etc/timezone host-path bind: swarm hard-rejects a task whose bind source is missing on the node (Debian-trixie nodes have no /etc/timezone). TZ now comes from the base env. /etc/localtime bind also dropped; the container timezone follows TZ.

Non-directive comments were removed from the touched env.j2 (banners, restatements, reference URLs, narration) so the changed files pass the comment-lint; only tool directives remain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 22:13:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 06aa25c095 feat(ai): integrate OpenWebUI/Ollama as a real integration_openai backend + chat test
- svc-ai-ollama variant-0: preload smollm2:135m (smallest instruct model for CI)
- integration_openai meta: Service URL -> {base}/api for the OpenWebUI backend
- web-app-openwebui: enable_api_keys config flag (services.yml + vars) rendered into
  ENABLE_API_KEYS so sk- API keys authenticate against the OpenAI-compatible API
- integration_openai spec: drop the DOM-gated skip, assert the partner backend host
  (openwebui chat.ai / flowise flow.ai), distinct from Nextcloud
- openwebui: OLLAMA_SERVICE_ENABLED playwright flag + test-ollama-chat.js asserting
  the preloaded model answers a chat completion over OpenWebUI -> Ollama

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 06:03:15 +02:00
kevinveenbirkenbachandClaude Opus 4.8 153bfb1898 feat(objstore): consume the object store across 14 web-app roles
Switch penpot, nextcloud, gitea, mattermost, openwebui, funkwhale, baserow, openproject, pixelfed, mastodon, bookwyrm, listmonk, peertube and opentalk to the engine-agnostic objstore lookup: the minio service entries are pinned off, a seaweedfs service (gated on the provider being in group_names, public for the media apps) replaces them, and the matrix variants pin the new flag. OpenTalk keeps object storage mandatory through an embedded fallback and provisions its bucket via weed shell.

Add objstore lookup SPOT keys consumed by these roles: endpoint (host:port), public_domain, public_bucket (public S3 host plus bucket) and public_url (scheme plus public_bucket, scheme derived from the provider TLS state). This removes the repeated lookup('config', lookup('objstore', ...), 'server.domains.canonical.api') / host:port concatenations from the consumer templates. Listmonk settings render their JSON values via the to_json filter.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 11:30:33 +02:00
kevinveenbirkenbachandClaude Opus 4.7 f9ef4bbbd9 feat(web-app-openwebui): native/LDAP/OIDC persona tests + role-mapping & first-run UX fixes
- Adds per-persona scenarios: native admin signup+sign-in, admin/biber LDAP, admin/biber OIDC, guest.
- env.j2: DEFAULT_USER_ROLE=user pulled out of OIDC block so native/LDAP users skip the pending gate.
- env.j2: LDAP_SEARCH_FILTER → (objectClass=*) standalone clause; OpenWebUI builds the uid= predicate itself.
- env.j2: OAUTH_ADMIN_ROLES prefixed with `/` to match Keycloak's full-path group claim.
- playwright.env.j2: expose ADMIN_EMAIL for the email-keyed native signup helper.
- _shared.js: API-driven ensureNativeAdminExists, idempotent on 400 "email taken" and 403 "access prohibited".
- _shared.js: signInViaLdap drives the LDAP-default Authenticate form (no mode picker).
- _shared.js: click-driven openwebuiLogout (avatar → Sign Out) with first-login modal dismissal.
- _shared.js: expectSignInRequiredAfterLogout drives Keycloak end-session confirm with contract-permitted cleanup fallback.
- _shared.js: actionableClick fires native DOM clicks for off-viewport SSO/LDAP submit buttons.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 04:23:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 958e77ca4b feat(email): wire SMTP env-var integration into openwebui, flowise, pretix, gitlab
Apps with native SMTP env-var support now consume `lookup('email', application_id)`
behind `email.enabled` guards:

- web-app-openwebui: SMTP_HOST/PORT/USERNAME/PASSWORD/FROM_ADDRESS
- web-app-flowise:   SMTP_HOST/PORT/USER/PASSWORD, SENDER_EMAIL
- web-app-pretix:    PRETIX_MAIL_FROM/HOST/PORT/USER/PASSWORD/TLS/SSL
- web-app-gitlab:    new GITLAB_OMNIBUS_SMTP list appended to GITLAB_OMNIBUS_ALL

openwebui and flowise also declare `compose.services.email` (gitlab and pretix
already did). gitlab and pretix map start_tls/tls from the lookup onto their
respective vendor TLS/SSL switches.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-21 23:12:28 +02:00
kevinveenbirkenbachandClaude Opus 4.6 b70a783423 refactor(lookups): migrate to runtime applications/users lookups
Replaces inline `applications`/`users` merging with dedicated lookup
plugins and runtime hydration. Introduces `add_csp_hash` filter,
integrates sort into `find_dock_val_by_bkp_entr`, splits the Keycloak
one-shot realm import into `05a_realm_import_run.yml`, and fixes the
Mastodon administrator task YAML. Removes obsolete `csp_hashes` filter,
`sys-token-store` Jinja filter, and empty CLI setup modules.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 23:44:19 +02:00
Environment TestandClaude Sonnet 4.6 750c110f45 fix(docs): apply documentation rules, fix URLs, and remove orphaned references
- Apply documentation.md rules across staged files (emojis after headlines, RFC 2119 keywords, correct link text, Credits format)
- Remove expired ChatGPT share URLs from 39+ role READMEs and templates
- Remove orphaned text after link deletion: bare # @see, {# @see  #}, Discussion: #}, empty Chat Conversation sections, dangling "Conversation is here available.", "Refer to the conversation with ChatGPT ()", "See ChatGPT Conversation", "ChatGPT Session Transcript" references
- Remove empty ## Further Resources and ## Credits sections where no links remain
- Fix URL reachability test: treat 401/403/405 as ok, 429/500/503 as warn, Timeout as warn; add # nocheck: url suppression (inline and line-above)
- Add lint test for broken relative and repo-root-absolute markdown links
- Add # nocheck: url to AUR git clone URLs, CSP allowlist entries, and other non-browser URLs
- Restore docs.infinito.nexus/setup in documentation.md MUST NOT example
- Fix Mobilizon geospatial endpoints (Addok, Photon) reverted to original values with # nocheck: url
- Add integration test rule: MUST NOT write unit tests for integration test logic

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 00:38:42 +02:00
kevinveenbirkenbach 8b1ae03e77 fix(env): prevent dotenv_quote double-quoting in env.j2 templates
- Replace `"{{ ... | dotenv_quote }}"` with `{{ ... | dotenv_quote }}` across roles
- Update integration test to enforce pure Jinja + dotenv_quote and forbid wrapped quotes

https://chatgpt.com/share/6974e6c6-51b8-800f-80a1-a76f885cf57e
2026-01-24 16:35:27 +01:00
kevinveenbirkenbach fdb2683ddb test(env): enforce dotenv_quote + double quotes for password vars in env.j2 templates
https://chatgpt.com/share/6974cbac-023c-800f-8198-fd85df2aa7bc
2026-01-24 15:32:33 +01:00
kevinveenbirkenbach aa7d0c642c feat(dotenv): add dotenv_quote filter and enforce quoted password vars in env.j2
https://chatgpt.com/share/6974cbac-023c-800f-8198-fd85df2aa7bc
2026-01-24 15:22:19 +01:00
kevinveenbirkenbach ecb8c67782 web-app-openwebui: add ldap service config and fix boolean checks in env template
- Add docker.services.ldap config with enabled/shared flags
- Fix Jinja conditionals by casting OPENWEBUI_OIDC_ENABLED and OPENWEBUI_LDAP_ENABLED to bool
- Align variable formatting in vars/main.yml

https://chatgpt.com/share/696864da-bd44-800f-8e2e-990eee8130e7
2026-01-15 04:53:57 +01:00
kevinveenbirkenbach f4cf55b3c8 Open WebUI OIDC & proxy fixes + Ollama preload + async-safe pull
- svc-ai-ollama:
  - Add preload_models (llama3, mistral, nomic-embed-text)
  - Pre-pull task: loop_var=model, async-safe changed_when/failed_when

- sys-svc-proxy (OpenResty):
  - Forward Authorization header
  - Ensure proxy_pass_request_headers on

- web-app-openwebui:
  - ADMIN_EMAIL from users.administrator.email
  - Request RBAC group scope in OAUTH_SCOPES

Ref: ChatGPT support (2025-09-23) — https://chatgpt.com/share/68d20588-2584-800f-aed4-26ce710c69c4
2025-09-23 04:27:46 +02:00
kevinveenbirkenbach 790c184e66 feat(web-app-openwebui): add bootstrap admin configuration via ADMIN_EMAIL
Introduce ADMIN_EMAIL and SHOW_ADMIN_DETAILS options to bootstrap the first
administrator account on fresh installations. This ensures at least one admin
exists without manual database intervention.

Conversation: https://chatgpt.com/share/68d18e02-d6b8-800f-aaab-920c61b9284a
2025-09-22 21:41:32 +02:00
kevinveenbirkenbach 08720a43c1 feat(web-app-openwebui): enable OIDC role-based admin mapping
Activate ENABLE_OAUTH_ROLE_MANAGEMENT and configure OAUTH_ROLES_CLAIM from
RBAC.GROUP.CLAIM. Define OAUTH_ADMIN_ROLES dynamically based on RBAC group
and application administrator naming convention.

Conversation: https://chatgpt.com/share/68d18e02-d6b8-800f-aaab-920c61b9284a
2025-09-22 20:27:01 +02:00
kevinveenbirkenbach 002f45d1df Added LDAP draft for Open WebUI - Deactivated just PoC, because OIDC is anyhow prefered 2025-09-22 20:02:36 +02:00
kevinveenbirkenbach 70d395ed15 feat(web-app-openwebui): add OIDC support via env.j2 with feature flag
Enables OIDC login by adding feature flag (features.oidc), rendering OIDC-related
environment variables, and introducing OPENWEBUI_OIDC_ENABLED.

Conversation: https://chatgpt.com/share/68d18e02-d6b8-800f-aaab-920c61b9284a
2025-09-22 19:57:55 +02:00
kevinveenbirkenbach 5d1210d651 feat(ai): introduce dedicated AI roles and wiring; clean up legacy AI stack
• Add svc-ai category under roles and load it in constructor stage

• Create new 'svc-ai-ollama' role (vars, tasks, compose, meta, README) and dedicated network

• Refactor former AI stack into separate app roles: web-app-flowise and web-app-openwebui

• Add web-app-minio role; adjust config (no central DB), meta (fa-database, run_after), compose networks include, volume key

• Provide user-focused READMEs for Flowise, OpenWebUI, MinIO, Ollama

• Networks: add subnets for web-app-openwebui, web-app-flowise, web-app-minio; rename web-app-ai → svc-ai-ollama

• Ports: rename ai_* keys to web-app-openwebui / web-app-flowise; keep minio_api/minio_console

• Add group_vars/all/17_ai.yml (OLLAMA_BASE_LOCAL_URL, OLLAMA_LOCAL_ENABLED)

• Replace hardcoded include paths with path_join in multiple roles (svc-db-postgres, sys-service, sys-stk-front-proxy, sys-stk-full-stateful, sys-svc-webserver, web-svc-cdn, web-app-keycloak)

• Remove obsolete web-app-ai templates/vars/env; split Flowise into its own role

• Minor config cleanups (CSP flags to {}, central_database=false)

https://chatgpt.com/share/68d15cb8-cf18-800f-b853-78962f751f81
2025-09-22 18:40:20 +02:00