kevinveenbirkenbachandClaude Opus 5
5324eadd8b
LITELLM_BACKEND_ROLES read lookup('deployment').groups, which deployment.py
fills from group_names. That variable is host-scoped: it lists only the groups
the host currently being processed belongs to.
On a swarm round the manager carries svc-ai-ollama, svc-ai-lmstudio and
svc-ai-litellm while the workers carry three groups and none of them is an AI
backend. Both branches of LITELLM_CHAT_MODEL therefore missed on a worker and,
with OPENROUTER_API_KEY empty, the model resolved to the empty string.
Only one task made that visible. Every stateful task in the gateway include
carries delegate_to STACK_HOST plus run_once, so it templates in the manager's
scope and wrote the correct model into the database. The assert carries
neither, so it re-templated in each worker's own scope and failed there while
the manager passed.
The manager-only membership is not a placement decision. The swarm test
inventory is provisioned with a single --host MGR, so every role of the round's
closure gets that one host, and extend_inventory then widens only
svc-swarm-node, svc-storage-nfs-client, svc-storage-nfs-server and the round's
own application. The three AI roles declare no placement at all; they are
manager-only because nothing widened them.
Add an inventory selector next to the host-scoped one, built from the play-wide
groups magic var, skipping groups without hosts and Ansible's implicit all and
ungrouped, sorted so the cache key is stable. The key takes it too, or two
hosts with equal group_names but different inventories would share an entry.
The blast radius is two readers, both in 16_ai.yml, and both ask only whether
svc-ai-ollama or svc-ai-lmstudio is present. On the manager the new value gains
exactly svc-storage-nfs-server and svc-bkp-nfs-2-local, neither of which either
reader tests, so the manager's answer does not move and only the workers change,
to match it.
This is the third step on the same line. 46a16127fc set .running; b149ad0d22
corrected that to .groups because a targeted apps= round leaves only the
command-line roles in the whitelist. .groups solves the targeted round and not
the multi-host one, because group_names stays host-scoped either way.
Measured in a two-host play with a real group_vars file and the real lookup:
the worker resolves ollama false through group_names and true through the new
selector, the manager is true both ways, and the assert that failed in CI
passes on both hosts.
Requirement 036 named the host-scoped backend selection as the thing that has
to change and deferred it. That residual is closed, so the paragraph now
describes the selector in use and why the two rejected ones are wrong.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
12 KiB
036 - Native AI Gateway Integration
User Story
As an operator of a self-hosted deployment, I want every role whose application ships a native AI feature to reach the platform's own LiteLLM gateway instead of a third-party provider, so that prompts, documents and telemetry never leave the deployment and one credential rotation covers every application.
Context
svc-ai-litellm already serves an OpenAI-compatible gateway, and web-app-nextcloud already binds its integration_openai app to that gateway through tasks/addons/integration_openai.yml, a credentials.litellm_api_key entry and a dedicated Playwright spec. That pattern is implemented once and is not required, described or verified for any other role.
A sweep over all 85 application roles for provider references (openai, ollama, llm, anthropic, aiprovider) in meta/, vars/ and templates/ returns only dashboard, flowise, hermes, litellm, matrix, minio, nextcloud, openclaw and openwebui. Everything below the AI-native roles is therefore unwired today.
Confirmed against upstream documentation, each accepting a custom OpenAI-compatible base URL:
| Role | Native surface | Endpoint |
|---|---|---|
web-app-nextcloud |
Assistant via integration_openai |
service URL plus API key |
web-app-mattermost |
Agents plugin | "OpenAI Compatible" service with API URL; a non-empty key is mandatory even when the upstream ignores it |
web-app-moodle |
AI subsystem provider | core aiprovider_openai; each action carries its own action_<name>_endpoint, so the pinned 4.5 branch needs no third-party provider plugin (aiprovider_openaicompatible requires Moodle 5.0) |
web-app-discourse |
Discourse AI | custom LLM; the host must be listed in DISCOURSE_ALLOWED_INTERNAL_HOSTS |
web-app-zammad |
Smart Assist | settings ai_provider (boolean "a provider is configured") plus ai_provider_config (provider: custom_open_ai, url, model, optional token). Needs 7.1.2; stable-6.5 declares neither setting. Setting.set runs AI::Provider::CustomOpenAI.ping!, so writing the config is itself a reachability proof |
web-app-matrix |
ChatGPT bridge | bridge configuration; see the credential defect below |
web-app-wordpress |
plugin-provided, no core surface | a connector plugin such as AI Engine exposes a custom OpenAI-compatible endpoint |
web-app-n8n |
AI Agent nodes | a stored openAiApi credential carries url plus apiKey. The built-in assistant is out of reach: 1.95.3 exposes only N8N_AI_ASSISTANT_BASE_URL, which expects n8n's own assistant service rather than an OpenAI-compatible endpoint, and no N8N_INSTANCE_AI_MODEL_URL exists in that release |
web-app-mediawiki |
AIEditingAssistant extension |
provider open-ai; $wgAIEditingAssistantActiveProviderConnection carries url, endpoint, model and secret. Hard-requires VisualEditorPlus, which is not bundled |
web-app-xwiki |
AI LLM Application extension | the role installs the extension and points its OpenAI-compatible provider at the gateway in tasks/addons/ai-llm.yml |
web-app-matrix is the one role that already ships an AI credential and points it at the vendor: meta/secrets.yml declares chatgpt_bridge_openai_api_key with the validation ^sk-[a-zA-Z0-9]{40,}$. That pattern enforces OpenAI's own key format and therefore rejects a LiteLLM virtual key by construction, so the schema has to change before the bridge can reach the gateway at all.
web-app-wordpress has no AI surface in core; the requirement applies to whichever connector plugin the role installs, and the role MUST pin one rather than leaving the choice to a site administrator. The pinned connector is ai-engine, whose custom engine type takes an OpenAI-compatible base URL and an API key per environment.
web-app-zammad was pinned at 6.5.0, which ships no AI surface at all. The role is bumped to 7.1.2, the newest line whose contract is still the ai_provider / ai_provider_config setting pair; past 7.1.x the provider moves into an AI::ProviderConnection record and the contract changes. 7.1.2 accepts Elasticsearch >= 7.8, < 10, so the role's existing 8.13.4 pin stands.
The three candidates that vendor documentation claimed were probed against running instances on 2026-08-27, and each is filed below on what the deployed artefact actually offers rather than on the claim:
| Role | Measured at | Finding | Disposition |
|---|---|---|---|
web-app-baserow |
2.3.3 | settings.BASEROW_OPENAI_BASE_URL reaches the OpenAI client at backend/src/baserow/core/generative_ai/generative_ai_model_types.py:288, but the only two surfaces that ever send a prompt live in baserow_premium and both open with LicenseHandler.raise_if_user_doesnt_have_feature(PREMIUM, request.user, workspace): AsyncGenerateAIFieldValuesView and GenerateFormulaWithAIView (premium/backend/src/baserow_premium/api/fields/views.py). An unlicensed deployment configures the endpoint and can never reach it |
excluded |
web-app-homeassistant |
2026.7 | openai_conversation/config_flow.py contains no base_url (0 matches), so the OpenAI agent cannot be pointed anywhere but the vendor. The only configurable local path is the separate ollama integration, whose config_flow.py carries CONF_URL (7 matches) but speaks Ollama's native API rather than the gateway's OpenAI-compatible surface |
excluded |
web-app-xwiki |
lts-postgres-tomcat |
no AI or LLM extension is present in data/extension/repository on the pinned image, so there is no AI surface to point at the gateway until the XWiki AI LLM Application extension is installed by the role |
confirmed, the role installs the extension in tasks/addons/ai-llm.yml |
web-app-baserow is the one row measured against the pinned source rather than a running instance. Confirm it on the deployed stack with POST /api/database/fields/<field_id>/generate-ai-field-values/ as the seeded superuser: an unlicensed instance answers HTTP 402.
Excluded, with the reason recorded so the question is not reopened:
| Role | Reason |
|---|---|
web-app-baserow |
the endpoint is configurable but both prompt-sending views are premium-licence gated; see the measurement above |
web-app-homeassistant |
the OpenAI agent takes no base URL and the ollama integration speaks Ollama's native API; see the measurement above |
web-app-gitlab |
Duo is bound to the vendor service and exposes no self-hosted endpoint |
web-app-odoo |
version 19 hardcodes a vendor model for AI server actions; a configurable endpoint needs third-party modules |
web-app-erpnext |
no AI surface in the core; every option is a separately installed app |
web-app-jira, web-app-confluence |
Atlassian Intelligence is vendor-hosted |
web-app-magento, web-app-shopware |
the shipped assistants are vendor services; only third-party modules expose an endpoint |
web-app-openproject |
no AI surface found in the product |
web-app-espocrm |
measured at tag 10.0.4: zero tree paths match AI/Intelligence/OpenAI/LLM/GPT/prompt while 116 match Mail on the same payload, so the search was live. The AI features live only in the closed-source Intelligence extension, which needs a purchased licence and is published in no espocrm repository. A version bump does not help: upstream documents the extension as requiring 10.0.3 or greater, so newer cores host it rather than absorb it |
web-app-peertube |
measured at tag v8.2.4 across three independent surfaces: config/production.yaml.example video_transcription: carries only enabled/engine/engine_path/model/model_path/timeout/remote_runners.enabled; support/docker/production/config/custom-environment-variables.yaml exposes exactly the seven matching PEERTUBE_VIDEO_TRANSCRIPTION_* variables, none of them a URL or a key; server/core/initializers/config.ts agrees. Transcription is local Whisper, not an OpenAI-compatible endpoint |
| infrastructure and static roles | checkmk, prometheus, matomo, seaweedfs, minio, mailu, keycloak, lam, fusiondirectory, phpldapadmin, phpmyadmin, pgadmin, pihole, semaphore, jenkins, gitea, hugo, sphinx, yourls, mini-qr, littlejs, chess, roulette-wheel, navigator, mig, dashboard, fediwall, postmarks carry no AI surface |
| fediverse and media roles | mastodon, pixelfed, friendica, socialhome, bookwyrm, mobilizon, funkwhale, bluesky, bridgy-fed, jellyfin ship no configurable AI surface |
| remaining application roles | akaunting, decidim, fider, listmonk, pretix, snipe-it, taiga, penpot, joomla, suitecrm, kix, opencloud, opentalk, jitsi, bigbluebutton, erpnext show no first-party AI surface with a configurable endpoint; revisit when upstream adds one |
Acceptance Criteria
- Each unverified candidate is probed against a running instance and moved into the confirmed table or the exclusion table with its reason, so the scope rests on measurement rather than vendor claims.
- Every role in the confirmed table declares its native AI surface in
meta/services.ymlwithenabledandsharedbound to'svc-ai-litellm' in group_names, so a deployment without the gateway configures no AI surface at all. - Every role in the confirmed table carries a
credentials.litellm_api_keyentry inmeta/secrets.yml, and the deploy writes that virtual key into the application's own AI configuration rather than a shared or hardcoded key. - Every role in the confirmed table points its AI base URL at the in-cluster gateway address, and a deploy-time assertion fails when the configured URL resolves outside the deployment.
- No role in the confirmed table retains a default that sends requests to a third-party provider when the gateway is enabled.
- A Playwright spec per role proves the configured surface answers a prompt through the gateway, mirroring
roles/web-app-nextcloud/files/playwright/addons/integration_openai.spec.js. - The deploy fails when a role declares the AI surface enabled while its configuration still carries an empty or unresolved API key, so a silently unauthenticated surface cannot reach a green deploy.
- No credential validation in
meta/secrets.ymlencodes a vendor key format, so a gateway-issued virtual key satisfies every AI credential the platform mints. docs/requirements/027-integration-matrix.mdlists the AI surface of every role in the confirmed table.
Consumers that must name a model (web-app-moodle, web-app-matrix) read LITELLM_CHAT_MODEL, which picks the first non-embedding preload model of svc-ai-ollama and mirrors the branch order of the gateway's own config.yaml.j2. The consumer resolves that branch from lookup('deployment').inventory, every group the round's inventory populates, read from the play-wide groups magic var. .running is wrong here: a targeted apps= round sets it to the whitelist, and svc-ai-ollama is pulled in as a dependency rather than named on the command line, so both backend branches fall through to the openrouter/auto vendor default. The gateway then answers Invalid model name passed in model=openrouter/auto with HTTP 400 and no consumer can complete a prompt. .groups is wrong for the mirror-image reason: it carries group_names, which is host-scoped, so on a multi-host round a worker that hosts neither the gateway nor a backend resolves an empty model while the manager resolves the right one.
Which variant exercises the gateway
svc-ai-litellm pulls svc-ai-ollama and svc-ai-lmstudio with it, so a consumer that enables the gateway adds roughly 6 GB to its round. Variant 0 is the fallback every higher round lands on, so a heavy shared dependency pinned there appears in every round: with the gateway on variant 0 of web-app-matrix, web-app-mattermost and web-app-zammad, the web-app-nextcloud rounds that pin those partners exceed the 64 GB budget guarded by tests/integration/roles/meta/variants/test_resource_budget.py.
The gateway is therefore pinned on variant 2 for every consumer that ships three variants, and left on variant 0 for the two-variant roles, whose round 2 falls back to it. One make compose-deploy variant=2 run exercises eight of the nine confirmed roles. web-app-nextcloud is the exception: its AI surface has always lived on its own variant 7, alongside the other bridged partners it distributes across budget bins.