kevinveenbirkenbach/computer-playbook
1
0 Fork 0
mirror of https://github.com/kevinveenbirkenbach/computer-playbook.git synced 2026-11-06 10:59:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
fix/dev-python/python-upper-bound
computer-playbook/docs/requirements/036-native-ai-gateway-integration.md
T
kevinveenbirkenbachandClaude Opus 5 5324eadd8b fix(ai): resolve the gateway backend from the inventory, not from one host 
LITELLM_BACKEND_ROLES read lookup('deployment').groups, which deployment.py
fills from group_names. That variable is host-scoped: it lists only the groups
the host currently being processed belongs to.

On a swarm round the manager carries svc-ai-ollama, svc-ai-lmstudio and
svc-ai-litellm while the workers carry three groups and none of them is an AI
backend. Both branches of LITELLM_CHAT_MODEL therefore missed on a worker and,
with OPENROUTER_API_KEY empty, the model resolved to the empty string.

Only one task made that visible. Every stateful task in the gateway include
carries delegate_to STACK_HOST plus run_once, so it templates in the manager's
scope and wrote the correct model into the database. The assert carries
neither, so it re-templated in each worker's own scope and failed there while
the manager passed.

The manager-only membership is not a placement decision. The swarm test
inventory is provisioned with a single --host MGR, so every role of the round's
closure gets that one host, and extend_inventory then widens only
svc-swarm-node, svc-storage-nfs-client, svc-storage-nfs-server and the round's
own application. The three AI roles declare no placement at all; they are
manager-only because nothing widened them.

Add an inventory selector next to the host-scoped one, built from the play-wide
groups magic var, skipping groups without hosts and Ansible's implicit all and
ungrouped, sorted so the cache key is stable. The key takes it too, or two
hosts with equal group_names but different inventories would share an entry.

The blast radius is two readers, both in 16_ai.yml, and both ask only whether
svc-ai-ollama or svc-ai-lmstudio is present. On the manager the new value gains
exactly svc-storage-nfs-server and svc-bkp-nfs-2-local, neither of which either
reader tests, so the manager's answer does not move and only the workers change,
to match it.

This is the third step on the same line. 46a16127fc set .running; b149ad0d22
corrected that to .groups because a targeted apps= round leaves only the
command-line roles in the whitelist. .groups solves the targeted round and not
the multi-host one, because group_names stays host-scoped either way.

Measured in a two-host play with a real group_vars file and the real lookup:
the worker resolves ollama false through group_names and true through the new
selector, the manager is true both ways, and the assert that failed in CI
passes on both hosts.

Requirement 036 named the host-scoped backend selection as the thing that has
to change and deferred it. That residual is closed, so the paragraph now
describes the selector in use and why the two rejected ones are wrong.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 19:28:44 +02:00

86 lines
12 KiB
Markdown
Raw Permalink Blame History

# 036 - Native AI Gateway Integration
## User Story
As an operator of a self-hosted deployment, I want every role whose application ships a native AI feature to reach the platform's own LiteLLM gateway instead of a third-party provider, so that prompts, documents and telemetry never leave the deployment and one credential rotation covers every application.
## Context
`svc-ai-litellm` already serves an OpenAI-compatible gateway, and `web-app-nextcloud` already binds its `integration_openai` app to that gateway through `tasks/addons/integration_openai.yml`, a `credentials.litellm_api_key` entry and a dedicated Playwright spec. That pattern is implemented once and is not required, described or verified for any other role.
A sweep over all 85 application roles for provider references (`openai`, `ollama`, `llm`, `anthropic`, `aiprovider`) in `meta/`, `vars/` and `templates/` returns only `dashboard`, `flowise`, `hermes`, `litellm`, `matrix`, `minio`, `nextcloud`, `openclaw` and `openwebui`. Everything below the AI-native roles is therefore unwired today.
Confirmed against upstream documentation, each accepting a custom OpenAI-compatible base URL:
| Role | Native surface | Endpoint |
| --- | --- | --- |
| `web-app-nextcloud` | Assistant via `integration_openai` | service URL plus API key |
| `web-app-mattermost` | Agents plugin | "OpenAI Compatible" service with API URL; a non-empty key is mandatory even when the upstream ignores it |
| `web-app-moodle` | AI subsystem provider | core `aiprovider_openai`; each action carries its own `action_<name>_endpoint`, so the pinned 4.5 branch needs no third-party provider plugin (`aiprovider_openaicompatible` requires Moodle 5.0) |
| `web-app-discourse` | Discourse AI | custom LLM; the host must be listed in `DISCOURSE_ALLOWED_INTERNAL_HOSTS` |
| `web-app-zammad` | Smart Assist | settings `ai_provider` (boolean "a provider is configured") plus `ai_provider_config` (`provider: custom_open_ai`, `url`, `model`, optional `token`). Needs 7.1.2; `stable-6.5` declares neither setting. `Setting.set` runs `AI::Provider::CustomOpenAI.ping!`, so writing the config is itself a reachability proof |
| `web-app-matrix` | ChatGPT bridge | bridge configuration; see the credential defect below |
| `web-app-wordpress` | plugin-provided, no core surface | a connector plugin such as AI Engine exposes a custom OpenAI-compatible endpoint |
| `web-app-n8n` | AI Agent nodes | a stored `openAiApi` credential carries `url` plus `apiKey`. The built-in assistant is out of reach: 1.95.3 exposes only `N8N_AI_ASSISTANT_BASE_URL`, which expects n8n's own assistant service rather than an OpenAI-compatible endpoint, and no `N8N_INSTANCE_AI_MODEL_URL` exists in that release |
| `web-app-mediawiki` | `AIEditingAssistant` extension | provider `open-ai`; `$wgAIEditingAssistantActiveProviderConnection` carries `url`, `endpoint`, `model` and `secret`. Hard-requires `VisualEditorPlus`, which is not bundled |
| `web-app-xwiki` | AI LLM Application extension | the role installs the extension and points its OpenAI-compatible provider at the gateway in `tasks/addons/ai-llm.yml` |
`web-app-matrix` is the one role that already ships an AI credential and points it at the vendor: `meta/secrets.yml` declares `chatgpt_bridge_openai_api_key` with the validation `^sk-[a-zA-Z0-9]{40,}$`. That pattern enforces OpenAI's own key format and therefore **rejects a LiteLLM virtual key by construction**, so the schema has to change before the bridge can reach the gateway at all.
`web-app-wordpress` has no AI surface in core; the requirement applies to whichever connector plugin the role installs, and the role MUST pin one rather than leaving the choice to a site administrator. The pinned connector is `ai-engine`, whose `custom` engine type takes an OpenAI-compatible base URL and an API key per environment.
`web-app-zammad` was pinned at 6.5.0, which ships no AI surface at all. The role is bumped to 7.1.2, the newest line whose contract is still the `ai_provider` / `ai_provider_config` setting pair; past 7.1.x the provider moves into an `AI::ProviderConnection` record and the contract changes. 7.1.2 accepts Elasticsearch `>= 7.8, < 10`, so the role's existing 8.13.4 pin stands.
The three candidates that vendor documentation claimed were probed against running instances on 2026-08-27, and each is filed below on what the deployed artefact actually offers rather than on the claim:
| Role | Measured at | Finding | Disposition |
| --- | --- | --- | --- |
| `web-app-baserow` | 2.3.3 | `settings.BASEROW_OPENAI_BASE_URL` reaches the OpenAI client at `backend/src/baserow/core/generative_ai/generative_ai_model_types.py:288`, but the only two surfaces that ever send a prompt live in `baserow_premium` and both open with `LicenseHandler.raise_if_user_doesnt_have_feature(PREMIUM, request.user, workspace)`: `AsyncGenerateAIFieldValuesView` and `GenerateFormulaWithAIView` (`premium/backend/src/baserow_premium/api/fields/views.py`). An unlicensed deployment configures the endpoint and can never reach it | excluded |
| `web-app-homeassistant` | 2026.7 | `openai_conversation/config_flow.py` contains no `base_url` (0 matches), so the OpenAI agent cannot be pointed anywhere but the vendor. The only configurable local path is the separate `ollama` integration, whose `config_flow.py` carries `CONF_URL` (7 matches) but speaks Ollama's native API rather than the gateway's OpenAI-compatible surface | excluded |
| `web-app-xwiki` | `lts-postgres-tomcat` | no AI or LLM extension is present in `data/extension/repository` on the pinned image, so there is no AI surface to point at the gateway until the XWiki AI LLM Application extension is installed by the role | confirmed, the role installs the extension in `tasks/addons/ai-llm.yml` |
`web-app-baserow` is the one row measured against the pinned source rather than a running instance. Confirm it on the deployed stack with `POST /api/database/fields/<field_id>/generate-ai-field-values/` as the seeded superuser: an unlicensed instance answers HTTP 402.
Excluded, with the reason recorded so the question is not reopened:
| Role | Reason |
| --- | --- |
| `web-app-baserow` | the endpoint is configurable but both prompt-sending views are premium-licence gated; see the measurement above |
| `web-app-homeassistant` | the OpenAI agent takes no base URL and the `ollama` integration speaks Ollama's native API; see the measurement above |
| `web-app-gitlab` | Duo is bound to the vendor service and exposes no self-hosted endpoint |
| `web-app-odoo` | version 19 hardcodes a vendor model for AI server actions; a configurable endpoint needs third-party modules |
| `web-app-erpnext` | no AI surface in the core; every option is a separately installed app |
| `web-app-jira`, `web-app-confluence` | Atlassian Intelligence is vendor-hosted |
| `web-app-magento`, `web-app-shopware` | the shipped assistants are vendor services; only third-party modules expose an endpoint |
| `web-app-openproject` | no AI surface found in the product |
| `web-app-espocrm` | measured at tag 10.0.4: zero tree paths match `AI`/`Intelligence`/`OpenAI`/`LLM`/`GPT`/`prompt` while 116 match `Mail` on the same payload, so the search was live. The AI features live only in the closed-source Intelligence extension, which needs a purchased licence and is published in no `espocrm` repository. A version bump does not help: upstream documents the extension as requiring 10.0.3 or greater, so newer cores host it rather than absorb it |
| `web-app-peertube` | measured at tag v8.2.4 across three independent surfaces: `config/production.yaml.example` `video_transcription:` carries only `enabled`/`engine`/`engine_path`/`model`/`model_path`/`timeout`/`remote_runners.enabled`; `support/docker/production/config/custom-environment-variables.yaml` exposes exactly the seven matching `PEERTUBE_VIDEO_TRANSCRIPTION_*` variables, none of them a URL or a key; `server/core/initializers/config.ts` agrees. Transcription is local Whisper, not an OpenAI-compatible endpoint |
| infrastructure and static roles | `checkmk`, `prometheus`, `matomo`, `seaweedfs`, `minio`, `mailu`, `keycloak`, `lam`, `fusiondirectory`, `phpldapadmin`, `phpmyadmin`, `pgadmin`, `pihole`, `semaphore`, `jenkins`, `gitea`, `hugo`, `sphinx`, `yourls`, `mini-qr`, `littlejs`, `chess`, `roulette-wheel`, `navigator`, `mig`, `dashboard`, `fediwall`, `postmarks` carry no AI surface |
| fediverse and media roles | `mastodon`, `pixelfed`, `friendica`, `socialhome`, `bookwyrm`, `mobilizon`, `funkwhale`, `bluesky`, `bridgy-fed`, `jellyfin` ship no configurable AI surface |
| remaining application roles | `akaunting`, `decidim`, `fider`, `listmonk`, `pretix`, `snipe-it`, `taiga`, `penpot`, `joomla`, `suitecrm`, `kix`, `opencloud`, `opentalk`, `jitsi`, `bigbluebutton`, `erpnext` show no first-party AI surface with a configurable endpoint; revisit when upstream adds one |
## Acceptance Criteria
- [ ] Each unverified candidate is probed against a running instance and moved into the confirmed table or the exclusion table with its reason, so the scope rests on measurement rather than vendor claims.
- [x] Every role in the confirmed table declares its native AI surface in `meta/services.yml` with `enabled` and `shared` bound to `'svc-ai-litellm' in group_names`, so a deployment without the gateway configures no AI surface at all.
- [x] Every role in the confirmed table carries a `credentials.litellm_api_key` entry in `meta/secrets.yml`, and the deploy writes that virtual key into the application's own AI configuration rather than a shared or hardcoded key.
- [x] Every role in the confirmed table points its AI base URL at the in-cluster gateway address, and a deploy-time assertion fails when the configured URL resolves outside the deployment.
- [x] No role in the confirmed table retains a default that sends requests to a third-party provider when the gateway is enabled.
- [x] A Playwright spec per role proves the configured surface answers a prompt through the gateway, mirroring `roles/web-app-nextcloud/files/playwright/addons/integration_openai.spec.js`.
- [x] The deploy fails when a role declares the AI surface enabled while its configuration still carries an empty or unresolved API key, so a silently unauthenticated surface cannot reach a green deploy.
- [x] No credential validation in `meta/secrets.yml` encodes a vendor key format, so a gateway-issued virtual key satisfies every AI credential the platform mints.
- [x] `docs/requirements/027-integration-matrix.md` lists the AI surface of every role in the confirmed table.
Consumers that must name a model (`web-app-moodle`, `web-app-matrix`) read `LITELLM_CHAT_MODEL`, which picks the first non-embedding preload model of `svc-ai-ollama` and mirrors the branch order of the gateway's own `config.yaml.j2`. The consumer resolves that branch from `lookup('deployment').inventory`, every group the round's inventory populates, read from the play-wide `groups` magic var. `.running` is wrong here: a targeted `apps=` round sets it to the whitelist, and `svc-ai-ollama` is pulled in as a dependency rather than named on the command line, so both backend branches fall through to the `openrouter/auto` vendor default. The gateway then answers `Invalid model name passed in model=openrouter/auto` with HTTP 400 and no consumer can complete a prompt. `.groups` is wrong for the mirror-image reason: it carries `group_names`, which is host-scoped, so on a multi-host round a worker that hosts neither the gateway nor a backend resolves an empty model while the manager resolves the right one.
### Which variant exercises the gateway
`svc-ai-litellm` pulls `svc-ai-ollama` and `svc-ai-lmstudio` with it, so a consumer that enables the gateway adds roughly 6 GB to its round. Variant 0 is the fallback every higher round lands on, so a heavy shared dependency pinned there appears in every round: with the gateway on variant 0 of `web-app-matrix`, `web-app-mattermost` and `web-app-zammad`, the `web-app-nextcloud` rounds that pin those partners exceed the 64 GB budget guarded by `tests/integration/roles/meta/variants/test_resource_budget.py`.
The gateway is therefore pinned on **variant 2** for every consumer that ships three variants, and left on variant 0 for the two-variant roles, whose round 2 falls back to it. One `make compose-deploy variant=2` run exercises eight of the nine confirmed roles. `web-app-nextcloud` is the exception: its AI surface has always lived on its own variant 7, alongside the other bridged partners it distributes across budget bins.
## See Also
- [031 - LLM Gateway Model Backends](031-llm-gateway-model-backends.md)
- [025 - MCP Role Integration](025-mcp-role-integration.md)
Reference in new issue View Git Blame Copy Permalink