diff --git a/.github/PULL_REQUEST_TEMPLATE/workstation.md b/.github/PULL_REQUEST_TEMPLATE/workstation.md index fc0d189fac..3b39816506 100644 --- a/.github/PULL_REQUEST_TEMPLATE/workstation.md +++ b/.github/PULL_REQUEST_TEMPLATE/workstation.md @@ -1,6 +1,6 @@ ## Summary -Briefly describe the `desk-*` change and the user workflow it affects. +Briefly describe the `dsk-*` change and the user workflow it affects. Examples: @@ -24,7 +24,7 @@ Select the primary intent of this PR: List the impacted roles and workstation context. -* Primary `desk-*` role(s): +* Primary `dsk-*` role(s): * Related `dev-*`, `drv-*`, `user-*`, or `sys-*` role(s): * Distro(s) and desktop environment(s) tested: @@ -36,7 +36,7 @@ Optional. Ignored unless a maintainer applies the **🧩 Subset** label; without ```yaml roles: - # - desk-firefox + # - dsk-firefox # - sys-version ``` diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 5bf51384fe..ba7a88d4e9 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -8,7 +8,7 @@ Then select the template that best matches your change: [![Server](https://img.shields.io/badge/Server-web--*-0A7B83?style=for-the-badge)](?quick_pull=1&template=server.md) -[![Workstation](https://img.shields.io/badge/Workstation-desk--*-2E6F40?style=for-the-badge)](?quick_pull=1&template=workstation.md) +[![Workstation](https://img.shields.io/badge/Workstation-dsk--*-2E6F40?style=for-the-badge)](?quick_pull=1&template=workstation.md) [![System](https://img.shields.io/badge/System-sys%2Fsvc%2Fdev-8B5E3C?style=for-the-badge)](?quick_pull=1&template=system.md) [![Pipeline](https://img.shields.io/badge/Pipeline-CI%2FCD-7A3E9D?style=for-the-badge)](?quick_pull=1&template=pipeline.md) diff --git a/AGENTS.md b/AGENTS.md index 14b5e52c05..f813d157a8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,7 +4,8 @@ - [CONTRIBUTING.md](CONTRIBUTING.md) is the single source of truth for contributor workflow, coding standards, testing, and review. You MUST read it. - You MUST read every file under `docs/contributing/` (full directory walk, including subdirectories) for the full contributor guidance. -- You MUST read every file under `docs/agents/` (full directory walk, including subdirectories) for the agent execution flow. +- You MUST read every file under `docs/agents/` (full directory walk, including subdirectories) for the agent execution flow, EXCEPT `docs/agents/action/`. +- `docs/agents/action/` holds one runbook per action and is read on demand, not at session start: read a runbook when the matching `i8-` skill routes you to it, or when you are about to perform that action without one. It is the bulk of the agent documentation, and every runbook in it is inert until its action is the task. - This file extends CONTRIBUTING.md with agent-specific instructions; on conflict between CONTRIBUTING.md and this file, this file wins. - This file is runtime-agnostic and binding for every agent (Claude Code, Codex, Gemini CLI, other). There are no tool-specific extension files. diff --git a/README.md b/README.md index 76b2b41d4b..c092ec683a 100644 --- a/README.md +++ b/README.md @@ -60,6 +60,7 @@ Every invokable role in the tested lifecycle envelope, with its upstream homepag | Name | Status | Description | More | Integrated with | |---|---|---|---|---| +| [Agent Broker](roles/svc-ai-agent-broker/) | beta | Per-user Hermes and OpenClaw agents started on demand behind an OpenAI-compatible endpoint | [🛠️](roles/svc-ai-agent-broker/README.md#quick-setup) | [Hermes Agent](roles/web-app-hermes/), [Keycloak](roles/web-app-keycloak/), [LiteLLM Gateway](roles/svc-ai-litellm/), [OpenClaw](roles/web-app-openclaw/) | | [Akaunting](roles/web-app-akaunting/) | beta | Self-hosted platform for online accounting, bookkeeping, and business finance management | [🌐](https://akaunting.com/) [🎬](https://www.youtube.com/watch?v=UTVe_Zob_k0) [🛠️](roles/web-app-akaunting/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [MariaDB](roles/svc-db-mariadb/), [Matomo](roles/web-app-matomo/), [Prometheus](roles/web-app-prometheus/), [SeaweedFS](roles/web-app-seaweedfs/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | | [API](roles/web-svc-api/) | alpha | Public REST API serving role data, history and translations of Infinito.Nexus for every ref of core and its forks | [🌐](https://fastapi.tiangolo.com/) [🛠️](roles/web-svc-api/README.md#quick-setup) | [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/) | | [Asset Mirror (Privacy Proxy)](roles/web-svc-mirror/) | alpha | Nginx-based privacy proxy that mirrors whitelisted third-party assets under a first-party domain | [🛠️](roles/web-svc-mirror/README.md#quick-setup) | [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/) | @@ -76,10 +77,14 @@ Every invokable role in the tested lifecycle envelope, with its upstream homepag | [Bridgy Fed](roles/web-app-bridgy-fed/) | beta | Federation bridge connecting ActivityPub (Fediverse), the AT Protocol (Bluesky), and the IndieWeb | [🌐](https://fed.brid.gy/) [🛠️](roles/web-app-bridgy-fed/README.md#quick-setup) | [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Matomo](roles/web-app-matomo/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/) | | [Checkmk](roles/web-app-checkmk/) | beta | Open-source IT and application monitoring server behind SSO with native LDAP user management | [🌐](https://checkmk.com/) [🎬](https://www.youtube.com/watch?v=opO-SOgOJ1I) [🛠️](roles/web-app-checkmk/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Hermes Agent](roles/web-app-hermes/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [Matomo](roles/web-app-matomo/), [Open WebUI](roles/web-app-openwebui/), [OpenClaw](roles/web-app-openclaw/), [OpenLDAP](roles/svc-db-openldap/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | | [Chess](roles/web-app-chess/) | beta | Federated chess server for playing and following rule-validated games over ActivityPub across the Fediverse | [🌐](https://castling.club/) [🛠️](roles/web-app-chess/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Matomo](roles/web-app-matomo/), [PostgreSQL](roles/svc-db-postgres/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/) | +| [Claude Code](roles/dsk-gnt-claude/) | alpha | Terminal coding agent from Anthropic, pointed at the platform's own model gateway instead of the vendor API | [🌐](https://code.claude.com/) [🛠️](roles/dsk-gnt-claude/README.md#quick-setup) | [LiteLLM Gateway](roles/svc-ai-litellm/) | +| [Code](roles/dsk-code/) | alpha | Visual Studio Code workstation editor in a selectable flavor, carrying the extensions of the agents the host deploys | [🌐](https://github.com/microsoft/vscode) [🛠️](roles/dsk-code/README.md#quick-setup) | [Claude Code](roles/dsk-gnt-claude/), [Codex](roles/dsk-gnt-codex/) | +| [Codex](roles/dsk-gnt-codex/) | alpha | Terminal coding agent from OpenAI, pointed at the platform's own model gateway instead of the vendor API | [🌐](https://github.com/openai/codex) [🛠️](roles/dsk-gnt-codex/README.md#quick-setup) | [LiteLLM Gateway](roles/svc-ai-litellm/) | | [Collabora](roles/web-svc-collabora/) | beta | Self-hosted online office backend for real-time in-browser editing of documents, spreadsheets, and presentations | [🌐](https://www.collaboraonline.com/) [🎬](https://www.youtube.com/watch?v=8pqkclWG2BY) [🛠️](roles/web-svc-collabora/README.md#quick-setup) | [Matomo](roles/web-app-matomo/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/) | | [Content Delivery Network](roles/web-svc-cdn/) | beta | Nginx-based content delivery network for serving shared static assets to internal web applications | [🛠️](roles/web-svc-cdn/README.md#quick-setup) | [Matomo](roles/web-app-matomo/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/) | | [Corporate Design](roles/web-svc-css/) | beta | Service-registry role owning the canonical `css` flag and redirecting the CSS hostname to the CDN upstream | [🛠️](roles/web-svc-css/README.md#quick-setup) | [Content Delivery Network](roles/web-svc-cdn/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/) | | [Coturn](roles/web-svc-coturn/) | beta | Free and open-source TURN/STUN relay server for NAT traversal in WebRTC and real-time communication | [🌐](https://github.com/coturn/coturn) [🛠️](roles/web-svc-coturn/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Prometheus](roles/web-app-prometheus/) | +| [Cursor](roles/dsk-gnt-cursor/) | alpha | AI-first code editor installed on the workstation, whose endpoint override stays an operator step | [🌐](https://cursor.com/) [🛠️](roles/dsk-gnt-cursor/README.md#quick-setup) | | | [Dashboard](roles/web-app-dashboard/) | beta | Self-hosted portal for unified access to all deployed applications via a customizable tile and menu interface | [🛠️](roles/web-app-dashboard/README.md#quick-setup) | [Assets Server](roles/web-svc-asset/), [Content Delivery Network](roles/web-svc-cdn/), [Corporate Design](roles/web-svc-css/), [Keycloak](roles/web-app-keycloak/), [Matomo](roles/web-app-matomo/), [Prometheus](roles/web-app-prometheus/), [Simple Icons](roles/web-svc-simpleicons/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | | [Decidim](roles/web-app-decidim/) | beta | Open-source participatory democracy platform for public consultations, civic processes, and community voting | [🌐](https://decidim.org/) [🎬](https://www.youtube.com/watch?v=luSdPZ-wUIY) [🛠️](roles/web-app-decidim/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [Matomo](roles/web-app-matomo/), [PostgreSQL](roles/svc-db-postgres/), [Prometheus](roles/web-app-prometheus/), [Redis](roles/svc-db-redis/), [SeaweedFS](roles/web-app-seaweedfs/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | | [Discourse](roles/web-app-discourse/) | beta | Open-source discussion and community forum platform with moderation, notifications, and plugin support | [🌐](https://www.discourse.org/) [🎬](https://www.youtube.com/watch?v=z3qp_uB46FA) [🛠️](roles/web-app-discourse/README.md#quick-setup) | [Assets Server](roles/web-svc-asset/), [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [LiteLLM Gateway](roles/svc-ai-litellm/), [Mailu](roles/web-app-mailu/), [Matomo](roles/web-app-matomo/), [PostgreSQL](roles/svc-db-postgres/), [Prometheus](roles/web-app-prometheus/), [Redis](roles/svc-db-redis/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | @@ -111,6 +116,7 @@ Every invokable role in the tested lifecycle envelope, with its upstream homepag | [Jenkins](roles/web-app-jenkins/) | beta | Open-source automation server for CI/CD pipelines, builds, and plugin-driven deployment workflows | [🌐](https://www.jenkins.io/) [🎬](https://www.youtube.com/watch?v=rNy00_gOo8o) [🛠️](roles/web-app-jenkins/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Hermes Agent](roles/web-app-hermes/), [Keycloak](roles/web-app-keycloak/), [Matomo](roles/web-app-matomo/), [Open WebUI](roles/web-app-openwebui/), [OpenClaw](roles/web-app-openclaw/), [OpenLDAP](roles/svc-db-openldap/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | | [Jitsi Meet](roles/web-app-jitsi/) | beta | Self-hosted WebRTC platform for end-to-end-encryptable video conferencing with federated authentication | [🌐](https://jitsi.org/) [🎬](https://www.youtube.com/watch?v=TB7LlM4erx8) [🛠️](roles/web-app-jitsi/README.md#quick-setup) | [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [Matomo](roles/web-app-matomo/), [OpenLDAP](roles/svc-db-openldap/), [Prometheus](roles/web-app-prometheus/), [Universal Logout](roles/web-svc-logout/) | | [Joomla](roles/web-app-joomla/) | beta | Open-source content management system for building websites, intranets, and online applications without writing code | [🌐](https://www.joomla.org/) [🎬](https://www.youtube.com/watch?v=mXFSIC_zM2o) [🛠️](roles/web-app-joomla/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [MariaDB](roles/svc-db-mariadb/), [Matomo](roles/web-app-matomo/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | +| [Jupyter](roles/svc-ai-jupyter/) | alpha | Kernel backend that runs the code a chat writes, in a container instead of in the browser sandbox | [🌐](https://jupyter.org/) [🛠️](roles/svc-ai-jupyter/README.md#quick-setup) | | | [Kata & gVisor](roles/svc-virt-kata/) | beta | Kernel-isolating container runtimes that sandbox untrusted workloads in a microVM or a userspace kernel | [🌐](https://katacontainers.io/) [🛠️](roles/svc-virt-kata/README.md#quick-setup) | | | [Keycloak](roles/web-app-keycloak/) | beta | Self-hosted identity and access management platform for single sign-on, multi-factor authentication, and user federation | [🌐](https://www.keycloak.org/) [🛠️](roles/web-app-keycloak/README.md#quick-setup) | [Content Delivery Network](roles/web-svc-cdn/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Mailu](roles/web-app-mailu/), [Matomo](roles/web-app-matomo/), [PostgreSQL](roles/svc-db-postgres/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/) | | [KIX](roles/web-app-kix/) | beta | Open-source ITSM and helpdesk platform (OTRS fork) for ticketing, CMDB, knowledge base, and reporting | [🌐](https://www.kixdesk.com/en/) [🎬](https://www.youtube.com/watch?v=D62emos1FQ8) [🛠️](roles/web-app-kix/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [Matomo](roles/web-app-matomo/), [OpenLDAP](roles/svc-db-openldap/), [PostgreSQL](roles/svc-db-postgres/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | @@ -119,7 +125,7 @@ Every invokable role in the tested lifecycle envelope, with its upstream homepag | [LID Switch Driver](roles/drv-lid-switch/) | alpha | Power-aware systemd logind configuration for predictable laptop lid-close behavior across battery, AC, and docked states | [🛠️](roles/drv-lid-switch/README.md#quick-setup) | | | [Listmonk](roles/web-app-listmonk/) | beta | Self-hosted platform for newsletters and mailing lists with subscriber segmentation and campaign analytics | [🌐](https://listmonk.app/) [🛠️](roles/web-app-listmonk/README.md#quick-setup) | [Assets Server](roles/web-svc-asset/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Hermes Agent](roles/web-app-hermes/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [Matomo](roles/web-app-matomo/), [Open WebUI](roles/web-app-openwebui/), [OpenClaw](roles/web-app-openclaw/), [PostgreSQL](roles/svc-db-postgres/), [Prometheus](roles/web-app-prometheus/), [SeaweedFS](roles/web-app-seaweedfs/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | | [LiteLLM Admin UI](roles/web-app-litellm/) | beta | Browser-facing admin interface for the shared LLM gateway, published on its own domain behind the front proxy | [🌐](https://docs.litellm.ai/) [🛠️](roles/web-app-litellm/README.md#quick-setup) | [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [LiteLLM Gateway](roles/svc-ai-litellm/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/) | -| [LiteLLM Gateway](roles/svc-ai-litellm/) | beta | OpenAI-compatible LLM gateway routing a single /v1 endpoint to local and hosted model backends with per-consumer keys | [🌐](https://docs.litellm.ai/) [🛠️](roles/svc-ai-litellm/README.md#quick-setup) | [LM Studio](roles/svc-ai-lmstudio/), [Ollama](roles/svc-ai-ollama/), [PostgreSQL](roles/svc-db-postgres/) | +| [LiteLLM Gateway](roles/svc-ai-litellm/) | beta | OpenAI-compatible LLM gateway routing a single /v1 endpoint to local and hosted model backends with per-consumer keys | [🌐](https://docs.litellm.ai/) [🛠️](roles/svc-ai-litellm/README.md#quick-setup) | [LM Studio](roles/svc-ai-lmstudio/), [Ollama](roles/svc-ai-ollama/), [PostgreSQL](roles/svc-db-postgres/), [System One](roles/svc-ai-s1/) | | [LittleJS](roles/web-app-littlejs/) | alpha | Self-hosted HTML5 game sandbox for browsing and running lightweight engine demos and games in the browser | [🌐](https://killedbyapixel.github.io/LittleJS/) [🎬](https://youtu.be/chuBzGjv7Ms) [🛠️](roles/web-app-littlejs/README.md#quick-setup) | [Content Delivery Network](roles/web-svc-cdn/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Matomo](roles/web-app-matomo/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/) | | [LM Studio](roles/svc-ai-lmstudio/) | beta | Headless local model server for open LLMs with an OpenAI-compatible API on the container network | [🌐](https://lmstudio.ai/) [🛠️](roles/svc-ai-lmstudio/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/) | | [Magento](roles/web-app-magento/) | alpha | Self-hosted e-commerce platform for catalog management, checkout, promotions, and multi-store online retail | [🌐](https://business.adobe.com/products/commerce/magento/open-source.html) [🎬](https://www.youtube.com/watch?v=nFdz9DZrdFc) [🛠️](roles/web-app-magento/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [Matomo](roles/web-app-matomo/), [Prometheus](roles/web-app-prometheus/), [Redis](roles/svc-db-redis/), [SeaweedFS](roles/web-app-seaweedfs/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | @@ -143,7 +149,7 @@ Every invokable role in the tested lifecycle envelope, with its upstream homepag | [Odoo ERP](roles/web-app-odoo/) | beta | Open-source ERP suite for CRM, sales, accounting, inventory, HR, and website management | [🌐](https://www.odoo.com/) [🎬](https://www.youtube.com/watch?v=FCmFgHVzd3E) [🛠️](roles/web-app-odoo/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [Matomo](roles/web-app-matomo/), [OpenLDAP](roles/svc-db-openldap/), [PostgreSQL](roles/svc-db-postgres/), [Prometheus](roles/web-app-prometheus/), [Redis](roles/svc-db-redis/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | | [Ollama](roles/svc-ai-ollama/) | beta | Self-hosted model server for running open LLMs locally behind a simple HTTP API | [🌐](https://ollama.com/) [🛠️](roles/svc-ai-ollama/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/) | | [OnlyOffice](roles/web-svc-onlyoffice/) | beta | Document server backend for real-time in-browser editing of documents, spreadsheets, and presentations | [🌐](https://www.onlyoffice.com/) [🎬](https://www.youtube.com/watch?v=uiFxnRK_aWM) [🛠️](roles/web-svc-onlyoffice/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Matomo](roles/web-app-matomo/), [PostgreSQL](roles/svc-db-postgres/), [Prometheus](roles/web-app-prometheus/), [Redis](roles/svc-db-redis/), [Tor Onion Service](roles/svc-net-tor/) | -| [Open WebUI](roles/web-app-openwebui/) | beta | Self-hosted chat interface for interacting with local and private AI models via backends such as Ollama | [🌐](https://openwebui.com/) [🛠️](roles/web-app-openwebui/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [LiteLLM Gateway](roles/svc-ai-litellm/), [Mailu](roles/web-app-mailu/), [Matomo](roles/web-app-matomo/), [OpenLDAP](roles/svc-db-openldap/), [Prometheus](roles/web-app-prometheus/), [SeaweedFS](roles/web-app-seaweedfs/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | +| [Open WebUI](roles/web-app-openwebui/) | beta | Self-hosted chat interface for interacting with local and private AI models via backends such as Ollama | [🌐](https://openwebui.com/) [🛠️](roles/web-app-openwebui/README.md#quick-setup) | [Agent Broker](roles/svc-ai-agent-broker/), [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [LiteLLM Gateway](roles/svc-ai-litellm/), [Mailu](roles/web-app-mailu/), [Matomo](roles/web-app-matomo/), [OpenLDAP](roles/svc-db-openldap/), [Prometheus](roles/web-app-prometheus/), [Qdrant](roles/svc-db-qdrant/), [SeaweedFS](roles/web-app-seaweedfs/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | | [OpenBao](roles/web-app-openbao/) | beta | Identity-based secrets, PKI and encryption manager with OIDC login, group-mapped policies and static-seal auto-unseal | [🌐](https://openbao.org/) [🎬](https://www.youtube.com/watch?v=hPBT-kSS-68) [🛠️](roles/web-app-openbao/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Matomo](roles/web-app-matomo/), [OpenLDAP](roles/svc-db-openldap/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | | [OpenClaw](roles/web-app-openclaw/) | beta | Self-hosted personal AI agent gateway with a browser control UI, sandboxed in a Kata microVM or gVisor runtime | [🌐](https://openclaw.ai/) [🛠️](roles/web-app-openclaw/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Kata & gVisor](roles/svc-virt-kata/), [Keycloak](roles/web-app-keycloak/), [LiteLLM Gateway](roles/svc-ai-litellm/), [Matomo](roles/web-app-matomo/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | | [OpenCloud](roles/web-app-opencloud/) | beta | Self-hosted platform for file sync, sharing, and collaboration with Keycloak SSO and central LDAP directory | [🌐](https://opencloud.eu/) [🎬](https://www.youtube.com/watch?v=Cg48rFNY0R4) [🛠️](roles/web-app-opencloud/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [Matomo](roles/web-app-matomo/), [OpenLDAP](roles/svc-db-openldap/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | @@ -156,6 +162,7 @@ Every invokable role in the tested lifecycle envelope, with its upstream homepag | [Penpot](roles/web-app-penpot/) | beta | Self-hosted design and prototyping platform for UI/UX work, real-time collaboration, and developer handoff | [🌐](https://penpot.app/) [🎬](https://www.youtube.com/watch?v=vAkb4QQmbfk) [🛠️](roles/web-app-penpot/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [Matomo](roles/web-app-matomo/), [OpenLDAP](roles/svc-db-openldap/), [PostgreSQL](roles/svc-db-postgres/), [Prometheus](roles/web-app-prometheus/), [Redis](roles/svc-db-redis/), [SeaweedFS](roles/web-app-seaweedfs/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | | [pgAdmin](roles/web-app-pgadmin/) | beta | Web-based administration and development platform for managing multiple PostgreSQL servers, with optional OAuth2 login | [🌐](https://www.pgadmin.org/) [🎬](https://www.youtube.com/watch?v=WFT5MaZN6g4) [🛠️](roles/web-app-pgadmin/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [Matomo](roles/web-app-matomo/), [PostgreSQL](roles/svc-db-postgres/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | | [PhpMyAdmin](roles/web-app-phpmyadmin/) | beta | Web-based administration tool for MySQL and MariaDB with schema management, querying, and import/export | [🌐](https://www.phpmyadmin.net/) [🛠️](roles/web-app-phpmyadmin/README.md#quick-setup) | [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [MariaDB](roles/svc-db-mariadb/), [Matomo](roles/web-app-matomo/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | +| [Pi Coding Agent](roles/dsk-gnt-pi/) | alpha | Minimal terminal coding agent with a lazy-loading skill system, wired to the platform's own model gateway | [🌐](https://github.com/earendil-works/pi) [🛠️](roles/dsk-gnt-pi/README.md#quick-setup) | [LiteLLM Gateway](roles/svc-ai-litellm/) | | [Pixelfed](roles/web-app-pixelfed/) | beta | Decentralized, federated platform for photo sharing and image feeds across the fediverse | [🌐](https://pixelfed.org/) [🛠️](roles/web-app-pixelfed/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [MariaDB](roles/svc-db-mariadb/), [Matomo](roles/web-app-matomo/), [Prometheus](roles/web-app-prometheus/), [Redis](roles/svc-db-redis/), [SeaweedFS](roles/web-app-seaweedfs/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | | [PostgreSQL](roles/svc-db-postgres/) | beta | Containerized relational database with automated database, user, and privilege provisioning | [🌐](https://www.postgresql.org/) [🛠️](roles/svc-db-postgres/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Tor Onion Service](roles/svc-net-tor/) | | [Postmarks](roles/web-app-postmarks/) | beta | Single-user ActivityPub bookmarking site federating public bookmarks to Mastodon and the Fediverse | [🌐](https://github.com/ckolderup/postmarks) [🛠️](roles/web-app-postmarks/README.md#quick-setup) | [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Matomo](roles/web-app-matomo/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | @@ -167,6 +174,7 @@ Every invokable role in the tested lifecycle envelope, with its upstream homepag | [Redirector](roles/web-opt-rdr-domains/) | beta | NGINX glue role for 301 redirects from source to target domains with automatic SSL for the source hosts | [🛠️](roles/web-opt-rdr-domains/README.md#quick-setup) | | | [Redis](roles/svc-db-redis/) | beta | In-memory key-value cache and store, shared centrally with per-consumer ACL isolation or embedded as a sidecar | [🌐](https://redis.io/) [🛠️](roles/svc-db-redis/README.md#quick-setup) | [Tor Onion Service](roles/svc-net-tor/) | | [Roulette Wheel](roles/web-app-roulette-wheel/) | alpha | Browser-based spinning wheel app for decision making, random selection, and gamification | [🌐](https://github.com/p-wojt/roulette-wheel) [🛠️](roles/web-app-roulette-wheel/README.md#quick-setup) | [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Matomo](roles/web-app-matomo/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/) | +| [Searxng](roles/svc-ai-searxng/) | alpha | Metasearch engine the platform queries for web results without handing a user's query to a tracking search provider | [🌐](https://docs.searxng.org/) [🛠️](roles/svc-ai-searxng/README.md#quick-setup) | | | [SeaweedFS](roles/web-app-seaweedfs/) | beta | Distributed S3-compatible object store for shared, per-consumer bucket provisioning across web-app roles | [🌐](https://seaweedfs.org/) [🛠️](roles/web-app-seaweedfs/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Prometheus](roles/web-app-prometheus/), [Universal Logout](roles/web-svc-logout/) | | [Self-Hosted Runner](roles/svc-runner/) | beta | Self-hosted CI runner infrastructure for executing GitHub Actions workflow jobs in ephemeral Docker containers | [🌐](https://github.com/actions/runner) [🛠️](roles/svc-runner/README.md#quick-setup) | | | [Semaphore UI](roles/web-app-semaphore/) | beta | Web UI and API for running Ansible, Terraform/OpenTofu, and shell automation on a schedule or on demand | [🌐](https://semaphoreui.com/) [🎬](https://www.youtube.com/watch?v=nzR4yDuZQMU) [🛠️](roles/web-app-semaphore/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [Matomo](roles/web-app-matomo/), [OpenLDAP](roles/svc-db-openldap/), [PostgreSQL](roles/svc-db-postgres/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | @@ -177,7 +185,9 @@ Every invokable role in the tested lifecycle envelope, with its upstream homepag | [Storage Optimizer](roles/svc-opt-ssd-hdd/) | beta | Tiering role placing Docker volumes on SSD or HDD by workload type, with symlinks keeping paths consistent | [🛠️](roles/svc-opt-ssd-hdd/README.md#quick-setup) | | | [SuiteCRM](roles/web-app-suitecrm/) | beta | Open-source CRM platform for sales, service, cases, and campaign management with workflow automation | [🌐](https://suitecrm.com/) [🎬](https://www.youtube.com/watch?v=pOAs_C3Fp_E) [🛠️](roles/web-app-suitecrm/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [MariaDB](roles/svc-db-mariadb/), [Matomo](roles/web-app-matomo/), [OpenLDAP](roles/svc-db-openldap/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | | [Swapfile](roles/svc-opt-swapfile/) | beta | On-target automation for creating and activating swap space via a cloned swapfile script | [🛠️](roles/svc-opt-swapfile/README.md#quick-setup) | | +| [System One](roles/svc-ai-s1/) | beta | Self-hosted System One classifier that answers typed questions so the gateway can route without generating | [🌐](https://huggingface.co/convaiinnovations/laya) [🎬](https://www.youtube.com/watch?v=UoRNo5LoDRE) [🛠️](roles/svc-ai-s1/README.md#quick-setup) | | | [Taiga](roles/web-app-taiga/) | beta | Open-source agile project management platform for Scrum, Kanban, and hybrid team workflows | [🌐](https://taiga.io/) [🎬](https://www.youtube.com/watch?v=TYPX8nQEU5I) [🛠️](roles/web-app-taiga/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [Matomo](roles/web-app-matomo/), [OpenLDAP](roles/svc-db-openldap/), [PostgreSQL](roles/svc-db-postgres/), [Prometheus](roles/web-app-prometheus/), [SeaweedFS](roles/web-app-seaweedfs/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) | +| [Tika](roles/svc-ai-tika/) | alpha | Central document extraction engine that turns uploaded files into text for retrieval-augmented chat | [🌐](https://tika.apache.org/) [🛠️](roles/svc-ai-tika/README.md#quick-setup) | | | [Tor Onion Service](roles/svc-net-tor/) | beta | Tor daemon publishing the node as a v3 onion service | [🛠️](roles/svc-net-tor/README.md#quick-setup) | [Asset Mirror (Privacy Proxy)](roles/web-svc-mirror/) | | [Typesense](roles/svc-db-typesense/) | beta | Central Dockerized search engine shared by multiple apps with per-consumer scoped API keys | [🌐](https://typesense.org/) [🛠️](roles/svc-db-typesense/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Tor Onion Service](roles/svc-net-tor/) | | [Unbound](roles/svc-dns-unbound/) | beta | Central recursive/forwarding DNSSEC resolver on a static IP for consumer stacks to target as their DNS | [🌐](https://nlnetlabs.nl/projects/unbound/about/) [🛠️](roles/svc-dns-unbound/README.md#quick-setup) | | diff --git a/cli/administration/deploy/ci/timings.py b/cli/administration/deploy/ci/timings.py new file mode 100644 index 0000000000..45b559b43d --- /dev/null +++ b/cli/administration/deploy/ci/timings.py @@ -0,0 +1,232 @@ +"""What each axis value costs in runner time, measured off finished runs. + +Usage: + python -m cli.administration.deploy.ci.timings [--runs N] [--repo owner/repo] + [--branch B] [--minimum N] + +A verification run wants the combination that reaches a verdict soonest, and +the axes differ by minutes: an onion row waits for circuits the clearnet row +never builds, and swarm brings a cluster up before the first service starts. +The combination that answers the question fastest is therefore not the one the +sweep rotation happens to draw. + +Nothing recorded that cost, so the choice used to be made by assertion. Deploy +job titles carry the whole row (:func:`utils.github.variant.axes.parse_label`) +and the API gives every job a start and an end, so the cost per axis value is +read back off the runs that already happened. + +Medians, not means: a job that hit a queue wait or a network stall is a long +tail on an otherwise cheap axis, and averaging lets one such job outvote a +hundred normal ones. + +A filesystem row states the kind the matrix *assigned*. A deploy may fall back +from it when the host kernel cannot serve it +(``scripts/tests/deploy/utils/filesystem/resolve.sh``) and the title does not +record that, so read that axis as a preference ranking and prove the kind +separately before pinning it. +""" + +from __future__ import annotations + +import argparse +import json +import statistics +import sys +from datetime import datetime +from typing import TYPE_CHECKING + +from cli.administration.deploy.ci.gh import ( + _gh, + current_branch, + fetch_jobs, + resolve_repo, +) +from cli.administration.deploy.ci.runs import _effective +from utils.github.variant import axes + +if TYPE_CHECKING: + from collections.abc import Iterable, Mapping + +AXES: tuple[str, ...] = ("mode", "tor", "distro", "filesystem") + + +def _seconds(job: Mapping[str, object]) -> float | None: + """Wall-clock of one job, or None when it never reached both timestamps.""" + started = job.get("startedAt") or job.get("started_at") + completed = job.get("completedAt") or job.get("completed_at") + if not started or not completed: + return None + try: + begin = datetime.fromisoformat(str(started)) + end = datetime.fromisoformat(str(completed)) + except ValueError: + return None + span = (end - begin).total_seconds() + return span if span > 0 else None + + +def _values(label: axes.Label) -> dict[str, str]: + """The axis values one parsed job title carries.""" + return { + "mode": label.mode, + "tor": "tor" if label.tor else "clearnet", + "distro": label.distro, + "filesystem": label.filesystem, + } + + +def samples(jobs: Iterable[Mapping[str, object]]) -> dict[str, dict[str, list[float]]]: + """Durations of the deploy jobs, grouped by axis and by value. + + Only successful jobs count. A failed job stops early or runs into a + timeout, so its duration measures the failure rather than the axis, and + these axes are picked between to reach a verdict cheaply. + + Args: + jobs: jobs of one or more runs, as the GitHub API returns them. + + Returns: + ``{axis: {value: [seconds, ...]}}``; values that never appeared are + left out entirely. + """ + out: dict[str, dict[str, list[float]]] = {axis: {} for axis in AXES} + for job in jobs: + if _effective(dict(job)) != "success": + continue + label = axes.parse_label(str(job.get("name", ""))) + if label is None: + continue + span = _seconds(job) + if span is None: + continue + for axis, value in _values(label).items(): + if value: + out[axis].setdefault(value, []).append(span) + return out + + +def ranking( + grouped: Mapping[str, Mapping[str, list[float]]], axis: str, *, minimum: int = 3 +) -> list[tuple[str, float, int]]: + """One axis' values from cheapest to dearest. + + Args: + grouped: :func:`samples` output. + axis: one of :data:`AXES`. + minimum: how many jobs a value needs before it is ranked at all. A + value seen once is a single observation, and steering a whole run + by it is how one queue wait becomes a policy. + + Returns: + ``[(value, median_seconds, sample_count), ...]``, cheapest first. + """ + rows = [ + (value, statistics.median(spans), len(spans)) + for value, spans in grouped.get(axis, {}).items() + if len(spans) >= minimum + ] + return sorted(rows, key=lambda row: row[1]) + + +def fastest( + grouped: Mapping[str, Mapping[str, list[float]]], + axis: str, + *, + among: Iterable[str] = (), + minimum: int = 3, +) -> str | None: + """The cheapest value of *axis*, or None when nothing is measured enough. + + Args: + grouped: :func:`samples` output. + axis: one of :data:`AXES`. + among: restrict the choice to these values; empty allows all of them. + minimum: as in :func:`ranking`. + + Returns: + The value name, or None when no allowed value cleared *minimum*. None + is a real answer: the measurement cannot carry a pin, and the caller + leaves that axis to the rotation instead of guessing it. + """ + allowed = set(among) + for value, _median, _count in ranking(grouped, axis, minimum=minimum): + if not allowed or value in allowed: + return value + return None + + +def collect(*, runs_back: int, repo: str, branch: str) -> list[dict]: + """Jobs of the last *runs_back* finished runs of *branch*. + + Args: + runs_back: how many finished runs to read. + repo: ``owner/repo`` the branch lives on. + branch: branch whose runs are read. + """ + listed = json.loads( + _gh( + [ + "run", + "list", + "--branch", + branch, + "--status", + "completed", + "-L", + str(runs_back), + "--json", + "databaseId", + ], + repo=repo, + ) + or "[]" + ) + jobs: list[dict] = [] + for entry in listed: + jobs += fetch_jobs(str(entry["databaseId"]), repo=repo) + return jobs + + +def _format(seconds: float) -> str: + return f"{int(seconds) // 60}m {int(seconds) % 60:02d}s" + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser( + prog="infinito administration deploy ci timings", + description=( + "Median wall-clock per axis value, read off the deploy jobs of " + "recent finished runs." + ), + ) + parser.add_argument("--runs", type=int, default=5) + parser.add_argument("--repo", default=None) + parser.add_argument("--branch", default=None) + parser.add_argument("--minimum", type=int, default=3) + args = parser.parse_args(argv) + + repo = args.repo or resolve_repo() + branch = args.branch or current_branch() + grouped = samples(collect(runs_back=args.runs, repo=repo, branch=branch)) + + if not any(grouped[axis] for axis in AXES): + print( + f"No finished deploy job of {branch} in the last {args.runs} run(s) " + "carries both timestamps; nothing to rank.", + file=sys.stderr, + ) + return 1 + + for axis in AXES: + rows = ranking(grouped, axis, minimum=args.minimum) + print(f"\n{axis}:") + if not rows: + print(f" no value reached {args.minimum} job(s)") + continue + for value, median, count in rows: + print(f" {value:<12} {_format(median):>8} {count} job(s)") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/cli/administration/deploy/ci/trigger/__main__.py b/cli/administration/deploy/ci/trigger/__main__.py index 57fd6a8e76..779f0e90e5 100644 --- a/cli/administration/deploy/ci/trigger/__main__.py +++ b/cli/administration/deploy/ci/trigger/__main__.py @@ -19,7 +19,7 @@ import argparse import sys from cli.administration.deploy.ci import gh, runs, selections -from cli.meta.ci import matrix, query +from cli.meta.ci import matrix, query, validate from utils.github import run_name from utils.github.variant import pools, selection, tor @@ -55,6 +55,53 @@ def _ranking(whitelist: str, config: dict[str, str]) -> list[dict[str, str]]: ) +def _refused(whitelist: str, priority: str, config: dict[str, str]) -> bool: + """Whether the selection cannot deploy on this branch, reported in full. + + ``call-orchestrator.yml`` already runs the same check as its first job and + every other job needs it, so a bad token never reaches a chunk. What it + does reach is the branch's concurrency group, which + ``entry-manual-steer.yml`` declares with ``cancel-in-progress``: dispatching + a run that cannot survive validation first **cancels the run currently in + flight** on that branch. That is the cost this refusal exists to avoid, not + the deploy minutes. + + Warnings are printed and do not refuse: a whitelist legitimately names + roles the lifecycle envelope filters out. + + Args: + whitelist: the whitelist input, empty for the diff or the all-sentinel. + priority: the priority line as dispatched. + config: the dispatch inputs the axes are resolved from. + """ + errors: list[str] = [] + warnings: list[str] = [] + for label, tokens in (("whitelist", whitelist), ("priority", priority)): + found, warned = validate.problems( + tokens, + modes=query.resolve_modes(config.get("mode") or query.ALL_MODES), + tor_mode=tor.resolve_tor_mode(config.get("tor")), + distros=pools.resolve_distros(config.get("distros")), + filesystems=pools.resolve_filesystems(config.get("filesystem")), + lifecycles=config.get("lifecycles", ""), + label=label, + ) + errors += found + warnings += warned + for warning in warnings: + print(f"warning: {warning}") + for error in errors: + print(f"error: {error}", file=sys.stderr) + if errors: + print( + f"\n{len(errors)} unusable selection(s); not dispatching. The run " + "would cancel the one currently in flight on this branch and then " + "fail its own validation job.", + file=sys.stderr, + ) + return bool(errors) + + def main(argv: list[str] | None = None) -> int: p = argparse.ArgumentParser( prog="infinito administration deploy ci trigger", @@ -138,11 +185,62 @@ def main(argv: list[str] | None = None) -> int: "source run's value, else the workflow default." ), ) + p.add_argument( + "--priority", + default=None, + metavar='"role#0@compose+clearnet%debian ..."', + help=( + "Lead the run with these selection tokens. Unlike --failed, which " + "derives them from a run's verdicts, this states them: a " + "verification run names the combination it wants to see, axis by " + "axis. Composes with --apps; --failed builds its own line." + ), + ) + p.add_argument( + "--workspace", + choices=("auto", "true", "false"), + default=None, + help=( + "Override the carried workspace input. 'false' leaves the " + "workspace test out, which a run that only has to reach a deploy " + "verdict does not need." + ), + ) + p.add_argument( + "--mode", + choices=("auto", "host", "compose", "swarm"), + default=None, + help="Override the carried deploy mode pool.", + ) + p.add_argument( + "--tor", + choices=("auto", "enforced", "exclusive", "disabled"), + default=None, + help="Override the carried tor axis.", + ) + p.add_argument( + "--distros", + default=None, + metavar='"debian arch"', + help="Override the carried distro pool.", + ) + p.add_argument( + "--filesystem", + default=None, + metavar='"ext4 btrfs"', + help=( + "Override the carried filesystem pool. A pool of exactly one is a " + "human naming the kind, so a host whose kernel cannot serve it " + "fails the row instead of substituting one." + ), + ) args = p.parse_args(argv) if args.strict and args.failed is None: p.error("--strict only applies with --failed") if args.roles_only and args.failed is None: p.error("--roles-only only applies with --failed") + if args.priority is not None and args.failed is not None: + p.error("--priority and --failed both build the priority line") branch = gh.current_branch() repo = gh.resolve_repo() @@ -198,8 +296,22 @@ def main(argv: list[str] | None = None) -> int: carried_whitelist = config.pop("whitelist", "") if not whitelist and carried_whitelist: whitelist = carried_whitelist - if args.chunk_gate is not None: - config["chunk_gate"] = args.chunk_gate + for name, value in ( + ("chunk_gate", args.chunk_gate), + ("workspace", args.workspace), + ("mode", args.mode), + ("tor", args.tor), + ("distros", args.distros), + ("filesystem", args.filesystem), + ): + if value is not None: + config[name] = value + + if args.priority is not None: + priority_entries = set(args.priority.split()) + if not priority_entries: + p.error("--priority was empty") + priority = " ".join(sorted(priority_entries)) if args.failed is not None: ranking = _ranking(whitelist, config) @@ -215,6 +327,9 @@ def main(argv: list[str] | None = None) -> int: else: config.pop("offset") + if _refused(whitelist if whitelist != _ALL else "", priority, config): + return 1 + if priority: label = f"priority {priority}, then the remaining roles" elif whitelist == _ALL: diff --git a/cli/build/docs/readme/generate.py b/cli/build/docs/readme/generate.py index 2299dbaee0..901f05ad66 100644 --- a/cli/build/docs/readme/generate.py +++ b/cli/build/docs/readme/generate.py @@ -32,6 +32,7 @@ from utils.roles.credits import author_urls from utils.roles.deploy import role_has_stack from utils.roles.entity.name import get_entity_name from utils.roles.mapping import ROLE_FILE_META_MAIN, ROLE_FILE_README +from utils.roles.meta_lookup import get_role_guide_companions from utils.roles.validation.invokable import _get_invokable_paths, _is_role_invokable from utils.software import SOFTWARE_REPOSITORY from utils.symbol_glossary import to_emoji @@ -99,6 +100,9 @@ def _base_context(role_dir, role_name: str, app_name: str, *, invokable: bool) - "application_author_url": author_urls().get(_role_author(role_dir)), "application_is_host": not role_has_stack(role_dir), "application_needs_clearnet_resolver": _needs_clearnet_resolver(role_dir), + "application_deploy_ids": ",".join( + [role_name, *get_role_guide_companions(role_dir, role_name=role_name)] + ), "software_repository": SOFTWARE_REPOSITORY, "cosmos_mermaid": derive_cosmos_mermaid(role_dir, role_name), "cosmos_legend": _cosmos_legend(), diff --git a/cli/meta/roles/applications/complexity/model.py b/cli/meta/roles/applications/complexity/model.py index 5ce71b6783..f7d05c27f7 100644 --- a/cli/meta/roles/applications/complexity/model.py +++ b/cli/meta/roles/applications/complexity/model.py @@ -9,10 +9,10 @@ from typing import TYPE_CHECKING, Any, NamedTuple from utils.cache.applications import get_variants from utils.env.parser import env_setting +from utils.roles.deploy import role_deploy_modes from utils.roles.lifecycle import tested_lifecycles from utils.roles.meta_lookup import ( MetaServicesShapeError, - get_role_mode_enabled, get_role_test_skips, ) from utils.roles.validation.invokable import list_invokables_by_type @@ -291,13 +291,10 @@ def compute_complexity_rows( rows = [] for name in names: stack = role_has_stack(roles_dir / name) - compose = name in compose_apps - swarm = name in swarm_apps and stack - host = ( - name in host_apps - and not stack - and get_role_mode_enabled(roles_dir / name, mode="host", role_name=name) - ) + offered = role_deploy_modes(roles_dir / name, role_name=name) + compose = name in compose_apps and offered.get("compose", False) + swarm = name in swarm_apps and offered.get("swarm", False) + host = name in host_apps and offered.get("host", False) skips = _role_test_skips(roles_dir, name) rows.append( _build_row(name, forward, reverse, max_level)._replace( @@ -354,14 +351,11 @@ def compute_variant_complexity_rows( continue name = role_dir.name lifecycle = _role_lifecycle(variants.get(name)) - compose = name in compose_apps - swarm_role = name in swarm_apps stack = role_has_stack(role_dir) - host = ( - name in host_apps - and not stack - and get_role_mode_enabled(role_dir, mode="host", role_name=name) - ) + offered = role_deploy_modes(role_dir, role_name=name) + compose = name in compose_apps and offered.get("compose", False) + swarm_role = name in swarm_apps and offered.get("swarm", False) + host = name in host_apps and offered.get("host", False) skips = _role_test_skips(roles_dir, name) for index, variant_config in enumerate(variants.get(name) or []): providers = direct_dep_roles( @@ -375,11 +369,11 @@ def compute_variant_complexity_rows( )._replace( lifecycle=lifecycle, compose=compose, - swarm=swarm_role and stack, + swarm=swarm_role, stack=stack, host=host, test_compose=compose and "compose" not in skips, - test_swarm=swarm_role and stack and "swarm" not in skips, + test_swarm=swarm_role and "swarm" not in skips, test_host=host and "host" not in skips, ) ) diff --git a/docs/agents/action/README.md b/docs/agents/action/README.md index ae48b3f513..1e0de1cbf3 100644 --- a/docs/agents/action/README.md +++ b/docs/agents/action/README.md @@ -2,3 +2,5 @@ This directory contains action-oriented runbooks for agent work. Its scope is how agents should develop, debug, test, refactor, commit, and handle pull requests while staying aligned with repository policy. + +Unlike the rest of `docs/agents/`, these files are read on demand rather than at session start: an `i8-` skill routes to the runbook of the action it covers, and an agent performing one of these actions without a skill reads that runbook first. They are the bulk of the agent documentation and each one is inert until its action is the task. diff --git a/docs/agents/action/ci-trigger-priority.md b/docs/agents/action/ci-trigger-priority.md new file mode 100644 index 0000000000..ff0691d52e --- /dev/null +++ b/docs/agents/action/ci-trigger-priority.md @@ -0,0 +1,169 @@ +# Lead a CI run with a priority line 🔎 + +Turn the questions a branch still has open into the priority line of one manual +CI run, so it answers as many of them as possible in the cheapest combination +that still counts as an answer. + +The run this builds differs from a retrigger: a retrigger replays what a run +already judged, this one asks about rows nobody has judged yet or whose verdict +is still disputed. It therefore states its axes instead of carrying them over. + +## Scope + +- At most 20 selections on the priority line. The bound is the point: a + priority line long enough to fill the whole queue window stops being a + priority. +- `workspace=false` and `chunk_gate=false`. +- Every selection names every axis: role, variant, mode, onion state, distro + and filesystem. Nothing is left to the rotation. + +## 1. Collect the candidates + +Two sources, run verdicts first: + +1. The selections of the last finished deploy run on the branch that are not + green. `cli.administration.deploy.ci.selections.failed_selections` reads + them straight out of the job titles, so each candidate arrives with the + variant, mode and onion state it failed in. +2. The root causes that are still open without a run to point at: a fix landed + whose effect nobody has seen, or a diagnosis that was never confirmed + against the deployed code. + +Take source 1 in full, then fill up to 20 from source 2. A row that source 1 +already covers is not added twice. + +Stop and say so when the two sources together yield fewer than three +candidates. A run that verifies two rows is cheaper to ask for by name. + +## 2. Measure the axes before choosing them + +```bash +python -m cli.administration.deploy.ci.timings --runs 6 +``` + +It reports the median wall-clock per axis value off the finished runs of the +branch, with the sample count behind each one. Measured on this repository, the +spread is not marginal: `swarm` costs roughly twice what `compose` costs, and +an onion row roughly a third more than a clearnet one. + +**Every token pins every axis.** `role#variant@mode+tor|clearnet%distro/filesystem` +carries all six, and none is left to the rotation. A run whose rows the +rotation is still free to move is not a verification run: the combination that +comes back is not the one the question was asked about, and a green row then +proves nothing about the red one it was supposed to answer. + +The measurement decides **which** value each axis takes, never **whether** it +takes one: + +| Axis | Pin the cheap value when | Pin the expensive value when | +|---|---|---| +| `mode` | the failure is not mode-specific | the row failed only in one mode, or the question is about swarm placement, overlay networking or replicas | +| `tor` | the onion is not involved | the failure mentions onion addresses, circuits or the tor proxy | +| `distro` | the failure is not package-manager specific | the question names a distro, or the row failed on exactly one | +| `filesystem` | the kind is deliverable, see the next step | the question is about the storage driver | + +When `timings.fastest` returns `None`, the measurement has too few samples to +choose. That is not permission to leave the axis open; it moves the choice one +rung down: + +1. the value the candidate's own failing row carried, read off the source run's + job title. It reproduces the failure, which is the point of the line. +2. failing that, the only value the measurement saw at all, even below the + sample minimum. +3. failing that, the run's pool default: `compose`, `clearnet`, `debian`, + `btrfs`. + +Say in the report which rung each pin came from. A pin from rung 3 is a guess +wearing a token, and the reader has to know that. + +## 3. The filesystem is the one axis with a host condition + +`scripts/tests/deploy/utils/filesystem/resolve.sh` serves a kind only when the +**host kernel** carries it: `zfs` needs `/dev/zfs` or a `modinfo zfs` hit, the +others need an entry in `/proc/filesystems` or a module. The distro images carry +only the userland (`scripts/install/filesystem.sh`). + +A pool or token of exactly one kind is read as a human naming it, so the +applying step **fails the row instead of substituting** a kind the host can +serve. Pinning a kind the runner cannot deliver therefore produces a red job +that says nothing about the question. + +The axis is still pinned; the condition decides **which kind**. Pick the kind +that satisfies both: + +1. the measurement has samples of it, and +2. no fallback was reported for it. One deploy job log that was assigned the + kind settles it: a delivered kind logs + `docker-dataroot-filesystem: status=applied requested= effective=`, + a substituted one logs a `fallback out of` note instead. Use the + `ci-artifacts` skill to pull that one log rather than every log of the run. + +No kind satisfies both, because the branch has no samples yet: pin `btrfs` and +say in the report that the pin rests on the pool default rather than on +evidence. Never drop the axis to avoid the question. + +## 4. Validate, then dispatch + +The trigger validates on its own before it dispatches, and refuses the whole +run when a token cannot deploy on this branch. That refusal is not about deploy +minutes: `call-orchestrator.yml` runs the same check as its first job and every +other job needs it, so a bad token never reaches a chunk. It reaches the +branch's concurrency group, which `entry-manual-steer.yml` declares with +`cancel-in-progress`, so the doomed dispatch cancels the run currently in flight +before failing its own validation. + +Check it first anyway when building the line by hand, because the same check +outside the trigger costs a second and names every problem at once: + +```bash +python -m cli.meta.ci.validate --priority "" --modes auto --tor auto +``` + +`--modes auto --tor auto` at run level, because the tokens pin those axes per +row: a run-level pool of one would refuse every token that names the other +value, and a line that verifies a swarm question and a compose question at once +needs both open at run level and closed per token. + +Then dispatch: + +```bash +python -m cli.administration.deploy.ci.trigger \ + --priority "" \ + --workspace false \ + --chunk-gate false +``` + +No `--mode`, `--tor`, `--distros` or `--filesystem`. Those inputs pin an axis +for the whole run, which the tokens already do per row, and a run-level pool +that disagrees with a token refuses it. The only reason to reach for them is a +line where every row genuinely shares one value, and even then the token form +is what the report has to show. + +Add `--mode`, `--tor`, `--distros` or `--filesystem` only for an axis that step +2 settled for the whole run rather than per token. + +A refusal names the token and why. Fix the token, never the check: a variant +that no longer exists means the list was carried over from an older run, and +dropping the pin only moves the wrong row into the queue. + +A warning does not refuse, and it carries two very different meanings. Tell +them apart before ignoring it: + +- **the role id is wrong.** Candidates assembled from memory or from a + conversation get prefixes wrong: `web-svc-openresty` warns, `svc-prx-openresty` + resolves. Check `roles/` for the id before dropping the candidate. +- **the role is outside the envelope.** `web-app-minio` warns whatever is + pinned, because its `lifecycle` is `eol`. That candidate does not belong on a + verification line at all; no run will judge it. + +## 5. Report what the run can and cannot answer + +Every axis is pinned, so the report is not about what was left open. It is +about what each pin rests on: the measurement, the row's own failing +combination, or the pool default. A pin from the pool default is a guess, and a +green row under it answers a question nobody asked. + +Name every question the run will **not** answer because a candidate was pinned +to one side of it. A line that verifies a fix in compose and says nothing about +swarm is a legitimate choice; a line that does that silently is not, because its +green result reads like a full answer. diff --git a/docs/agents/action/iteration/compose.md b/docs/agents/action/iteration/compose.md index fd735dc721..120331244e 100644 --- a/docs/agents/action/iteration/compose.md +++ b/docs/agents/action/iteration/compose.md @@ -22,6 +22,9 @@ For iterating on GitHub Actions workflows with Act, see [Workflow Loop](workflow - ON NO: omit the arg entirely. - SIDE EFFECT (yes): inventory initializer auto-removes the provider roles `web-app-matomo`, `web-app-dashboard`, `web-app-prometheus`, `web-app-mailu`, and `web-svc-css`. Do NOT list them in `apps=`. - PERSIST: record answer at top of iteration. Reuse for all subsequent deploys without re-asking. + - WHY IT BITES: `test-e2e-cli` runs in the destructor stage (`tasks/stages/03_destructor.yml`), i.e. after EVERY role of the play. A deploy that still carries the five providers therefore pays for matomo, dashboard and prometheus BEFORE the CLI test you are waiting for even starts. Dropping `disable=` does not only cost resources, it delays the evidence. +- Every deploy opens with a `=== compose-deploy:` line listing the arguments it actually resolved. Read it instead of trusting the command you typed: rewriting `apps=` is where `disable=` and `full_cycle=` get lost, and the line shows what is missing before the play starts. +- `full_cycle=true` is the expensive form, not the complete one: it adds pass 2, credential rotation and the recovery drill. It does not gate the CLI tests, which run on `RUNTIME` instead. - You MUST run `make test` before every deploy. Only proceed with the deploy if all tests pass. - You MUST prepend `INFINITO_PLAYWRIGHT_KEEP=true` to every `make compose-deploy` command in the iteration (any mode), so trace, screenshot, and video of passing Playwright tests stay inspectable. Omit only when the user has explicitly opted out of per-test artefacts. For the full propagation chain see [Playwright Tests](../../../contributing/actions/testing/playwright.md#artefact-retention-). - Unless the user explicitly says to reuse the existing setup, you MUST start once with `make compose-deploy mode=reinstall apps= full_cycle=true` to establish the baseline inventory and clean app state. `full_cycle=true` adds the async update pass (pass 2) and MUST stay on unless the user explicitly asks to skip it. diff --git a/docs/agents/action/verify-parallel.md b/docs/agents/action/verify-parallel.md new file mode 100644 index 0000000000..405c14e58d --- /dev/null +++ b/docs/agents/action/verify-parallel.md @@ -0,0 +1,85 @@ +# Verify in two lanes at once 🛤️ + +Answer the branch's open questions from both ends at the same time: a CI run +that judges every row on clean runners, and a local loop that reproduces the +same roles on the stack you can actually inspect. + +Neither lane alone closes a question. CI says whether a row comes up green on a +clean host but hands you a log hours later. The local stack answers in minutes +and lets you exec into the container, but it is one host with one slot and it +never proves the swarm path the way five distros do. + +## What runs in parallel, and what must not + +The two lanes run at the same time because they run on different machines. The +CI run is remote; the local loop is this host. + +Inside the local lane nothing is parallel. `iteration/compose.md` requires +deploys to be serial, and a second deploy or a `make test` started next to a +running one is the SIGKILL that costs the whole round. + +## 1. Dispatch the CI lane + +Invoke the `i8-ci-trigger-priority` skill. It builds the priority line and +dispatches it; this lane does not re-derive candidates or axes of its own, so +that the two skills cannot drift apart on what a verification row is. + +Keep two things from its report: the run URL, and the candidate table saying +what each pin rests on. The second one decides the local lane's scope. + +Stop here if it refuses to dispatch. A refused line means a token cannot deploy +on this branch, and the local lane alone answers nothing about CI. + +## 2. Arm the watch + +``` +/loop 30m /triage +``` + +30 minutes, because a deploy row takes 47 minutes in compose and 84 in swarm at +the median (`cli.administration.deploy.ci.timings`). A tick under fifteen +minutes mostly re-reads the same in-progress jobs; one over an hour lets a whole +chunk finish unnoticed. + +**Never dispatch again while the watch is armed.** `entry-manual-steer.yml` +declares the branch's concurrency group with `cancel-in-progress`, so a second +dispatch cancels the run being triaged. The loop then triages a cancelled run +and reports every unfinished row as a failure, which is a phantom: those rows +never reached a verdict. + +## 3. Run the local lane against what CI cannot settle for you + +Take the roles in this order, and stop when the stack is busy rather than +queueing them: + +1. **a fix that landed after the source run's verdict.** CI has never judged it; + the local stack can, today. +2. **a pin that came from the pool default.** The CI row will be green or red + for a combination nobody chose, so its verdict answers a different question + than the one asked. +3. **a role whose failure was about container state** rather than placement: + credentials, migrations, a healthcheck, a missing route. Those reproduce + locally in full. Placement, overlay networking and replica counts do not. + +Drive each one through [iteration/compose.md](iteration/compose.md) under the +`robot` skill. A role whose question is swarm-specific goes through +[iteration/swarm.md](iteration/swarm.md) instead; there is no point reproducing +an overlay problem in compose. + +## 4. Let the lanes correct each other + +The lanes disagree more often than they agree, and the disagreement is the +finding: + +- **local green, CI red**: the failure is in the axis the local stack does not + have. Read the CI artifact before touching the role. +- **local red, CI green**: usually local state the runner starts without, a + stale image or a volume from an earlier iteration. Confirm before reporting a + regression. +- **both red, different errors**: two causes, not one. Split them. + +## 5. Close each question explicitly + +A question is answered when its CI row is green **and** the local lane cleared +it, or when one lane was deliberately out of scope and the report says which. +Anything else is open, however green the run looks. diff --git a/docs/contributing/artefact/git/pull-request.md b/docs/contributing/artefact/git/pull-request.md index a848bcbf55..36db9bb546 100644 --- a/docs/contributing/artefact/git/pull-request.md +++ b/docs/contributing/artefact/git/pull-request.md @@ -7,7 +7,7 @@ You MUST pick the template that matches your change. The table below links the a | Change type | Template | |---|---| | Server and `web-*` changes | [server.md](../../../../.github/PULL_REQUEST_TEMPLATE/server.md) | -| Workstation and `desk-*` changes | [workstation.md](../../../../.github/PULL_REQUEST_TEMPLATE/workstation.md) | +| Workstation and `dsk-*` changes | [workstation.md](../../../../.github/PULL_REQUEST_TEMPLATE/workstation.md) | | Shared system changes such as `sys-*`, `svc-*`, `dev-*`, `drv-*`, `pkgmgr`, `update-*`, or `user-*` | [system.md](../../../../.github/PULL_REQUEST_TEMPLATE/system.md) | | CI/CD and workflow changes | [pipeline.md](../../../../.github/PULL_REQUEST_TEMPLATE/pipeline.md) | | Documentation-only changes limited to `.md` or `.rst` files outside `AGENTS.md` and `docs/agents/*` | [documentation.md](../../../../.github/PULL_REQUEST_TEMPLATE/documentation.md) | diff --git a/docs/contributing/design/architecture.md b/docs/contributing/design/architecture.md index f5c2567e83..ddb677689b 100644 --- a/docs/contributing/design/architecture.md +++ b/docs/contributing/design/architecture.md @@ -29,7 +29,7 @@ A role usually owns one capability: - `svc-*` for shared services such as databases, proxies, mail, DNS, VPN, backups, and identity support - `web-app-*` for end-user applications - `dev-*` for development tooling -- `desk-*` for workstation setup +- `dsk-*` for workstation setup - `web-svc-*` and `web-opt-*` for web-facing helpers and redirects For example, `web-app-nextcloud` owns the Nextcloud deployment, `sys-svc-webserver-https` owns the HTTPS plumbing, and `sys-ctl-cln-docker` handles Docker cleanup. diff --git a/docs/contributing/tools/agents/cheatsheet.md b/docs/contributing/tools/agents/cheatsheet.md index 9ebae273cd..0f71e7e572 100644 --- a/docs/contributing/tools/agents/cheatsheet.md +++ b/docs/contributing/tools/agents/cheatsheet.md @@ -36,5 +36,7 @@ from its procedure file. Run `make install-skills` to (re)install them. | Staged changes are ready to be committed | `i8-commit` | | A branch is ready to be opened as a pull request | `i8-pull-request` | | Pushing a branch through the manual-CI draft → ready-for-review cycle | `i8-push-trigger-pull` | +| Leading a CI run with a priority line that verifies the branch's open questions | `i8-ci-trigger-priority` | +| Verifying from both ends at once: the CI run on a triage loop plus local iteration | `i8-verify-parallel` | | Writing a new requirement | `i8-requirement-create` | | Implementing an existing requirement file end to end | `i8-requirement-implement` | diff --git a/docs/requirements/026-unified-addon-syntax.md b/docs/requirements/026-unified-addon-syntax.md index c340f137ce..3c647d63b6 100644 --- a/docs/requirements/026-unified-addon-syntax.md +++ b/docs/requirements/026-unified-addon-syntax.md @@ -15,7 +15,7 @@ The repository already deploys role-level extension units, but each role spells - [web-app-mediawiki](../../roles/web-app-mediawiki/) calls them `extensions` and hard-codes the list in [vars/main.yml](../../roles/web-app-mediawiki/vars/main.yml). - [web-app-xwiki](../../roles/web-app-xwiki/) calls them `plugins` (with nested `items[].id` + `version`) under `xwiki.plugins` in [meta/services.yml](../../roles/web-app-xwiki/meta/services.yml). - [web-app-joomla](../../roles/web-app-joomla/) builds a single OIDC `plugin` in [tasks/08_oidc_plugin.yml](../../roles/web-app-joomla/tasks/08_oidc_plugin.yml). -- [desk-gnome-extensions](../../roles/desk-gnome-extensions/) calls them `plugins`/`extensions` and loops over `services.gnome-extensions.plugins`. +- [dsk-gnome-extensions](../../roles/dsk-gnome-extensions/) calls them `plugins`/`extensions` and loops over `services.gnome-extensions.plugins`. The [per-role meta layout](../contributing/design/role/services/layout.md) already lists `addons`, `plugins`, and `modules` among the keys *inlined* under the primary service entity. This requirement promotes that concept to a first-class, per-file `meta/addons/.yml` topic (one file per addon, file root IS the addon spec), the same move requirement 011 made for `meta/info.yml` and requirement 008/009 made for other meta topics. @@ -61,10 +61,10 @@ Each row is a migration target. The "Bridges" column is the cross-role dependenc | [web-app-discourse](../../roles/web-app-discourse/) | `plugin` | `docker_manager, discourse-activity-pub, discourse-akismet, discourse-ldap-auth` | `discourse-ldap-auth` → `svc-db-openldap` | [meta/services.yml](../../roles/web-app-discourse/meta/services.yml) `discourse.plugins` | | [web-app-pretix](../../roles/web-app-pretix/) | `plugin` | `oidc` (v2.3.1) | `sso` → `web-app-keycloak` | [meta/services.yml](../../roles/web-app-pretix/meta/services.yml) `pretix.plugins` | | [web-app-mattermost](../../roles/web-app-mattermost/) | `plugin` | volume-managed plugins (`plugins`, `client-plugins`) | none declared today | [vars/main.yml](../../roles/web-app-mattermost/vars/main.yml), [meta/volumes.yml](../../roles/web-app-mattermost/meta/volumes.yml) | -| [desk-chromium](../../roles/desk-chromium/) | `extension` | CRX ids, `force_installed` (e.g. uBlock Origin) | none (desktop role) | [meta/services.yml](../../roles/desk-chromium/meta/services.yml) `plugins` | -| [desk-firefox](../../roles/desk-firefox/) | `extension` | XPI urls (uBlock Origin, KeePassXC) | none (desktop role) | [meta/services.yml](../../roles/desk-firefox/meta/services.yml) `plugins` | -| [desk-gnome](../../roles/desk-gnome/) | `extension` | gnome-shell extensions (enable/disable tuples) | none (desktop role) | [meta/services.yml](../../roles/desk-gnome/meta/services.yml) `plugins` | -| [desk-gnome-extensions](../../roles/desk-gnome-extensions/) | `extension` | configured list (`services.gnome-extensions.plugins`) | none (desktop role) | [meta/services.yml](../../roles/desk-gnome-extensions/meta/services.yml) | +| [dsk-chromium](../../roles/dsk-chromium/) | `extension` | CRX ids, `force_installed` (e.g. uBlock Origin) | none (desktop role) | [meta/services.yml](../../roles/dsk-chromium/meta/services.yml) `plugins` | +| [dsk-firefox](../../roles/dsk-firefox/) | `extension` | XPI urls (uBlock Origin, KeePassXC) | none (desktop role) | [meta/services.yml](../../roles/dsk-firefox/meta/services.yml) `plugins` | +| [dsk-gnome](../../roles/dsk-gnome/) | `extension` | gnome-shell extensions (enable/disable tuples) | none (desktop role) | [meta/services.yml](../../roles/dsk-gnome/meta/services.yml) `plugins` | +| [dsk-gnome-extensions](../../roles/dsk-gnome-extensions/) | `extension` | configured list (`services.gnome-extensions.plugins`) | none (desktop role) | [meta/services.yml](../../roles/dsk-gnome-extensions/meta/services.yml) | ### Out of scope: database/runtime engine extensions @@ -271,9 +271,9 @@ Rules: - [ ] [web-app-discourse](../../roles/web-app-discourse/) declares its plugins in `meta/addons/` with `mechanism: plugin`; `discourse-ldap-auth` carries `bridges: [ldap]` resolving to the `ldap` service block, while `docker_manager`, `discourse-activity-pub`, and `discourse-akismet` carry no `bridges`. - [ ] [web-app-pretix](../../roles/web-app-pretix/) declares its `oidc` plugin in `meta/addons/` with `mechanism: plugin`, the `2.3.1` pin preserved as a string `version`, and `bridges: [sso]`. - [ ] [web-app-mattermost](../../roles/web-app-mattermost/) declares its plugins in `meta/addons/` with `mechanism: plugin`; the plugin volumes remain in `meta/volumes.yml`. -- [x] [desk-chromium](../../roles/desk-chromium/) and [desk-firefox](../../roles/desk-firefox/) declare their browser extensions in `meta/addons/` with `mechanism: extension`, no `bridges`, and ship NO Playwright specs (desktop roles have no web surface; the lint forbids specs under `desk-*`). -- [x] [desk-gnome](../../roles/desk-gnome/) declares its gnome-shell extensions in `meta/addons/` with `mechanism: extension` and no `bridges`. -- [ ] [desk-gnome-extensions](../../roles/desk-gnome-extensions/) declares its extensions in `meta/addons/` with `mechanism: extension` and no `bridges`. +- [x] [dsk-chromium](../../roles/dsk-chromium/) and [dsk-firefox](../../roles/dsk-firefox/) declare their browser extensions in `meta/addons/` with `mechanism: extension`, no `bridges`, and ship NO Playwright specs (desktop roles have no web surface; the lint forbids specs under `dsk-*`). +- [x] [dsk-gnome](../../roles/dsk-gnome/) declares its gnome-shell extensions in `meta/addons/` with `mechanism: extension` and no `bridges`. +- [ ] [dsk-gnome-extensions](../../roles/dsk-gnome-extensions/) declares its extensions in `meta/addons/` with `mechanism: extension` and no `bridges`. - [x] [web-app-mobilizon](../../roles/web-app-mobilizon/) and [web-app-bookwyrm](../../roles/web-app-bookwyrm/) keep their `extensions:` (PostgreSQL/engine extensions) in `meta/services.yml`; they are classified `db-extension` and explicitly NOT migrated to `meta/addons/`. ### Cross-role integration (matrix backlog) diff --git a/docs/requirements/035-mcp-proxy-expansion.md b/docs/requirements/035-mcp-proxy-expansion.md index 621582e611..09ff86a825 100644 --- a/docs/requirements/035-mcp-proxy-expansion.md +++ b/docs/requirements/035-mcp-proxy-expansion.md @@ -480,15 +480,15 @@ An adapter could reach these, and none has been curated yet. The five that used `web-svc-collabora`, `web-svc-onlyoffice`. -### no_surface (96) +### no_surface (97) These roles remain out of shared MCP discovery. A future exception requires a new requirement with a fixed operation list, dedicated identity, isolation boundary, human approval for mutations, and audit trail. -**`host_execution_boundary` (39):** `desk-bluray-player`, `desk-chromium`, `desk-copyq`, `desk-docker`, `desk-dotlinker`, `desk-firefox`, `desk-git`, `desk-gnome`, `desk-gnome-caffeine`, `desk-gnome-extensions`, `desk-gnome-terminal`, `desk-gnucash`, `desk-jrnl`, `desk-keepassxc`, `desk-libreoffice`, `desk-micro`, `desk-neovim`, `desk-nextcloud`, `desk-obs`, `desk-qbittorrent`, `desk-retroarch`, `desk-spotify`, `desk-ssh`, `desk-torbrowser`, `desk-virtualbox`, `desk-zoom`, `dev-arduino`, `dev-core`, `dev-java`, `dev-locales`, `dev-make`, `dev-nix`, `dev-nodejs`, `dev-python`, `drv-epson-multiprinter`, `drv-intel`, `drv-lid-switch`, `drv-non-free`, `gen-hunspell`. +**`host_execution_boundary` (45):** `dev-arduino`, `dev-core`, `dev-java`, `dev-locales`, `dev-make`, `dev-nix`, `dev-nodejs`, `dev-python`, `drv-epson-multiprinter`, `drv-intel`, `drv-lid-switch`, `drv-non-free`, `dsk-bluray-player`, `dsk-chromium`, `dsk-code`, `dsk-copyq`, `dsk-docker`, `dsk-dotlinker`, `dsk-firefox`, `dsk-git`, `dsk-gnome`, `dsk-gnome-caffeine`, `dsk-gnome-extensions`, `dsk-gnome-terminal`, `dsk-gnt-claude`, `dsk-gnt-codex`, `dsk-gnt-cursor`, `dsk-gnt-pi`, `dsk-gnucash`, `dsk-jrnl`, `dsk-keepassxc`, `dsk-laya`, `dsk-libreoffice`, `dsk-micro`, `dsk-neovim`, `dsk-nextcloud`, `dsk-obs`, `dsk-qbittorrent`, `dsk-retroarch`, `dsk-spotify`, `dsk-ssh`, `dsk-torbrowser`, `dsk-virtualbox`, `dsk-zoom`, `gen-hunspell`. These roles operate a workstation, developer toolchain, device, or host package; bridging them would amount to shared shell, filesystem, browser-session, device, or host execution without an application-specific remote identity. -**`privileged_control_plane` (25):** `svc-bkp-local-2-device`, `svc-bkp-nfs-2-local`, `svc-bkp-remote-2-local`, `svc-bkp-secrets-2-local`, `svc-bkp-volume-2-local`, `svc-dns-unbound`, `svc-net-firewall`, `svc-net-tor`, `svc-net-wireguard-core`, `svc-net-wireguard-firewalled`, `svc-net-wireguard-plain`, `svc-opt-keyboard-color`, `svc-opt-ssd-hdd`, `svc-opt-swapfile`, `svc-prx-openresty`, `svc-registry-cache`, `svc-registry-docker`, `svc-runner`, `svc-storage-nfs-client`, `svc-storage-nfs-server`, `svc-swarm-manager`, `svc-swarm-node`, `svc-virt-kata`, `update`, `web-app-openbao`. +**`privileged_control_plane` (27):** `svc-ai-agent-broker`, `svc-ai-jupyter`, `svc-bkp-local-2-device`, `svc-bkp-nfs-2-local`, `svc-bkp-remote-2-local`, `svc-bkp-secrets-2-local`, `svc-bkp-volume-2-local`, `svc-dns-unbound`, `svc-net-firewall`, `svc-net-tor`, `svc-net-wireguard-core`, `svc-net-wireguard-firewalled`, `svc-net-wireguard-plain`, `svc-opt-keyboard-color`, `svc-opt-ssd-hdd`, `svc-opt-swapfile`, `svc-prx-openresty`, `svc-registry-cache`, `svc-registry-docker`, `svc-runner`, `svc-storage-nfs-client`, `svc-storage-nfs-server`, `svc-swarm-manager`, `svc-swarm-node`, `svc-virt-kata`, `update`, `web-app-openbao`. These roles can recover secrets, alter routing or host state, run code, change deployment state, or reach storage/control-plane sockets; they require a separate audited operations gateway and human approval rather than general-purpose application MCP. **`shared_engine_isolation` (4):** `svc-db-memcached`, `svc-db-openldap`, `svc-db-postgres`, `svc-db-rabbitmq`. @@ -501,7 +501,7 @@ These UIs expose identity or database administration rather than a bounded appli **`duplicate_owner` (1):** `web-app-litellm`. This role is only the UI of the separately classified LiteLLM service role, which owns any future MCP gateway contract. -**`no_remote_surface` (24):** `svc-ai-ollama`, `svc-ai-robot`, `web-app-chess`, `web-app-dashboard`, `web-app-docs`, `web-app-fediwall`, `web-app-hugo`, `web-app-littlejs`, `web-app-mig`, `web-app-mini-qr`, `web-app-navigator`, `web-app-roulette-wheel`, `web-opt-rdr-domains`, `web-opt-rdr-www`, `web-svc-asset`, `web-svc-cdn`, `web-svc-coturn`, `web-svc-css`, `web-svc-file`, `web-svc-html`, `web-svc-legal`, `web-svc-logout`, `web-svc-mirror`, `web-svc-simpleicons`. +**`no_remote_surface` (27):** `svc-ai-ollama`, `svc-ai-robot`, `svc-ai-s1`, `svc-ai-searxng`, `svc-ai-tika`, `web-app-chess`, `web-app-dashboard`, `web-app-docs`, `web-app-fediwall`, `web-app-hugo`, `web-app-littlejs`, `web-app-mig`, `web-app-mini-qr`, `web-app-navigator`, `web-app-roulette-wheel`, `web-opt-rdr-domains`, `web-opt-rdr-www`, `web-svc-asset`, `web-svc-cdn`, `web-svc-coturn`, `web-svc-css`, `web-svc-file`, `web-svc-html`, `web-svc-legal`, `web-svc-logout`, `web-svc-mirror`, `web-svc-simpleicons`. These roles have no independently useful authenticated remote action contract at the pinned implementation. Static published content MAY later be consumed through one owning application's bounded `resource_readonly` adapter, but a new MCP sidecar that merely reimplements a static site's behavior does not count as integration. Model tool calling in Ollama does not by itself make Ollama an MCP client or server. diff --git a/docs/requirements/037-per-user-agents.md b/docs/requirements/037-per-user-agents.md new file mode 100644 index 0000000000..a5c7ceafe1 --- /dev/null +++ b/docs/requirements/037-per-user-agents.md @@ -0,0 +1,106 @@ +# 037 - Per-User Agents on Demand from Open WebUI + +## User Story + +As a platform user with agent access, I want Open WebUI to start my own Hermes or OpenClaw agent the first time I address it, isolated from every other user's agent, so that I work with a personal agent without an administrator provisioning one for me, while administrators decide through RBAC who may use agents at all. + +## Scope + +- A new role `svc-ai-agent-broker` runs the broker and a restricted container-API proxy. +- `web-app-hermes` and `web-app-openclaw` each gain the RBAC role `agent-user` and keep their existing shared instance unchanged. +- `web-app-openwebui` gains a second OpenAI connection pointing at the broker, forwards the caller's identity and restricts each agent model to its group. +- Compose and swarm are both supported. +- `svc-ai-robot` stays out of the platform list. It runs the same agent in embodied mode, which means it drops the isolating runtime on purpose and hands the container the host devices it is meant to drive. A broker platform is defined by the opposite: `runsc`, no host device, no host mount. Serving an embodied agent per user would either break that invariant or ship a robot that cannot reach its hardware. + +## Design + +### Request flow + +1. Open WebUI sends `POST /v1/chat/completions` with `model: hermes` or `model: openclaw` to the broker, authenticated with the broker key and carrying the `X-OpenWebUI-User-*` headers (`ENABLE_FORWARD_USER_INFO_HEADERS=true`). +2. The broker rejects any request whose bearer is not the broker key, and any request without a user id header. +3. The broker resolves the caller's groups from Keycloak and refuses with HTTP 403 when the caller is not in the `agent-user` group of the requested platform. +4. The broker looks up the caller's agent for that platform by name. When none exists it creates one; when it is stopped it starts it; it then waits until the agent answers its health endpoint. +5. The broker forwards the request to the agent with the agent's own bearer key and streams the answer back unchanged. +6. `GET /v1/models` lists both platforms; Open WebUI shows each only to its group through model access grants. + +### Model visibility + +With the broker deployed, Open WebUI runs with `BYPASS_MODEL_ACCESS_CONTROL=false`. A deploy-time step grants every LiteLLM model public read access and each agent model read access for its `agent-user` group only. A `svc-ai-litellm` deploy repeats that step in a running Open WebUI, so a model added to the gateway is visible to every user without an Open WebUI deploy. + +### Agent container + +- One container (compose) or one single-replica service (swarm) per user and platform, named from the platform and a hash of the user id, labelled with owner and platform. +- Runtime is the isolating runtime selected by `svc-virt-kata`; the broker stops and refuses an agent that came up under `runc`. +- No privileged mode, no host network, pid, ipc or user namespace, no host bind mount, no container-engine socket. +- A dedicated volume per user and platform holds the agent's state; it survives a stop. +- A dedicated network per agent whose only other member is the broker. +- A random bearer key per agent, known only to the broker, used in both directions. +- The agent holds no model credential: it calls `http://agent-broker:/llm/v1`, and the broker relays the call to LiteLLM with the broker's own virtual key and the owner in the OpenAI `user` field, and logs a `relay` event with owner, platform, path and upstream status. +- CPU, memory and pids limits come from the platform role's `meta/services.yml`. + +### Lifecycle + +`svc-ai-agent-broker/meta/services.yml` carries under `agent-broker.agents`: + +| Key | Meaning | +|---|---| +| `model` | model alias every agent is configured with; defaults to the gateway's chat model | +| `idle_stop` | `true` stops an agent after `idle_minutes` without a request; `false` keeps it running | +| `idle_minutes` | idle time before a stop | +| `max_running` | upper bound of concurrently running agents; a request beyond it gets HTTP 503 | +| `start_timeout` | seconds an agent gets to answer its health endpoint | +| `access_cache_seconds` | how long a Keycloak membership answer is reused | + +A stopped agent is started again on the next request with its volume re-attached. An agent serving a request is never stopped, and the idle time is re-read under the agent's lock before the stop. When a caller has lost the `agent-user` group, the broker refuses the request and stops that caller's running agent. + +### Context window + +A model that serves agents declares its context window in tokens: `context` on an `AI_LOCAL_MODELS` entry or on a `services.litellm.remote_models` entry. LiteLLM sends it to Ollama as `num_ctx` and publishes it as `model_info.max_input_tokens`, and the broker writes it into the agent config as Hermes `model.context_length` and OpenClaw `contextWindow`. Without it Ollama truncates the agent's prompt to its own default window and the agents assume windows of their own. + +A platform may refuse a window it considers too small. Hermes requires 64000 tokens and answers `HTTP 500` with "below the minimum 64,000 required by Hermes Agent" for anything less, so `web-app-hermes` declares `services.hermes.min_context` and the broker withholds any platform whose minimum the selected model misses: it is absent from `/v1/models` and answers 404, while every other platform keeps serving. Refusing the whole deploy instead would block a pairing that works, since OpenClaw declares no minimum and the broker pulls Hermes into its closure either way. A model that declares no window at all is reported rather than withheld: an agent that does not know its window runs. + +### Container API + +The broker reaches the container engine only through a filtered unix socket in a volume it shares with the socket proxy. The proxy serves no TCP port; in compose it also runs with `network_mode: none`, while a swarm task carries the stack's default overlay attachment. The proxy admits only the container, service, task, network, volume and image calls the broker makes, and refuses every other method and path, including exec and delete. The image, runtime, limits and mounts of an agent are fixed by the broker, never taken from the request. + +The proxy filters methods and paths, not request bodies. `POST /volumes/create` stays on the allowlist because the broker creates a volume per agent, and a local-driver volume with `o=bind,device=/` is a host bind mount that arrives as a volume. Fields such as `Privileged`, `PidMode` or `CapAdd` on `containers/create` are likewise unfiltered. The isolation therefore holds against a compromised **agent**, which reaches no socket at all, not against a compromised **broker**. + +## Acceptance Criteria + +### Role and configuration + +- [ ] `svc-ai-agent-broker` deploys in compose and swarm mode and passes the repository lint suite. +- [x] The `agent-broker.agents` keys reach the running broker from `meta/services.yml`, so an inventory override changes its behaviour. +- [x] The socket proxy refuses `POST /containers/{id}/exec`, a container or service create carrying a host bind mount, and `DELETE /containers/{id}`. +- [x] The socket proxy serves the engine on the unix socket shared with the broker and publishes no port; in compose it runs with `network_mode: none`. + +### Access control + +- [x] `web-app-hermes` and `web-app-openclaw` each declare the RBAC role `agent-user`, which provisions a Keycloak group. +- [x] Open WebUI lists `hermes` and `openclaw` only to a user in the matching group, while every LiteLLM model stays visible to all users. +- [x] A model added to LiteLLM becomes visible to all Open WebUI users after a `svc-ai-litellm` deploy alone. +- [x] A direct call to the broker for a user outside the group gets HTTP 403 and creates no container. +- [x] A request with a wrong or missing broker key gets HTTP 401. + +### Agent lifecycle + +- [x] The first prompt of an entitled user creates exactly one agent for that user and platform and returns the agent's answer in Open WebUI. +- [x] A second prompt from the same user reuses the same agent without restarting it. +- [x] A stopped agent is started again by the next prompt, keeps its container and its state. +- [x] With `idle_stop: true` the sweep stops an agent that has been idle for `idle_minutes`, and never one that is serving a request or was just used. +- [x] With `idle_stop: false` the sweep stops nothing. + +### Isolation + +- [x] Two users get two different containers, volumes, networks and bearer keys. +- [x] An agent inspected on the engine runs under the isolating runtime, not `runc`. +- [x] A file written by one user's agent is not visible to the other user's agent. +- [x] One agent cannot reach the other agent's network address. + +### Tests + +- [x] `web-app-openwebui` ships a Playwright spec that logs in as `biber` without `agent-user` and asserts that neither agent model is listed or answers, while every served gateway model is listed. +- [x] The same spec grants `biber` the `agent-user` groups, logs in again and asserts an answer from `hermes` and from `openclaw` through Open WebUI. +- [x] `svc-ai-agent-broker` ships a CLI test that proves the broker-key refusal, the RBAC refusal, two distinct agents under the isolating runtime, file isolation and network isolation on the engine. +- [x] The same CLI test proves from the broker's `relay` events that a `hermes` and an `openclaw` agent each got a model answer from LiteLLM, independent of what the model wrote. +- [x] Unit tests cover the idle sweep: it stops an idle agent, skips one that is serving a request or was just used, and does nothing while `idle_stop` is false. diff --git a/docs/requirements/038-system-one-model-router.md b/docs/requirements/038-system-one-model-router.md new file mode 100644 index 0000000000..87c7745171 --- /dev/null +++ b/docs/requirements/038-system-one-model-router.md @@ -0,0 +1,211 @@ +# 038 - System One Model Router + +## User Story + +As a consumer of the Infinito.Nexus LLM gateway, I want to send a prompt to one alias and have a fast typed decision pick the model that can actually serve it, so that I neither hard-code a model per call site nor pay a large model to answer a question a small one handles. + +## Background + +[031](031-llm-gateway-model-backends.md) made [`svc-ai-litellm`](../../roles/svc-ai-litellm/) the single place model backends are wired, and consumers now carry a gateway URL plus a model alias. That moved the wiring, not the choice: every consumer still names one alias, and `LITELLM_CHAT_MODEL` in [16_ai.yml](../../group_vars/all/16_ai.yml) resolves it once per deploy for everyone. + +That choice is wrong in two directions at once. A prompt that needs a large context window reaches a model that cannot hold it, which is how [`web-app-hermes`](../../roles/web-app-hermes/) answers HTTP 500 when the agent model declares fewer than 64000 tokens. And a one-line factual question reaches whatever the deploy happened to pick, which is usually the largest model available. + +Daniel Kahneman's split names the shape. The models behind the gateway are System II: slow, general, token-generating. What is missing is System I: fast, typed, no generation, deciding which System II model answers. TypeSafe released [Jev](https://en.wikipedia.org/wiki/Jev_(AI_model)) on 2026-09-15 as a model of exactly that kind, and [jev-router](https://github.com/prismhq/jev-router) wires one into LiteLLM as a pre-call hook. This requirement takes that shape and keeps the decision deterministic; decision 8 records why. + +This is the **routing plane**. The model plane is [031](031-llm-gateway-model-backends.md), the tool plane is [025](025-mcp-role-integration.md), and the agent fleet that benefits most is [032](032-agent-employees-firecracker.md). + +## Confirmed Decisions + +These choices are settled at requirement creation time and bound the implementation. Re-opening any of them MUST be recorded in the implementing PR. + +1. **The router is a LiteLLM pre-call hook, not a semantic router.** The hook summarizes the request, filters the gateway's own model list down to the routes that can serve it, and picks one. LiteLLM's built-in `auto_router` is explicitly rejected: it routes by embedding similarity to example utterances, which answers "what is this about" rather than "which model can serve this", and it pulls in the `semantic-router` extra plus an embedding backend the gateway does not otherwise need. + +2. **The hook ships as a mounted Python file, not a derived image.** LiteLLM's `proxy_server.py` resolves a custom handler through `get_instance_fn(value, config_file_path)`, which loads `module.instance` out of the directory holding `config.yaml`. A `.py` file rendered next to it is therefore imported at proxy start with no image change, and `services.litellm.image` stays on its pin. [`probe_auth.py`](../../roles/svc-ai-litellm/files/python/probe_auth.py) is the precedent for a Python file the role ships and the container runs. + +3. **The router is an alias beside the default, and it fails loud.** The gateway publishes one additional `model_name` (`auto`). `LITELLM_CHAT_MODEL` and every existing alias keep resolving exactly as they do today, and a consumer opts in by naming `auto`. + + Every path that cannot route rejects the request with its reason instead of answering. The hook returns a string when the proxy has no router or when no served model can take the request, and the proxy turns that into a `RejectedRequestError` carrying the text. The alias's own `model_list` entry exists only so the name is listable and carries `mock_response: litellm.InternalServerError`, so reaching it at all raises. Both halves matter, because a router that silently served whatever was first would hide its own failure behind a plausible reply, and the alias exists precisely so callers do not have to know which model answered. + +4. **The alias is the switch.** `services.litellm.router_alias` defaults to `auto`; setting it empty publishes no route and renders no `callbacks` entry, so the hook is not even a condition of starting the proxy. One knob, because a separate enable flag would let `services.yml` read as on while the router is off. + +5. **Eligibility is read from LiteLLM's own catalog; a declaration is the override.** The image ships `model_prices_and_context_window_backup.json`, 1595 models deep, carrying `max_input_tokens`, `supports_vision`, `supports_function_calling` and `input_cost_per_token`. The hook reads a route's capabilities from `litellm.model_cost`, keyed on the route's own `litellm_params.model` and then on that name without its first segment, because `ollama/llama3` is filed prefixed while `openai/gpt-4o-mini` is filed as `gpt-4o-mini`. It never probes a backend at request time. + + A model entry MAY still carry a `traits` mapping (`vision`, `tools`, `max_output`, `cost`), and it wins over the catalog. That is for what the catalog does not know: `openrouter/auto`, the mock, and a local tag such as `ollama/qwen2.5:0.5b` that is absent although its family is present. `context` keeps its existing home on the entry, feeds `AI_MODEL_CONTEXTS` and `num_ctx`, and outranks the catalogued window. + + Declaring what the catalog already states is forbidden, because it duplicates a fact this project does not own and rots silently in the dangerous direction: a stale `supports_vision` keeps a model eligible for requests it can no longer serve. Cost is not declared either. Whether a route is free follows from what it is: a route with an `api_base` or a `mock_response` is served by this deployment and costs nothing, and a keyed route the catalog does not price sorts behind every route it does. + + A capability neither catalogued nor declared counts as absent, and a limit neither catalogued nor declared is not enforced. + + The hook reads every catalogued field a request can act on, not a chosen few: the context window and output cap, `supports_vision`, `supports_function_calling`, `supports_tool_choice`, `supports_parallel_function_calling`, `supports_response_schema`, `supports_pdf_input`, `supports_audio_input`, `max_images_per_prompt`, `supports_system_messages`, `mode`, `deprecation_date`, both per-token prices, `supports_prompt_caching` and `tpm`. + + Two polarities govern them, and mixing them up routes silently wrong. A capability the request **demands** must be proven, because such a flag is catalogued wherever it exists: an absent `supports_vision` means no vision, and the route is dropped only when the request carries an image. A **disqualifier** acts on explicit evidence only, because silence is the norm for it: 256 of 1595 entries carry `supports_system_messages` at all and a local tag is catalogued not at all, so reading silence as denial would drop nearly every route. A model is therefore excluded only when the catalog states a non-chat `mode`, denies system messages outright, or gives a `deprecation_date` already past. + + Three of those are live rather than theoretical. 314 of the 1595 entries are not chat models, so without the `mode` filter an embedding model in the `model_list` is a valid candidate that fails at the backend. 53 of the 54 dated entries are already past their deprecation date. And the catalog ships a `sample_spec` documentation stub whose `mode` and `deprecation_date` are prose, so only an ISO date counts and anything else reads as undated. + +6. **Variant 1 of `svc-ai-ollama` serves three models with different windows.** A router is only observable when the routes differ in something the request can demand. Variant 1 therefore preloads `qwen2.5:0.5b`, `smollm2:135m` and `llama3.2:1b` with declared windows of 4096, 16384 and 65536. A declared window below what the tag supports is deliberate: it bounds the KV cache, and it is the number the filter reads. + + The CLI probe sizes its prompt off the smallest declared window, so the model holding that window is the one the request must exclude, and the model that answers instead is what shows the filter ran. `smollm2:135m` is deliberately not the smallest: `8dbdd31b22` measured it running to `n_gen=22011` with context shifts at `n_left=4091`, and a route's `max_tokens` is a default a consumer can override rather than a cap, so it is kept away from the window at which it was seen to run away. + + Variant 0 preloads nothing and the gateway serves its mock, so none of this costs anything in the rounds that are not about serving. + +7. **A request stays inside the deployment while a local model can serve it.** A prompt can carry a credential, so locality is its own rank above price rather than a consequence of local models happening to be free. `services.litellm.router_remote_fallback` defaults to `false`: while any local route is deployed, a request no local model can take is refused with its reason instead of reaching a third party, and the refusal names the switch that would allow it. With no local route deployed the rule does not apply, because there would be nothing to hold the request on. + + A structured secret in the prompt forces the same restriction even when the switch is on. Only unambiguous shapes count (PEM headers, `sk-`, `ghp_`, `github_pat_`, `AKIA`, `xox[bapsr]-`, `AIza`); a pattern like `password=` would match prose and teach callers to click past the refusal. This is a second lock and not the guarantee, because any such pattern has false negatives. The guarantee is that this restriction is a filter applied before any ranking: a restricted request has no remote candidate left to rank, whatever weight locality carries in decision 9. + +8. **The decision is deterministic, and a learned decision model is out of scope.** Among the routes that survive the filter, each is scored on the factors of decision 9 and the highest scorer wins, with the alias breaking a tie. The same request therefore returns the same model twice. No inference runs, nothing is downloaded, and the CI rows mean the same thing as production. + + **This decision was reversed.** A System One *model* was weighed and rejected here on the ground that the filter in decision 5 already leaves one to three candidates, so a model choosing between two that can both answer buys accuracy nobody can measure while adding a container, a checkpoint and a second thing that can be down when the gateway starts. The operator reversed it: [`svc-ai-s1`](../../roles/svc-ai-s1/) now serves that decision under decision 11's `system_one` strategy. + + What the rejection got right still holds and is now a cost rather than an argument. jeff ships no container image, so the role builds one from a pinned commit. It answers over a 575M GLiFormer encoder, so it carries a checkpoint. And the `auto` alias now depends on a service that can be unavailable, which is why an unreachable decider rejects loudly instead of falling back. + + What the rejection got wrong was the reach of its last point. A learned decider does make a CI row speak less directly for production, but only for the rounds that deploy it; the `weighted` strategy still decides every round that does not, and it remains reproducible. + + The remaining reproductions of Jev, [open-alternative-jev](https://github.com/ikermoel/open-alternative-jev) on any open-weights model and OpenJev on DiffusionGemma 26B-A4B, are not used. + +9. **How much each factor counts is a number in `services.yml`, between 0 and 1.** `services.litellm.router_weights` carries one weight per factor: `locality`, `cost` and `speed`. Each factor is scaled to 0..1 *inside the candidate set* before its weight applies, because the raw units do not compare - a per-token price is a millionth of a cent and a rate is tens of tokens per second. A factor whose value is identical across the set scores every route 1 and therefore decides nothing, rather than ranking on a rounding artefact. + + The factors are `locality`, `cost`, `speed`, `caching` and `throughput`. `cost` is the expected bill for this request, input price times estimated input tokens plus output price times requested output, because ranking on the input price alone reads the cheaper half: an output token commonly costs several times an input one, and a short prompt with a long answer inverts the order the input price suggests. `caching` prefers a route that can reuse a cached prefix, which is the largest real saving for an agent with a stable system prompt. `throughput` reads the catalogued `tpm`, the only quota the catalog carries. + + The shipped weights are `locality: 1.0`, `cost: 0.4`, `speed: 0.25`, `caching: 0.15`, `throughput: 0.1`, and the constraint behind them is that locality **exceeds** the sum of the others, so no combination of the rest outscores staying in the cluster. That is a preference, not the protection: decision 7's restriction removes remote candidates before this runs, which is why a weighting that sends everything to the cheapest provider still cannot send a credential there. + + Weights are a preference because the operator is the only one who knows what this deployment is for. A cluster whose local models are toys wants `speed` high; a cluster billing a provider per token wants `cost` high; a cluster handling client data leaves `locality` where it is. Encoding one of those as the rule would be this project guessing. + +10. **Speed is measured at deploy time, not declared, and only for a model that has no rate yet.** A rate is a property of this machine under this load, not of the model, so no catalogue carries one and a hand-written number in `services.yml` would be a guess that rots. After the gateway is warm, [measure_speed.py](../../roles/svc-ai-litellm/files/python/measure_speed.py) asks it which models it serves and sends one warm-up request plus `services.litellm.router_speed_samples` (12) measured requests to each unranked one, taking their median as output tokens per second. The warm-up is discarded at every setting, including a one-sample run: a rate that still carries the model load time ranks a small model below a larger one, and the router then routes on that inversion. + + A model that already carries a rate is not measured at all. Re-measuring everything on every deploy would send traffic whose result the tolerance below discards anyway, and that traffic is the entire live surface of this step: it is what can hang, what can fail the deploy, and what the deploy waits for. `services.litellm.router_speed_remeasure` discards the stored numbers once, for the case where the hardware under them changed. + + The result is stored on the stack host and rendered back into `model_info.traits.speed` on the same deploy, so the gateway restarts once with the numbers it just produced. A re-measured rate that moved less than 10% keeps its stored value, because no two runs measure the same machine and an unclamped result would restart the gateway for nothing. A model the measurement could not reach keeps its stored rate rather than losing it, so one bad minute does not unrank a model against its peers. + + Two budgets bound the step. A measured request may take `services.litellm.router_speed_timeout` (60s); the discarded first request gets `services.litellm.upstream_timeout` instead, because it pays the model load and holding it to the sample budget would time out exactly the models worth measuring. A request that exceeds its budget raises, which abandons that model rather than the deploy. + + Every matrix variant overrides the sample count to 1. At that setting the single sample is kept rather than discarded as cold, because what a CI round proves is that the measurement, the store and the re-render ran, not what the rate was on a shared runner. + + A declared `traits.speed` still wins, because a declaration is the operator overriding the measurement and the next deploy must not overwrite it back. + +11. **Two strategies decide among the survivors, and `services.litellm.router_strategy` picks one.** Both run behind the same filter and behind decision 7's locality restriction; they differ only in what ranks the routes that are left. + + `weighted` is decision 9's score. It ranks on standing preferences and, once the filter has run, ignores the prompt entirely: the same candidate set returns the same winner whatever was asked. The name is the mechanism, and it names its own configuration: `router_strategy: weighted` is the strategy that reads `router_weights`, which `system_one` ignores. + + `system_one` puts the choice to [`svc-ai-s1`](../../roles/svc-ai-s1/). The surviving candidates become the options of a single `choice` question and the prompt becomes the state it classifies, so the decision reads what the request is about rather than only how large it is. An option the router did not offer is refused rather than routed to, because the filter excluded the others for a reason. + + Neither strategy is chosen by hand. `services.litellm.router_strategy` derives from `services.jeff.enabled`: with the System One service deployed the gateway asks it, and without it there is nothing to ask, so the gateway falls to the strategy it can compute itself. Reading the service flag rather than `group_names` keeps one path from the deployment to the behaviour. + + A jeff that cannot be reached rejects the request. It is not routed around, because the caller named an alias precisely so it would not have to know which model answered, and substituting a different decider would make that answer mean something else. + +12. **Variant 2 of `svc-ai-litellm` carries `system_one` on three mocks, and no fourth variant is added.** The strategy is only observable when the routes differ in the one thing it ranks on, so that variant publishes mocks with windows of 4096, 16384 and 131072. Mocks pull no model and cost no wall clock, and they carry no measured rate, so nothing but the window separates them and the round tests exactly the axis the strategy adds. `mock/deterministic` keeps its name and its 131072 window, because the agent broker runs its agents on it in every round. + + The order of those mocks is load-bearing and the router is not what reads it. `AI_AGENT_MODEL` takes `AI_MOCK_ALIASES | first`, so the mock declared first becomes the agent broker's model and its window becomes the context the broker gates platforms on. Declaring the 4096 mock first withheld `hermes`, whose `min_context` is 64000, and the broker's CLI test failed with the reason spelled out. The widest mock therefore leads. A sharper fix would be to have `AI_AGENT_MODEL` pick the mock with the largest declared context instead of the first one, which would make the order irrelevant. + + Variant 2 also pins `services.jeff.enabled` true, which is what selects the strategy and what `test_bond_baseline` requires of a loose-bond partner: a service nothing ever enables is a service nothing ever tests. Variant 1 keeps `weighted` with three real models, so each strategy owns one observable round. + + A fourth variant was built first and then withdrawn: `tests/integration/roles/meta/variants/test_limit.py` caps a role at three, on the stated ground that variants exercise dynamic-flag polarities rather than enumerate features, and the `# nocheck: variant-limit` escape would have been a suppression bought to keep a design that fits in three anyway. + +13. **A declared window is an upper bound the backend honours, and the token demand is a floor.** Two numbers guard the same decision from opposite sides, and each was wrong in the direction that hid the other. `smollm2:135m` was declared at 16384 against a trained `max_position_embeddings` of 8192 with no rope scaling, so Ollama clamped `num_ctx` and then truncated the prompt to half the window, which is the `n_left=4091` recorded in `8dbdd31b22`. Meanwhile `CHARS_PER_TOKEN = 3` was justified as erring high, but the router's own probe builds its oversized prompt from `"x "`, which tokenizes at two characters per token: 16384 characters are 8193 real tokens on all three served tokenizers against an estimate of 5461. + + Neither correction works alone, which is why both ship. Declaring 8192 while still estimating 5461 leaves the model eligible; estimating 8192 while still declaring 16384 does the same. Together the demand becomes 8192 plus the 16 output tokens, which exceeds the honest window and excludes the route. The constant is therefore `services.litellm.router_min_chars_per_token`, named as a floor rather than an average, and it is rendered into both the hook and the probe so the two cannot drift. + + The probe's own check was vacuous beside this: it compared a declared window against an estimated demand, both produced by this deployment, so it structurally could not observe a declaration the backend does not honour. It now also reads `usage.prompt_tokens` from the answer, the one number in the exchange that the backend produced. + +## Component Roles + +The router sits inside the gateway. No consumer learns a second endpoint. + +| Component | Role | Interactive? | +|---|---|---| +| [`web-app-openwebui`](../../roles/web-app-openwebui/) | Human chat UI, may select `auto` like any other model | Human, on demand | +| `web-app-hermes`, `web-app-openclaw` ([032](032-agent-employees-firecracker.md)) | Autonomous agents, benefit most from capability filtering | Autonomous | +| [`svc-ai-litellm`](../../roles/svc-ai-litellm/) | Hosts the pre-call hook; publishes the `auto` alias | - | +| [`svc-ai-ollama`](../../roles/svc-ai-ollama/), [`svc-ai-lmstudio`](../../roles/svc-ai-lmstudio/), OpenRouter | System II candidates, unchanged by this requirement | - | + +```mermaid +flowchart TB + consumer([Consumer asks for model 'auto']) + + subgraph gw["svc-ai-litellm"] + hook["pre-call hook
summarize request"] + caps["capabilities()
litellm.model_cost + declared traits"] + elig["eligible()
drop what cannot serve it"] + keep["local_reason()
drop remote when it must stay in"] + dec["decide()
weighted score or system_one choice"] + jeff["svc-ai-s1
typed choice, no generation"] + weights["router_strategy + router_weights
locality, cost, speed, caching, throughput"] + rates["measure_speed.py
1 warm-up + 12 measured per model at deploy"] + end + + subgraph sys2["System II candidates"] + small["local model"] + remote["external provider"] + end + + consumer --> hook --> caps --> elig --> keep --> dec + weights --> dec + dec -->|choice question| jeff + jeff -->|chosen alias| dec + rates -->|traits.speed| caps + dec -->|chosen alias| sys2 + small & remote -->|completion| consumer +``` + +## Where each guarantee is enforced + +| Guarantee | Enforced by | +| --- | --- | +| The `auto` alias exists and every other alias still resolves | [test_config_model_list.py](../../tests/unit/python/roles/svc-ai-litellm/templates/test_config_model_list.py), and the role's own routing probe on every deploy | +| The callback names the file the role mounts | the same test, reading the target out of [meta/volumes.yml](../../roles/svc-ai-litellm/meta/volumes.yml) rather than repeating it | +| A gateway with no router alias loads no hook | the same test | +| The alias raises rather than serving a default | the same test | +| A request whose demands exceed a model's capabilities never reaches it | [test_router_hook.py](../../tests/unit/python/roles/svc-ai-litellm/templates/test_router_hook.py), against an injected catalog | +| A declaration wins over the catalog, and an uncatalogued route claims nothing | the same test | +| The decision repeats for the same request | the same test | +| A request stays local while a local model can serve it, switch or not | the same test, over the switch and over a prompt carrying a secret | +| Speed decides among equal-cost routes, and unknown does not count as fast | the same test | +| Each factor carries the weight `services.yml` gives it, and a zero weight removes it | the same test, rendering the hook with the shipped weights and with a single-factor one | +| No weighting sends a restricted request out of the cluster | the same test, weighting speed alone with the fallback switch on | +| The shipped weights keep locality worth at least the other factors combined | the same test, reading them out of [meta/services.yml](../../roles/svc-ai-litellm/meta/services.yml) rather than repeating them | +| A measured rate reaches `traits.speed`, and a declared one outranks it | [test_config_model_list.py](../../tests/unit/python/roles/svc-ai-litellm/templates/test_config_model_list.py) | +| The cold first sample is discarded, one slow sample does not move the median, and an unreachable model keeps its rate | [test_measure_speed.py](../../tests/unit/python/roles/svc-ai-litellm/files/test_measure_speed.py) | +| A model that already carries a rate sends no request, and the remeasure switch sends them again | the same test | +| An embedding model, a deprecated model and a model that denies system messages are never routed to | [test_router_hook.py](../../tests/unit/python/roles/svc-ai-litellm/templates/test_router_hook.py), against an injected catalog | +| A demanded capability must be proven, while a disqualifier needs explicit evidence | the same test, over an uncatalogued local route and a catalogued embedding one | +| A prose deprecation value is not read as a date | the same test, modelling the catalog's `sample_spec` stub | +| The expected bill counts the output price, and a long answer can invert the input-price order | the same test | +| `system_one` routes to the alias jeff chose, and `weighted` does not read the prompt at all | the same test, over one route set and both strategies, with jeff stubbed | +| The candidates become the choice options and the prompt becomes the state | the same test | +| A prompt above the server's state limit is truncated rather than rejected as a routing failure | the same test | +| An option the router never offered is refused | the same test | +| An unreachable decider rejects rather than falling back to another one | the same test | +| The cold first request gets the longer budget, and a single-sample run keeps its one measurement | the same test | +| The alias is served and is not read as a route without a backend | [probe.py](../../roles/svc-ai-litellm/files/test/probe.py), run on every gateway deploy | +| The alias routes rather than falling back, proven by the model that answers | [probe.py](../../roles/svc-ai-litellm/files/test/probe.py), on every gateway deploy that serves more than one declared window | +| A consumer selecting `auto` receives a completion | [test-system-one-router.js](../../roles/web-app-openwebui/files/playwright/test-system-one-router.js) | + +## Acceptance Criteria + +- [x] `svc-ai-litellm` publishes an additional `auto` alias, and every alias that resolved before this change still resolves after it. +- [x] The pre-call hook is a Python file the role renders next to `config.yaml` and LiteLLM imports by `module.instance`; `services.litellm.image` and its pinned version are unchanged. +- [x] A gateway that publishes no router alias renders no `callbacks` entry and does not load the hook, and `services.litellm.router_alias` is the only switch. +- [x] A request the router cannot serve is rejected with the reason, and the alias's own route raises rather than answering from a default. +- [x] The hook excludes a candidate whose capabilities cannot serve the request (context window, vision, tools, output length), reading them from LiteLLM's catalog and letting a declared `traits` mapping override it. +- [x] A model entry declares none of what the catalog already states, and a route's price follows from whether it is local, mocked or keyed. +- [x] The same request picks the same model twice, with no inference and no download on any path. +- [x] Every routing factor carries a 0..1 weight in `services.litellm.router_weights`, a zero weight removes the factor, and no weighting lets a restricted request reach a remote model. +- [x] The hook reads every catalogued field a request can act on, a demanded capability is proven before use, and a disqualifier fires only on explicit evidence. +- [x] `services.litellm.router_strategy` derives from `services.jeff.enabled` and selects `weighted` or `system_one`, and the two return different models for the same candidate set. +- [ ] `svc-ai-s1` builds, serves `/healthz`, and answers the gateway's choice question with an alias the router offered. +- [ ] Variant 2 deploys with three mocks of distinct windows and `system_one`, and the routing probe shows the choice following the prompt. +- [ ] The deploy sends `services.litellm.router_speed_samples` requests to each served model, and the measured rates reach `model_info.traits.speed` in the rendered config on the same deploy. +- [x] The routing probe treats the alias as served rather than as a route without a backend, and fails when the gateway withholds it. +- [ ] A request naming `auto` returns a completion whose served model is one of the eligible candidates, verified through the gateway with a consumer virtual key. The CLI probe sends a prompt the smallest declared window cannot hold and fails when the answer names the alias itself or a model whose window is too small. +- [ ] A Playwright spec verifies that selecting `auto` in Open WebUI returns an answer. +- [ ] The routing path comes up green in both compose and swarm modes. + +## Cross-linking + +- Implementing PR: *to be linked*. + +## See Also + +- Model plane this builds on: [031-llm-gateway-model-backends.md](031-llm-gateway-model-backends.md) +- Agent fleet that consumes the gateway: [032-agent-employees-firecracker.md](032-agent-employees-firecracker.md) +- MCP tool plane: [025-mcp-role-integration.md](025-mcp-role-integration.md) +- Upstream shape this follows: [jev-router](https://github.com/prismhq/jev-router) diff --git a/group_vars/all/16_ai.yml b/group_vars/all/16_ai.yml index c79faa45c5..6391ff45fa 100644 --- a/group_vars/all/16_ai.yml +++ b/group_vars/all/16_ai.yml @@ -9,8 +9,19 @@ LITELLM_BASE_LOCAL_URL: "http://{{ lookup('config','svc-ai-litellm', 'services.l LMSTUDIO_BASE_LOCAL_URL: "http://{{ lookup('config','svc-ai-lmstudio', 'services.lmstudio.name') }}:{{ lookup('config','svc-ai-lmstudio', 'services.lmstudio.ports.internal.http') }}" +S1_BASE_LOCAL_URL: "http://{{ lookup('config','svc-ai-s1', 'services.s1.name') }}:{{ lookup('config','svc-ai-s1', 'services.s1.ports.internal.http') }}" + LITELLM_OPENAI_BASE_LOCAL_URL: "{{ LITELLM_BASE_LOCAL_URL }}/v1" +LITELLM_LOOPBACK_HOSTS: + - "127.0.0.1" + - "::1" + - "localhost" + +LITELLM_BASE_LOOPBACK_URL: "http://{{ LITELLM_LOOPBACK_HOSTS | first }}:{{ lookup('config','svc-ai-litellm', 'services.litellm.ports.local.http') }}" + +LITELLM_OPENAI_BASE_LOOPBACK_URL: "{{ LITELLM_BASE_LOOPBACK_URL }}/v1" + LITELLM_BACKEND_ROLES: "{{ lookup('deployment').inventory }}" LITELLM_OLLAMA_BACKEND: >- @@ -22,23 +33,41 @@ LITELLM_LMSTUDIO_BACKEND: >- and (lookup('config', 'svc-ai-litellm', 'services.lmstudio.shared') | bool) }} AI_LOCAL_MODELS: - - alias: qwen2.5:0.5b - name: qwen2.5-0.5b-instruct - repo: lmstudio-community/Qwen2.5-0.5B-Instruct-GGUF - url: https://huggingface.co/lmstudio-community/Qwen2.5-0.5B-Instruct-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-Q8_0.gguf - source: https://huggingface.co/lmstudio-community/Qwen2.5-0.5B-Instruct-GGUF - file: Qwen2.5-0.5B-Instruct-Q8_0.gguf - sha256: "1d96e30253a76970864323987e3e77eeaf170312952fd61d18d343c25cfd6e52" - bytes: 531068224 + - alias: llama3.2:1b + context: 65536 + name: llama-3.2-1b-instruct + repo: lmstudio-community/Llama-3.2-1B-Instruct-GGUF + url: https://huggingface.co/lmstudio-community/Llama-3.2-1B-Instruct-GGUF/resolve/main/Llama-3.2-1B-Instruct-Q4_K_M.gguf + source: https://huggingface.co/lmstudio-community/Llama-3.2-1B-Instruct-GGUF + quant: q4_k_m + file: Llama-3.2-1B-Instruct-Q4_K_M.gguf + sha256: "f7ede42862ceca07ad1c88a97b67520019c4ac7e5ced250d2e696fa62ab189af" + bytes: 807690688 + +AI_MODEL_CONTEXTS: >- + {{ (AI_LOCAL_MODELS + (lookup('config', 'svc-ai-litellm', 'services.litellm.remote_models'))) + | selectattr('context', 'defined') | list | items2dict(key_name='alias', value_name='context') }} + +AI_MOCK_PROVIDER: mock AI_REMOTE_ALIASES: >- - {{ ['openai/gpt-4o-mini'] * (lookup('api_enabled', 'openai') | int) - + ['anthropic/claude-sonnet-4-5'] * (lookup('api_enabled', 'anthropic') | int) - + ['openrouter/auto'] * (lookup('api_enabled', 'openrouter') | int) }} + {{ lookup('config', 'svc-ai-litellm', 'services.litellm.remote_models') + | selectattr('provider', 'in', + [AI_MOCK_PROVIDER] + + ['openai'] * (lookup('api_enabled', 'openai') | int) + + ['anthropic'] * (lookup('api_enabled', 'anthropic') | int) + + ['openrouter'] * (lookup('api_enabled', 'openrouter') | int)) + | map(attribute='alias') | list }} -LITELLM_CHAT_MODEL: "{{ (lookup('config', 'svc-ai-ollama', 'services.ollama.preload_models') | map(attribute='alias') | reject('search', 'embed') | list | first | - default('')) if 'svc-ai-ollama' in LITELLM_BACKEND_ROLES else ((lookup('config', 'svc-ai-lmstudio', 'services.lmstudio.preload_models') | map(attribute='alias') - | reject('search', 'embed') | list | first | default('')) if 'svc-ai-lmstudio' in LITELLM_BACKEND_ROLES else (AI_REMOTE_ALIASES | first | default(''))) }}" +AI_MOCK_ALIASES: >- + {{ lookup('config', 'svc-ai-litellm', 'services.litellm.remote_models') + | selectattr('provider', 'eq', AI_MOCK_PROVIDER) | map(attribute='alias') | list }} + +LITELLM_CHAT_MODEL: "{{ (((lookup('config', 'svc-ai-ollama', 'services.ollama.preload_models') | map(attribute='alias') | reject('search', 'embed') | list) if 'svc-ai-ollama' + in LITELLM_BACKEND_ROLES else []) + ((lookup('config', 'svc-ai-lmstudio', 'services.lmstudio.preload_models') | map(attribute='alias') | reject('search', 'embed') + | list) if 'svc-ai-lmstudio' in LITELLM_BACKEND_ROLES else []) + AI_REMOTE_ALIASES) | first | default('') }}" + +AI_AGENT_MODEL: "{{ AI_MOCK_ALIASES | first | default(LITELLM_CHAT_MODEL, true) }}" LITELLM_CHAT_MODEL_SERVED: >- {{ ((LITELLM_OLLAMA_BACKEND | bool) diff --git a/inventories/bundles/workstations/admin-network/inventory.yml b/inventories/bundles/workstations/admin-network/inventory.yml index b87fd3f391..62b89ef4c5 100644 --- a/inventories/bundles/workstations/admin-network/inventory.yml +++ b/inventories/bundles/workstations/admin-network/inventory.yml @@ -20,6 +20,6 @@ all: hosts: workstation: children: - desk-git: + dsk-git: hosts: workstation: diff --git a/inventories/bundles/workstations/admin/inventory.yml b/inventories/bundles/workstations/admin/inventory.yml index e77d650043..a32d7ac3d6 100644 --- a/inventories/bundles/workstations/admin/inventory.yml +++ b/inventories/bundles/workstations/admin/inventory.yml @@ -17,6 +17,6 @@ all: hosts: workstation: children: - desk-git: + dsk-git: hosts: workstation: diff --git a/inventories/bundles/workstations/browser/inventory.yml b/inventories/bundles/workstations/browser/inventory.yml index 15cc5135d8..b56a67691c 100644 --- a/inventories/bundles/workstations/browser/inventory.yml +++ b/inventories/bundles/workstations/browser/inventory.yml @@ -13,9 +13,9 @@ all: hosts: workstation: children: - desk-chromium: + dsk-chromium: hosts: workstation: - desk-firefox: + dsk-firefox: hosts: workstation: diff --git a/inventories/bundles/workstations/dev-arduino/inventory.yml b/inventories/bundles/workstations/dev-arduino/inventory.yml index e1bce4cc96..ef2fecafb1 100644 --- a/inventories/bundles/workstations/dev-arduino/inventory.yml +++ b/inventories/bundles/workstations/dev-arduino/inventory.yml @@ -17,7 +17,7 @@ all: hosts: workstation: children: - desk-git: + dsk-git: hosts: workstation: dev-arduino: diff --git a/inventories/bundles/workstations/dev-core/inventory.yml b/inventories/bundles/workstations/dev-core/inventory.yml index c6f86d8a7d..84cb6746ab 100644 --- a/inventories/bundles/workstations/dev-core/inventory.yml +++ b/inventories/bundles/workstations/dev-core/inventory.yml @@ -17,7 +17,7 @@ all: hosts: workstation: children: - desk-git: + dsk-git: hosts: workstation: dev-core: diff --git a/inventories/bundles/workstations/dev-java/inventory.yml b/inventories/bundles/workstations/dev-java/inventory.yml index ddc4d5954a..0c260a4b28 100644 --- a/inventories/bundles/workstations/dev-java/inventory.yml +++ b/inventories/bundles/workstations/dev-java/inventory.yml @@ -18,7 +18,7 @@ all: hosts: workstation: children: - desk-git: + dsk-git: hosts: workstation: dev-core: diff --git a/inventories/bundles/workstations/dev-php/inventory.yml b/inventories/bundles/workstations/dev-php/inventory.yml index 7ac2b79385..ef3706d6e1 100644 --- a/inventories/bundles/workstations/dev-php/inventory.yml +++ b/inventories/bundles/workstations/dev-php/inventory.yml @@ -18,7 +18,7 @@ all: hosts: workstation: children: - desk-git: + dsk-git: hosts: workstation: dev-core: diff --git a/inventories/bundles/workstations/dev-python/inventory.yml b/inventories/bundles/workstations/dev-python/inventory.yml index 9b0cbb7da3..bd18e31ff7 100644 --- a/inventories/bundles/workstations/dev-python/inventory.yml +++ b/inventories/bundles/workstations/dev-python/inventory.yml @@ -17,7 +17,7 @@ all: hosts: workstation: children: - desk-git: + dsk-git: hosts: workstation: dev-python: diff --git a/inventories/bundles/workstations/dev-shell/inventory.yml b/inventories/bundles/workstations/dev-shell/inventory.yml index 282c138500..dda91bb10f 100644 --- a/inventories/bundles/workstations/dev-shell/inventory.yml +++ b/inventories/bundles/workstations/dev-shell/inventory.yml @@ -18,7 +18,7 @@ all: hosts: workstation: children: - desk-git: + dsk-git: hosts: workstation: dev-core: diff --git a/inventories/bundles/workstations/game-compose/inventory.yml b/inventories/bundles/workstations/game-compose/inventory.yml index cd499b6071..7ed022a5fc 100644 --- a/inventories/bundles/workstations/game-compose/inventory.yml +++ b/inventories/bundles/workstations/game-compose/inventory.yml @@ -29,6 +29,6 @@ all: hosts: workstation: children: - desk-retroarch: + dsk-retroarch: hosts: workstation: diff --git a/inventories/bundles/workstations/office/inventory.yml b/inventories/bundles/workstations/office/inventory.yml index e8d9ce4567..a5f1f28478 100644 --- a/inventories/bundles/workstations/office/inventory.yml +++ b/inventories/bundles/workstations/office/inventory.yml @@ -17,12 +17,12 @@ all: hosts: workstation: children: - desk-libreoffice: + dsk-libreoffice: hosts: workstation: - desk-chromium: + dsk-chromium: hosts: workstation: - desk-firefox: + dsk-firefox: hosts: workstation: diff --git a/meta/categories.yml b/meta/categories.yml index a0d47de229..ebdc2990d4 100644 --- a/meta/categories.yml +++ b/meta/categories.yml @@ -150,7 +150,7 @@ roles: run_after: [drv] invokable: true - desk: + dsk: title: "Desktop" hanzi: "桌面" # Means "desktop" or literally "desk surface". description: "Desktop environment roles & apps (GNOME, browser, LibreOffice, etc.)" @@ -158,6 +158,12 @@ roles: stage: workstation invokable: true + gnt: + title: "Agents" + hanzi: "代理" # Means "agent" or "proxy". + description: "Coding agents installed on the workstation (Claude Code, Cursor, Pi)" + icon: "fas fa-robot" + web: title: "Web Infrastructure" hanzi: "网络" # Means "network" or "web". diff --git a/meta/packages.yml b/meta/packages.yml index c8edf26784..2d0e56f5ae 100644 --- a/meta/packages.yml +++ b/meta/packages.yml @@ -164,6 +164,16 @@ nfs-client: RedHat: - nfs-utils +npm: + Archlinux: + - npm + Debian: + - npm + RedHat: + names: + - npm + virtual: true + php: Archlinux: - php diff --git a/plugins/filter/dotenv.py b/plugins/filter/dotenv.py index 0b459dd8b7..cf95b597a8 100644 --- a/plugins/filter/dotenv.py +++ b/plugins/filter/dotenv.py @@ -11,7 +11,11 @@ # container. Wrapping the value would leak literal quotes into # ``$KEY`` ("\"password\""), breaking DB connect strings, URL parsers, # and any ``int(os.environ['KEY'])`` consumer. -# So in swarm mode the filter returns the value unchanged. +# So in swarm mode the filter returns the value unchanged, including a +# value carrying spaces: quoting one produced ``AGENT_PLATFORMS="{...}"`` +# in the broker's container, which ``json.loads`` then read as a string +# rather than a mapping. A shell-sourced file needs the opposite and must +# not contain a space at all - join such a value on a comma. # # The mode is read from ``DEPLOYMENT_MODE`` in the rendering context. # When the context is missing (e.g. unit tests calling the filter @@ -43,11 +47,7 @@ def _quote_compose_style(value: Any) -> str: def _passthrough_swarm(value: Any) -> str: if value is None: return "" - s = str(value) - if any(c in s for c in " \t\n"): - s = s.replace("\\", "\\\\").replace('"', '\\"') - return f'"{s}"' - return s + return str(value) @jinja2.pass_context diff --git a/plugins/filter/litellm/__init__.py b/plugins/filter/litellm/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/plugins/filter/litellm/model_routes.py b/plugins/filter/litellm/model_routes.py new file mode 100644 index 0000000000..d99f19b23e --- /dev/null +++ b/plugins/filter/litellm/model_routes.py @@ -0,0 +1,155 @@ +"""Turn the model sources LiteLLM serves into one ordered route list. + +The proxy config used to grow a loop per source, so the rule for "how is this +alias called" lived in four places that drifted apart: the LM Studio branch +published no context window although its README promised one, and a mock needed +a fifth branch. Here the sources meet once, the template renders one loop, and +every shape is a unit test instead of a rendered fixture. + +Order is the contract: the first route of the list is what ``LITELLM_CHAT_MODEL`` +names, so a locally served model outranks a declared one, and the earlier +backend outranks the later. +""" + + +def _local_route( + alias, model, api_base, *, max_tokens, timeout, context, traits, api_key=None +): + params = { + "model": model, + "api_base": api_base, + "rpm": 1000, + "max_tokens": max_tokens, + "timeout": timeout, + } + if api_key: + params["api_key"] = api_key + if context: + params["num_ctx"] = context + return {"alias": alias, "params": params, "context": context, "traits": traits} + + +def litellm_model_routes( + ollama_models, + lmstudio_models, + declared_models, + *, + ollama_url, + lmstudio_url, + max_tokens, + timeout, + mock_provider, + router_alias=None, +): + """Every model LiteLLM publishes, as ``{alias, params, context, traits}``. + + Args: + ollama_models: preload entries svc-ai-ollama serves; empty when it is + not a backend of this deployment. + lmstudio_models: preload entries svc-ai-lmstudio serves, same rule. An + alias Ollama already serves is dropped here rather than published + twice. + declared_models: ``services.litellm.remote_models`` already narrowed to + the providers this host holds a key for. An entry whose provider is + ``mock`` answers from its own ``response`` and calls nothing. + ollama_url: base URL of the Ollama API. + lmstudio_url: base URL of the LM Studio API, without the ``/v1`` suffix. + max_tokens: output bound every route carries. + timeout: upstream timeout every calling route carries. + mock_provider: the provider name that marks a mock, passed in rather + than hardcoded so this and the Ansible side cannot drift apart. + router_alias: when given and at least one route exists, one further + route under that alias. The pre-call hook rewrites it to whichever + route can serve the request, so this entry exists to make the alias + listable and MUST never answer: it raises, because a router that + silently served the default would hide its own failure behind a + plausible reply. + + Returns: + The routes in publication order: Ollama, then LM Studio, then declared, + then the router alias. + """ + routes = [] + served = set() + + for model in ollama_models or []: + alias = model["alias"] + served.add(alias) + routes.append( + _local_route( + alias, + f"ollama/{alias}", + ollama_url, + max_tokens=max_tokens, + timeout=timeout, + context=model.get("context"), + traits=model.get("traits"), + ) + ) + + for model in lmstudio_models or []: + alias = model["alias"] + if alias in served: + continue + served.add(alias) + routes.append( + _local_route( + alias, + f"openai/{model['name']}", + f"{lmstudio_url}/v1", + max_tokens=max_tokens, + timeout=timeout, + context=model.get("context"), + traits=model.get("traits"), + api_key="lm-studio", + ) + ) + + for model in declared_models or []: + alias = model["alias"] + context = model.get("context") + if model.get("provider") == mock_provider: + params = { + "model": f"openai/{alias}", + "api_key": mock_provider, + "mock_response": model["response"], + "max_tokens": max_tokens, + } + else: + params = { + "model": model["model"], + "api_key": f"os.environ/{model['provider'].upper()}_API_KEY", + "max_tokens": max_tokens, + "timeout": timeout, + } + routes.append( + { + "alias": alias, + "params": params, + "context": context, + "traits": model.get("traits"), + } + ) + + if router_alias and routes: + routes.append( + { + "alias": router_alias, + "params": { + "model": f"openai/{router_alias}", + "api_key": router_alias, + "mock_response": "litellm.InternalServerError", + }, + "context": None, + "traits": None, + } + ) + + return routes + + +class FilterModule: + """Expose the LiteLLM route builder to the proxy config template.""" + + def filters(self): + return {"litellm_model_routes": litellm_model_routes} diff --git a/plugins/lookup/ca_injected.py b/plugins/lookup/ca_injected.py index fb7e98f6e6..de474ed8ac 100644 --- a/plugins/lookup/ca_injected.py +++ b/plugins/lookup/ca_injected.py @@ -10,6 +10,12 @@ lived as four hand-copied expressions - both `sys-svc-compose` handlers, `sys-svc-compose-ca/vars/main.yml` and the msmtp template - which is one copy per place that can drift. +Those three ask whether the application is *served* over the self-signed CA. +An application that only *calls* one has no domain of its own and would be +skipped, so a service may declare ``ca_client: true`` and be answered on the +deployment's TLS mode alone. Without it such a container verifies against an +empty trust store and fails every outbound call at the TLS handshake. + Returns a plain bool, so callers read as a predicate: when: lookup('ca_injected', application_id) @@ -26,6 +32,7 @@ from ansible.plugins.loader import lookup_loader from ansible.plugins.lookup import LookupBase from plugins.filter.has.domain import has_domain +from utils.tls_common import AVAILABLE_FLAVORS, as_str, require SELF_SIGNED_MODE = "self_signed" @@ -53,7 +60,21 @@ class LookupModule(LookupBase): )[0] if not has_domain(_run("domains", []), application_id): - return [False] + services = _run("config", [application_id, "services", {}]) or {} + if not any( + _to_bool(service.get("ca_client", False)) + for service in services.values() + if isinstance(service, dict) + ): + return [False] + mode = as_str(require(variables or {}, "TLS_MODE", str)) + if mode not in AVAILABLE_FLAVORS: + raise AnsibleError( + f"ca_injected: TLS_MODE must be one of " + f"{sorted(AVAILABLE_FLAVORS)}, got '{mode}'" + ) + return [mode == SELF_SIGNED_MODE] + if not _to_bool(_run("tls", [application_id, "enabled"])): return [False] return [_run("tls", [application_id, "mode"]) == SELF_SIGNED_MODE] diff --git a/plugins/lookup/compose_file_args.py b/plugins/lookup/compose_file_args.py index e935d8f6c6..7b9e0de235 100644 --- a/plugins/lookup/compose_file_args.py +++ b/plugins/lookup/compose_file_args.py @@ -8,8 +8,7 @@ # - Always include base compose.yml # - Include compose.override.yml ONLY when the ROLE (application_id) provides one # (same logic as tasks/04_files.yml with_first_found) -# - Include compose.ca.override.yml ONLY when: -# the app has a domain AND TLS is enabled AND TLS mode == "self_signed" +# - Include compose.ca.override.yml ONLY when lookup('ca_injected') is true # # Optional kwargs: # - include_ca (bool, default True): @@ -59,27 +58,6 @@ def _maybe_template(templar: Any, value: Any) -> Any: return value -def _value_has_domain(v: Any) -> bool: - if v is None: - return False - if isinstance(v, str): - return v.strip() != "" - if isinstance(v, (list, tuple, set)): - return any(_value_has_domain(x) for x in v) - if isinstance(v, dict): - return any(_value_has_domain(x) for x in v.values()) - return False - - -def _has_domain(domains: Any, application_id: str) -> bool: - """ - Dependency-free domain existence check that matches the intent of the old filter. - """ - if isinstance(domains, dict): - return _value_has_domain(domains.get(application_id)) - return _value_has_domain(domains) - - def _role_provides_override(*, application_id: str, templar: Any) -> bool: """ Mirror tasks/04_files.yml "with_first_found" logic: @@ -183,35 +161,15 @@ class LookupModule(LookupBase): parts.append(f"{flag} {override}") if include_ca: - domains = lookup_loader.get( - "domains", loader=self._loader, templar=templar - ).run([], variables=variables)[0] - if _has_domain(domains, application_id): - tlsr = lookup_loader.get("tls", self._loader, self._templar) - tls = tlsr.run([application_id], variables=variables)[0] - - if not isinstance(tls, dict): + injected = lookup_loader.get( + "ca_injected", loader=self._loader, templar=templar + ).run([application_id], variables=variables)[0] + if injected: + if not _as_str(ca_override): raise AnsibleError( - f"compose_file_args: tls returned non-dict: {type(tls)}" + "compose_file_args: compose.files.compose_ca_override is " + "required when the CA is injected" ) - if "enabled" not in tls: - raise AnsibleError( - "compose_file_args: tls did not return 'enabled'" - ) - if "mode" not in tls: - raise AnsibleError("compose_file_args: tls did not return 'mode'") - - enabled = bool(tls["enabled"]) - mode = _as_str(tls["mode"]) - if not mode: - raise AnsibleError("compose_file_args: tls returned empty 'mode'") - - if enabled and mode == "self_signed": - if not _as_str(ca_override): - raise AnsibleError( - "compose_file_args: compose.files.compose_ca_override is required " - "when TLS is enabled and mode is self_signed" - ) - parts.append(f"{flag} {ca_override}") + parts.append(f"{flag} {ca_override}") return [" ".join(parts)] diff --git a/plugins/lookup/group_roles.py b/plugins/lookup/group_roles.py index 65df86a0bf..dda7a8e00f 100644 --- a/plugins/lookup/group_roles.py +++ b/plugins/lookup/group_roles.py @@ -7,6 +7,14 @@ over the group instead of including a generated task file per group: `role` is the directory under ``roles/``, `app` the role's application_id -- the value the deployment whitelist and `application_allowed` speak in. + +Passing ``inventory_groups`` (the ``groups`` magic variable) drops the roles no +host in the inventory carries, so they never enter the loop at all. That list +is identical on every host, which is the whole point: 8aea5a9b35 measured that +a per-host condition above the loop makes ansible insert blocks in host order +rather than loop order, after which svc-storage-nfs-client mounted an export +svc-storage-nfs-server had not deployed yet. Narrowing to a single host stays +in the body's ``when``, never here. """ from __future__ import annotations @@ -37,7 +45,12 @@ class LookupModule(LookupBase): raise AnsibleError("group_roles lookup: the group must not be empty") roles_dir = str(kwargs.get("roles_dir") or Path.cwd() / "roles") + inventory_groups = kwargs.get("inventory_groups") try: - return [[dict(entry) for entry in ordered_roles(roles_dir, group)]] + entries = [dict(entry) for entry in ordered_roles(roles_dir, group)] except Exception as exc: raise AnsibleError(f"group_roles: {exc}") from exc + + if inventory_groups is None: + return [entries] + return [[entry for entry in entries if inventory_groups.get(entry["app"])]] diff --git a/plugins/lookup/roles_with_service.py b/plugins/lookup/roles_with_service.py index 7c1001640e..e354ad359f 100644 --- a/plugins/lookup/roles_with_service.py +++ b/plugins/lookup/roles_with_service.py @@ -23,6 +23,9 @@ Kwargs: ``deployment`` restricts to roles present anywhere in ``groups``, which is what a container-network consumer can actually reach; ``all`` returns every declaring role, deployed or not. + include_headless: ``True`` also returns declaring roles without a + canonical domain, with empty ``canonical_domain`` and + ``canonical_url``. direction: opt-in MCP filter. When set (``server``/``client``/``both``), only roles whose block declares that ``direction`` (or ``both``) are returned, and each entry additionally carries ``transport``, @@ -140,6 +143,7 @@ class LookupModule(LookupBase): str(direction_raw).strip().lower() if direction_raw is not None else None ) deployed = _deployed_roles(scope, vars_) + include_headless = bool(kwargs.get("include_headless", False)) tls_lookup = lookup_loader.get( "tls", loader=self._loader, templar=self._templar @@ -172,7 +176,7 @@ class LookupModule(LookupBase): if get_entity_name(str(role_id)) == service_name: continue canonical = _resolve_canonical_domain(str(role_id), app_config) - if not canonical and direction is None: + if not canonical and direction is None and not include_headless: continue canonical_url = "" if canonical: diff --git a/roles/desk-bluray-player/vars/main.yml b/roles/desk-bluray-player/vars/main.yml deleted file mode 100644 index 81afa1e2cc..0000000000 --- a/roles/desk-bluray-player/vars/main.yml +++ /dev/null @@ -1,2 +0,0 @@ ---- -application_id: desk-bluray-player diff --git a/roles/desk-chromium/vars/main.yml b/roles/desk-chromium/vars/main.yml deleted file mode 100644 index cc358c59fc..0000000000 --- a/roles/desk-chromium/vars/main.yml +++ /dev/null @@ -1,3 +0,0 @@ ---- - -application_id: "desk-chromium" diff --git a/roles/desk-docker/vars/main.yml b/roles/desk-docker/vars/main.yml deleted file mode 100644 index 55d675b60e..0000000000 --- a/roles/desk-docker/vars/main.yml +++ /dev/null @@ -1,2 +0,0 @@ ---- -application_id: desk-docker diff --git a/roles/desk-dotlinker/vars/main.yml b/roles/desk-dotlinker/vars/main.yml deleted file mode 100644 index e45e48e9c1..0000000000 --- a/roles/desk-dotlinker/vars/main.yml +++ /dev/null @@ -1,2 +0,0 @@ ---- -application_id: desk-dotlinker diff --git a/roles/desk-git/vars/main.yml b/roles/desk-git/vars/main.yml deleted file mode 100644 index 77d9d0452e..0000000000 --- a/roles/desk-git/vars/main.yml +++ /dev/null @@ -1,2 +0,0 @@ ---- -application_id: desk-git diff --git a/roles/desk-gnome-caffeine/tasks/main.yml b/roles/desk-gnome-caffeine/tasks/main.yml deleted file mode 100644 index 2cea32cf43..0000000000 --- a/roles/desk-gnome-caffeine/tasks/main.yml +++ /dev/null @@ -1,3 +0,0 @@ ---- -- include_tasks: 00_core.yml - when: run_once_desk_gnome_caffeine is not defined diff --git a/roles/desk-gnome-extensions/tasks/main.yml b/roles/desk-gnome-extensions/tasks/main.yml deleted file mode 100644 index e1cd60b232..0000000000 --- a/roles/desk-gnome-extensions/tasks/main.yml +++ /dev/null @@ -1,3 +0,0 @@ ---- -- include_tasks: 00_core.yml - when: run_once_desk_gnome_extensions is not defined diff --git a/roles/desk-gnome-terminal/tasks/main.yml b/roles/desk-gnome-terminal/tasks/main.yml deleted file mode 100644 index c0e0659c92..0000000000 --- a/roles/desk-gnome-terminal/tasks/main.yml +++ /dev/null @@ -1,3 +0,0 @@ ---- -- include_tasks: 00_core.yml - when: run_once_desk_gnome_terminal is not defined diff --git a/roles/desk-gnome-terminal/vars/main.yml b/roles/desk-gnome-terminal/vars/main.yml deleted file mode 100644 index f65a8128fc..0000000000 --- a/roles/desk-gnome-terminal/vars/main.yml +++ /dev/null @@ -1,2 +0,0 @@ ---- -application_id: desk-gnome-terminal diff --git a/roles/desk-gnome/tasks/main.yml b/roles/desk-gnome/tasks/main.yml deleted file mode 100644 index 7c23ac3b14..0000000000 --- a/roles/desk-gnome/tasks/main.yml +++ /dev/null @@ -1,3 +0,0 @@ ---- -- include_tasks: 00_core.yml - when: run_once_desk_gnome is not defined diff --git a/roles/desk-gnucash/vars/main.yml b/roles/desk-gnucash/vars/main.yml deleted file mode 100644 index a8d51b17ae..0000000000 --- a/roles/desk-gnucash/vars/main.yml +++ /dev/null @@ -1,2 +0,0 @@ ---- -application_id: desk-gnucash diff --git a/roles/desk-jrnl/vars/main.yml b/roles/desk-jrnl/vars/main.yml deleted file mode 100644 index dcbe9345b6..0000000000 --- a/roles/desk-jrnl/vars/main.yml +++ /dev/null @@ -1,2 +0,0 @@ ---- -application_id: desk-jrnl diff --git a/roles/desk-keepassxc/vars/main.yml b/roles/desk-keepassxc/vars/main.yml deleted file mode 100644 index 6745634147..0000000000 --- a/roles/desk-keepassxc/vars/main.yml +++ /dev/null @@ -1,2 +0,0 @@ ---- -application_id: desk-keepassxc diff --git a/roles/desk-obs/vars/main.yml b/roles/desk-obs/vars/main.yml deleted file mode 100644 index b3a3bf8f77..0000000000 --- a/roles/desk-obs/vars/main.yml +++ /dev/null @@ -1,2 +0,0 @@ ---- -application_id: desk-obs diff --git a/roles/desk-qbittorrent/vars/main.yml b/roles/desk-qbittorrent/vars/main.yml deleted file mode 100644 index 360df72b00..0000000000 --- a/roles/desk-qbittorrent/vars/main.yml +++ /dev/null @@ -1,2 +0,0 @@ ---- -application_id: desk-qbittorrent diff --git a/roles/desk-retroarch/vars/main.yml b/roles/desk-retroarch/vars/main.yml deleted file mode 100644 index 7a88b64df1..0000000000 --- a/roles/desk-retroarch/vars/main.yml +++ /dev/null @@ -1,2 +0,0 @@ ---- -application_id: desk-retroarch diff --git a/roles/desk-spotify/vars/main.yml b/roles/desk-spotify/vars/main.yml deleted file mode 100644 index 6c9e8b3a77..0000000000 --- a/roles/desk-spotify/vars/main.yml +++ /dev/null @@ -1,2 +0,0 @@ ---- -application_id: desk-spotify diff --git a/roles/desk-ssh/tasks/main.yml b/roles/desk-ssh/tasks/main.yml deleted file mode 100644 index 48545ea9c1..0000000000 --- a/roles/desk-ssh/tasks/main.yml +++ /dev/null @@ -1,3 +0,0 @@ ---- -- include_tasks: 00_core.yml - when: run_once_desk_ssh is not defined diff --git a/roles/desk-torbrowser/vars/main.yml b/roles/desk-torbrowser/vars/main.yml deleted file mode 100644 index 2232241f41..0000000000 --- a/roles/desk-torbrowser/vars/main.yml +++ /dev/null @@ -1,2 +0,0 @@ ---- -application_id: desk-torbrowser diff --git a/roles/desk-virtualbox/vars/main.yml b/roles/desk-virtualbox/vars/main.yml deleted file mode 100644 index b6abdfe19b..0000000000 --- a/roles/desk-virtualbox/vars/main.yml +++ /dev/null @@ -1,2 +0,0 @@ ---- -application_id: desk-virtualbox diff --git a/roles/desk-zoom/vars/main.yml b/roles/desk-zoom/vars/main.yml deleted file mode 100644 index dfa28993f8..0000000000 --- a/roles/desk-zoom/vars/main.yml +++ /dev/null @@ -1,2 +0,0 @@ ---- -application_id: desk-zoom diff --git a/roles/dev-java/README.md b/roles/dev-java/README.md index a9e2a9d421..c10c562a87 100644 --- a/roles/dev-java/README.md +++ b/roles/dev-java/README.md @@ -18,9 +18,9 @@ flowchart LR svc_java["java"] end subgraph dependents [Dependents] - dpt_desk_bluray_player["desk-bluray-player 💻 ⚙️"] + dpt_dsk_bluray_player["dsk-bluray-player 💻 ⚙️"] end - svc_java -- "1:1" --> dpt_desk_bluray_player + svc_java -- "1:1" --> dpt_dsk_bluray_player ``` Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. diff --git a/roles/dev-npm/meta/packages.yml b/roles/dev-npm/meta/packages.yml deleted file mode 100644 index be120f96bf..0000000000 --- a/roles/dev-npm/meta/packages.yml +++ /dev/null @@ -1,10 +0,0 @@ ---- -npm: - Archlinux: - - npm - Debian: - - npm - RedHat: - names: - - npm - virtual: true diff --git a/roles/desk-bluray-player/README.md b/roles/dsk-bluray-player/README.md similarity index 96% rename from roles/desk-bluray-player/README.md rename to roles/dsk-bluray-player/README.md index 4cf6bedf0c..707220aebc 100644 --- a/roles/desk-bluray-player/README.md +++ b/roles/dsk-bluray-player/README.md @@ -22,7 +22,7 @@ flowchart LR subgraph deps [Dependencies] dep_dev_java["dev-java 💻 ⚙️"] end - subgraph role [desk-bluray-player 💻] + subgraph role [dsk-bluray-player 💻] svc_bluray_player["bluray-player"] end dep_dev_java -- "1:1" --> svc_bluray_player @@ -57,7 +57,7 @@ Clone, set up the workstation, and deploy Blue-ray Disc onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-bluray-player full_cycle=false +make compose-deploy mode=reinstall apps=dsk-bluray-player full_cycle=false ``` ### Production @@ -71,7 +71,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-bluray-player +APP=dsk-bluray-player DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-bluray-player/meta/info.yml b/roles/dsk-bluray-player/meta/info.yml similarity index 100% rename from roles/desk-bluray-player/meta/info.yml rename to roles/dsk-bluray-player/meta/info.yml diff --git a/roles/desk-bluray-player/meta/main.yml b/roles/dsk-bluray-player/meta/main.yml similarity index 100% rename from roles/desk-bluray-player/meta/main.yml rename to roles/dsk-bluray-player/meta/main.yml diff --git a/roles/desk-bluray-player/meta/packages.yml b/roles/dsk-bluray-player/meta/packages.yml similarity index 100% rename from roles/desk-bluray-player/meta/packages.yml rename to roles/dsk-bluray-player/meta/packages.yml diff --git a/roles/desk-bluray-player/meta/services.yml b/roles/dsk-bluray-player/meta/services.yml similarity index 100% rename from roles/desk-bluray-player/meta/services.yml rename to roles/dsk-bluray-player/meta/services.yml diff --git a/roles/desk-bluray-player/tasks/main.yml b/roles/dsk-bluray-player/tasks/main.yml similarity index 100% rename from roles/desk-bluray-player/tasks/main.yml rename to roles/dsk-bluray-player/tasks/main.yml diff --git a/roles/dsk-bluray-player/vars/main.yml b/roles/dsk-bluray-player/vars/main.yml new file mode 100644 index 0000000000..97e22637ec --- /dev/null +++ b/roles/dsk-bluray-player/vars/main.yml @@ -0,0 +1,2 @@ +--- +application_id: dsk-bluray-player diff --git a/roles/desk-chromium/README.md b/roles/dsk-chromium/README.md similarity index 97% rename from roles/desk-chromium/README.md rename to roles/dsk-chromium/README.md index c3ba4428aa..28d7c7f8a5 100644 --- a/roles/desk-chromium/README.md +++ b/roles/dsk-chromium/README.md @@ -14,7 +14,7 @@ The diagram places Chromium in the Infinito.Nexus cosmos: the components it depl ```mermaid flowchart LR - subgraph role [desk-chromium 💻] + subgraph role [dsk-chromium 💻] svc_chromium["chromium"] end ``` @@ -43,7 +43,7 @@ Clone, set up the workstation, and deploy Chromium onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-chromium full_cycle=false +make compose-deploy mode=reinstall apps=dsk-chromium full_cycle=false ``` ### Production @@ -57,7 +57,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-chromium +APP=dsk-chromium DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-chromium/meta/addons/keepassxc-browser.yml b/roles/dsk-chromium/meta/addons/keepassxc-browser.yml similarity index 100% rename from roles/desk-chromium/meta/addons/keepassxc-browser.yml rename to roles/dsk-chromium/meta/addons/keepassxc-browser.yml diff --git a/roles/desk-chromium/meta/addons/privacy-badger.yml b/roles/dsk-chromium/meta/addons/privacy-badger.yml similarity index 100% rename from roles/desk-chromium/meta/addons/privacy-badger.yml rename to roles/dsk-chromium/meta/addons/privacy-badger.yml diff --git a/roles/desk-chromium/meta/addons/ublock-origin.yml b/roles/dsk-chromium/meta/addons/ublock-origin.yml similarity index 100% rename from roles/desk-chromium/meta/addons/ublock-origin.yml rename to roles/dsk-chromium/meta/addons/ublock-origin.yml diff --git a/roles/desk-chromium/meta/info.yml b/roles/dsk-chromium/meta/info.yml similarity index 100% rename from roles/desk-chromium/meta/info.yml rename to roles/dsk-chromium/meta/info.yml diff --git a/roles/desk-chromium/meta/main.yml b/roles/dsk-chromium/meta/main.yml similarity index 100% rename from roles/desk-chromium/meta/main.yml rename to roles/dsk-chromium/meta/main.yml diff --git a/roles/desk-chromium/meta/packages.yml b/roles/dsk-chromium/meta/packages.yml similarity index 100% rename from roles/desk-chromium/meta/packages.yml rename to roles/dsk-chromium/meta/packages.yml diff --git a/roles/desk-chromium/meta/services.yml b/roles/dsk-chromium/meta/services.yml similarity index 100% rename from roles/desk-chromium/meta/services.yml rename to roles/dsk-chromium/meta/services.yml diff --git a/roles/desk-chromium/tasks/main.yml b/roles/dsk-chromium/tasks/main.yml similarity index 80% rename from roles/desk-chromium/tasks/main.yml rename to roles/dsk-chromium/tasks/main.yml index 8effd2fe30..b012d714b7 100644 --- a/roles/desk-chromium/tasks/main.yml +++ b/roles/dsk-chromium/tasks/main.yml @@ -3,10 +3,10 @@ package_install: id: chromium -- name: 📂 Include dependency 'desk-keepassxc' +- name: 📂 Include dependency 'dsk-keepassxc' include_role: - name: desk-keepassxc - when: run_once_desk_keepassxc is not defined + name: dsk-keepassxc + when: run_once_dsk_keepassxc is not defined - name: ✅ Ensure Chromium policies directory exists file: diff --git a/roles/desk-chromium/templates/extensions_policy.json.j2 b/roles/dsk-chromium/templates/extensions_policy.json.j2 similarity index 100% rename from roles/desk-chromium/templates/extensions_policy.json.j2 rename to roles/dsk-chromium/templates/extensions_policy.json.j2 diff --git a/roles/dsk-chromium/vars/main.yml b/roles/dsk-chromium/vars/main.yml new file mode 100644 index 0000000000..2a583daf21 --- /dev/null +++ b/roles/dsk-chromium/vars/main.yml @@ -0,0 +1,3 @@ +--- + +application_id: "dsk-chromium" diff --git a/roles/dsk-code/README.md b/roles/dsk-code/README.md new file mode 100644 index 0000000000..c855683b1a --- /dev/null +++ b/roles/dsk-code/README.md @@ -0,0 +1,86 @@ +# Code + +## Description + +[Visual Studio Code](https://github.com/microsoft/vscode) in a selectable flavor: the distribution's Code - OSS build, VSCodium, or Microsoft's own binary. + +## Overview + +`services.code.flavor` picks one of `code-oss`, `vscodium` or `vscode`. The three builds conflict at package level, so the role removes the other two before installing the chosen one. + +The role also carries the editor extensions of the agents the same host deploys: when `dsk-gnt-claude` is in the host's groups, the Claude Code extension is installed into the chosen flavor. Code - OSS and VSCodium resolve extensions through Open VSX, which carries that extension as well. + +`dsk-gnt-cursor` and `dsk-gnt-pi` carry no entry in that map. Cursor is a full editor rather than an extension, and neither vendor publishes an extension on Open VSX, so there is nothing to install into Code for them. Both agents are used from their own binary. + +## Cosmos + +The diagram places Code in the Infinito.Nexus cosmos: the components it deploys (capabilities), the central services it consumes (dependencies), and its outward reach (federation and bridged external networks). + +```mermaid +flowchart LR + subgraph deps [Dependencies] + dep_dsk_gnt_claude["dsk-gnt-claude 💻"] + dep_dsk_gnt_codex["dsk-gnt-codex 💻"] + end + subgraph role [dsk-code 💻] + svc_code["code"] + svc_claude["claude"] + svc_codex["codex"] + end + dep_dsk_gnt_claude -. "0..1" .-> svc_claude + dep_dsk_gnt_codex -. "0..1" .-> svc_codex +``` + +Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. + +## Features + +- **Flavor selectable:** one setting switches between Code - OSS, VSCodium and the Microsoft build. +- **Conflict-free switch:** the other flavors are removed first, because they claim the same binary. +- **Agent-aware:** an agent role on the same host brings its editor extension along. +- **Refused early:** an unknown flavor aborts the deploy with the list of carried flavors. + +## Quick Setup + +### Development + +Clone, set up the workstation, and deploy Code onto the local stack: + +```bash +git clone https://github.com/infinito-nexus/core.git +cd core +make onboard +make compose-deploy mode=reinstall apps=dsk-code full_cycle=false +``` + +### Production + +Install Code directly onto the target machine: clone the repository, install the OS prerequisites and the repository toolchain, then deploy against localhost over a local connection (no SSH, no container): + +```bash +git clone https://github.com/infinito-nexus/core.git +cd core +bash scripts/install/package.sh +make install +source scripts/meta/env/load.sh + +APP=dsk-code +DOMAIN= +TLS_MODE=self_signed +SSH_PUBLIC_KEY="" +INVENTORY=inventories/production +infinito administration inventory provision "$INVENTORY" \ + --inventory-file "$INVENTORY/devices.yml" \ + --host localhost \ + --include "$APP" \ + --vars "{\"TLS_MODE\": \"$TLS_MODE\", \"DOMAIN_PRIMARY\": \"$DOMAIN\", \"users\": {\"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}" +infinito administration deploy dedicated "$INVENTORY/devices.yml" \ + --password-file "$INVENTORY/.password" \ + --diff -vv +``` + +## Credits + +Implemented by **[Kevin Veen-Birkenbach](https://www.veen.world)**. +Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world). +Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license). diff --git a/roles/dsk-code/meta/addons/claude-code.yml b/roles/dsk-code/meta/addons/claude-code.yml new file mode 100644 index 0000000000..299df86833 --- /dev/null +++ b/roles/dsk-code/meta/addons/claude-code.yml @@ -0,0 +1,7 @@ +--- +required: false +enabled: "{{ lookup('config', 'dsk-code', 'services.claude.enabled') | bool }}" +mechanism: extension +source: upstream +config: + extension_id: anthropic.claude-code diff --git a/roles/dsk-code/meta/addons/codex.yml b/roles/dsk-code/meta/addons/codex.yml new file mode 100644 index 0000000000..be4dc6b764 --- /dev/null +++ b/roles/dsk-code/meta/addons/codex.yml @@ -0,0 +1,7 @@ +--- +required: false +enabled: "{{ lookup('config', 'dsk-code', 'services.codex.enabled') | bool }}" +mechanism: extension +source: upstream +config: + extension_id: openai.chatgpt diff --git a/roles/dsk-code/meta/info.yml b/roles/dsk-code/meta/info.yml new file mode 100644 index 0000000000..0690fbcdf0 --- /dev/null +++ b/roles/dsk-code/meta/info.yml @@ -0,0 +1,5 @@ +--- +# nocheck: info-media no official demo video +logo: + class: fas fa-code +homepage: https://github.com/microsoft/vscode diff --git a/roles/dsk-code/meta/main.yml b/roles/dsk-code/meta/main.yml new file mode 100644 index 0000000000..634f568934 --- /dev/null +++ b/roles/dsk-code/meta/main.yml @@ -0,0 +1,32 @@ +--- +galaxy_info: + author: Kevin Veen-Birkenbach + description: Visual Studio Code workstation editor in a selectable flavor, carrying the extensions of the agents the host deploys + license: Infinito.Nexus Community License (Non-Commercial) + company: | + Kevin Veen-Birkenbach + https://www.veen.world + galaxy_tags: + - editor + - ide + - vscode + - workstation + issue_tracker_url: https://s.infinito.nexus/issues + min_ansible_version: "2.9" + platforms: + - name: ArchLinux + versions: + - all + - name: Debian + versions: + - all + - name: EL + versions: + - all + - name: Fedora + versions: + - all + - name: Ubuntu + versions: + - all +dependencies: [] diff --git a/roles/dsk-code/meta/packages.yml b/roles/dsk-code/meta/packages.yml new file mode 100644 index 0000000000..9d6b5edcd0 --- /dev/null +++ b/roles/dsk-code/meta/packages.yml @@ -0,0 +1,22 @@ +--- +code-oss: + Archlinux: + - code + Debian: [] # nocheck: empty - Debian and Ubuntu archive no Code - OSS build; TODO: install the VSCodium apt repository build through a repo source + RedHat: [] # nocheck: empty - Fedora and CentOS Stream archive no Code - OSS build; TODO: install the VSCodium rpm repository build through a repo source + +vscode: + Archlinux: + source: aur + names: + - visual-studio-code-bin + Debian: [] # nocheck: empty - Visual Studio Code is proprietary and shipped only from Microsoft's own apt repository; TODO: add that repository through a repo source + RedHat: [] # nocheck: empty - Visual Studio Code is proprietary and shipped only from Microsoft's own rpm repository; TODO: add that repository through a repo source + +vscodium: + Archlinux: + source: aur + names: + - vscodium-bin + Debian: [] # nocheck: empty - VSCodium ships its own signed apt repository rather than a Debian or Ubuntu package; TODO: add that repository through a repo source + RedHat: [] # nocheck: empty - VSCodium ships its own signed rpm repository rather than a Fedora or EPEL package; TODO: add that repository through a repo source diff --git a/roles/dsk-code/meta/services.yml b/roles/dsk-code/meta/services.yml new file mode 100644 index 0000000000..18991ff7f2 --- /dev/null +++ b/roles/dsk-code/meta/services.yml @@ -0,0 +1,22 @@ +--- +code: + modes: + host: + enabled: true + lifecycle: alpha + flavor: code-oss + min_storage: "1GB" + cpus: "2.0" + mem_reservation: "512m" + mem_limit: "4g" + pids_limit: 1024 +# nocheck: playwright-service-flag, dynamic-flag workstation agent, carried as an editor extension when the host deploys it +claude: + bond: 0.5 + enabled: "{{ 'dsk-gnt-claude' in group_names }}" + shared: "{{ 'dsk-gnt-claude' in group_names }}" +# nocheck: playwright-service-flag, dynamic-flag workstation agent, carried as an editor extension when the host deploys it +codex: + bond: 0.5 + enabled: "{{ 'dsk-gnt-codex' in group_names }}" + shared: "{{ 'dsk-gnt-codex' in group_names }}" diff --git a/roles/dsk-code/tasks/00_core.yml b/roles/dsk-code/tasks/00_core.yml new file mode 100644 index 0000000000..090000b476 --- /dev/null +++ b/roles/dsk-code/tasks/00_core.yml @@ -0,0 +1,34 @@ +--- +- name: "🛑 Refuse a flavor the role does not carry" + ansible.builtin.assert: + that: + - code_flavor in code_flavors + fail_msg: "services.code.flavor is '{{ code_flavor }}'; carried flavors are {{ code_flavors.keys() | list | join(', ') }}." + quiet: true + +- name: "🧹 Remove the flavors that conflict with {{ code_flavor }}" + package_install: + id: "{{ item }}" + state: absent + loop: "{{ code_conflicting_flavors }}" + +- name: "📦 install {{ code_flavor }}" + package_install: + id: "{{ code_flavor }}" + register: code_install + +- name: "🧩 Install the extension of an agent this host deploys" + ansible.builtin.command: + cmd: "{{ code_binary }} --install-extension {{ item.value.config.extension_id }} --force" + become: false + become_user: "{{ lookup('users', WORKSTATION_USER).username }}" + register: code_extension + changed_when: "'successfully installed' in (code_extension.stdout | lower)" + loop: "{{ code_addons | dict2items }}" + loop_control: + label: "{{ item.key }}" + when: + - code_install is not skipped + - item.value.enabled | bool + +- include_tasks: utils/once/flag.yml diff --git a/roles/dsk-code/tasks/addons/claude-code.yml b/roles/dsk-code/tasks/addons/claude-code.yml new file mode 100644 index 0000000000..ed97d539c0 --- /dev/null +++ b/roles/dsk-code/tasks/addons/claude-code.yml @@ -0,0 +1 @@ +--- diff --git a/roles/dsk-code/tasks/addons/codex.yml b/roles/dsk-code/tasks/addons/codex.yml new file mode 100644 index 0000000000..ed97d539c0 --- /dev/null +++ b/roles/dsk-code/tasks/addons/codex.yml @@ -0,0 +1 @@ +--- diff --git a/roles/dsk-code/tasks/main.yml b/roles/dsk-code/tasks/main.yml new file mode 100644 index 0000000000..ec9e88054d --- /dev/null +++ b/roles/dsk-code/tasks/main.yml @@ -0,0 +1,3 @@ +--- +- include_tasks: 00_core.yml + when: run_once_dsk_code is not defined diff --git a/roles/dsk-code/vars/main.yml b/roles/dsk-code/vars/main.yml new file mode 100644 index 0000000000..dd3b9c5232 --- /dev/null +++ b/roles/dsk-code/vars/main.yml @@ -0,0 +1,13 @@ +--- +application_id: dsk-code +code_flavor: "{{ lookup('config', application_id, 'services.code.flavor') }}" +code_flavors: + code-oss: + binary: code + vscodium: + binary: codium + vscode: + binary: code +code_binary: "{{ code_flavors[code_flavor].binary }}" +code_conflicting_flavors: "{{ code_flavors.keys() | reject('equalto', code_flavor) | list }}" +code_addons: "{{ lookup('config', application_id, 'addons', {}) }}" diff --git a/roles/desk-copyq/README.md b/roles/dsk-copyq/README.md similarity index 95% rename from roles/desk-copyq/README.md rename to roles/dsk-copyq/README.md index 5dee8ba69f..da81e450f9 100644 --- a/roles/desk-copyq/README.md +++ b/roles/dsk-copyq/README.md @@ -14,7 +14,7 @@ The diagram places CopyQ in the Infinito.Nexus cosmos: the components it deploys ```mermaid flowchart LR - subgraph role [desk-copyq 💻] + subgraph role [dsk-copyq 💻] svc_copyq["copyq"] end ``` @@ -35,7 +35,7 @@ Clone, set up the workstation, and deploy CopyQ onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-copyq full_cycle=false +make compose-deploy mode=reinstall apps=dsk-copyq full_cycle=false ``` ### Production @@ -49,7 +49,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-copyq +APP=dsk-copyq DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-copyq/meta/info.yml b/roles/dsk-copyq/meta/info.yml similarity index 100% rename from roles/desk-copyq/meta/info.yml rename to roles/dsk-copyq/meta/info.yml diff --git a/roles/desk-copyq/meta/main.yml b/roles/dsk-copyq/meta/main.yml similarity index 100% rename from roles/desk-copyq/meta/main.yml rename to roles/dsk-copyq/meta/main.yml diff --git a/roles/desk-copyq/meta/packages.yml b/roles/dsk-copyq/meta/packages.yml similarity index 100% rename from roles/desk-copyq/meta/packages.yml rename to roles/dsk-copyq/meta/packages.yml diff --git a/roles/desk-copyq/meta/services.yml b/roles/dsk-copyq/meta/services.yml similarity index 100% rename from roles/desk-copyq/meta/services.yml rename to roles/dsk-copyq/meta/services.yml diff --git a/roles/desk-copyq/tasks/main.yml b/roles/dsk-copyq/tasks/main.yml similarity index 100% rename from roles/desk-copyq/tasks/main.yml rename to roles/dsk-copyq/tasks/main.yml diff --git a/roles/desk-copyq/tasks/server.yml b/roles/dsk-copyq/tasks/server.yml similarity index 100% rename from roles/desk-copyq/tasks/server.yml rename to roles/dsk-copyq/tasks/server.yml diff --git a/roles/desk-copyq/vars/main.yml b/roles/dsk-copyq/vars/main.yml similarity index 78% rename from roles/desk-copyq/vars/main.yml rename to roles/dsk-copyq/vars/main.yml index e759fac8f1..c958ef6581 100644 --- a/roles/desk-copyq/vars/main.yml +++ b/roles/dsk-copyq/vars/main.yml @@ -1,3 +1,3 @@ --- -application_id: desk-copyq +application_id: dsk-copyq copyq_server_enabled: "{{ lookup('config', application_id, 'services.copyq.server_enabled') }}" diff --git a/roles/desk-docker/README.md b/roles/dsk-docker/README.md similarity index 86% rename from roles/desk-docker/README.md rename to roles/dsk-docker/README.md index 0a3ce58367..07a114d983 100644 --- a/roles/desk-docker/README.md +++ b/roles/dsk-docker/README.md @@ -6,7 +6,7 @@ Installs Docker and Docker Compose, and adds a user to the Docker group for non- ## Overview -This playbook, `desk-docker`, is part of a larger collection housed within the `infinito` repository. It is specifically tailored for setting up Docker and Docker Compose on personal computers (PCs) used for development purposes. The primary goal is to facilitate a development environment on individual workstations rather than configuring servers for hosting or distributing Docker images. +This playbook, `dsk-docker`, is part of a larger collection housed within the `infinito` repository. It is specifically tailored for setting up Docker and Docker Compose on personal computers (PCs) used for development purposes. The primary goal is to facilitate a development environment on individual workstations rather than configuring servers for hosting or distributing Docker images. ## Cosmos @@ -14,7 +14,7 @@ The diagram places Workstation Docker in the Infinito.Nexus cosmos: the componen ```mermaid flowchart LR - subgraph role [desk-docker 💻] + subgraph role [dsk-docker 💻] svc_docker["docker"] end ``` @@ -23,7 +23,7 @@ Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional ( ## Features -The `main.yml` file in the `desk-docker` role consists of two primary tasks: +The `main.yml` file in the `dsk-docker` role consists of two primary tasks: 1. **Install Docker**: This task uses the `community.general.pacman` module to install `docker` and `docker-compose` on the system. It ensures that these packages are present on the PC. @@ -39,7 +39,7 @@ Clone, set up the workstation, and deploy Workstation Docker onto the local stac git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-docker full_cycle=false +make compose-deploy mode=reinstall apps=dsk-docker full_cycle=false ``` ### Production @@ -53,7 +53,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-docker +APP=dsk-docker DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" @@ -82,7 +82,7 @@ The playbook is designed for developers who require Docker in their local develo To run this playbook: 1. Clone the `infinito` repository. -2. Navigate to the `roles/desk-docker` directory. +2. Navigate to the `roles/dsk-docker` directory. 3. Run the playbook using the appropriate Ansible commands, ensuring that you have the necessary privileges. ## Important Notes diff --git a/roles/desk-docker/meta/info.yml b/roles/dsk-docker/meta/info.yml similarity index 100% rename from roles/desk-docker/meta/info.yml rename to roles/dsk-docker/meta/info.yml diff --git a/roles/desk-docker/meta/main.yml b/roles/dsk-docker/meta/main.yml similarity index 100% rename from roles/desk-docker/meta/main.yml rename to roles/dsk-docker/meta/main.yml diff --git a/roles/desk-docker/meta/services.yml b/roles/dsk-docker/meta/services.yml similarity index 100% rename from roles/desk-docker/meta/services.yml rename to roles/dsk-docker/meta/services.yml diff --git a/roles/desk-docker/tasks/main.yml b/roles/dsk-docker/tasks/main.yml similarity index 100% rename from roles/desk-docker/tasks/main.yml rename to roles/dsk-docker/tasks/main.yml diff --git a/roles/dsk-docker/vars/main.yml b/roles/dsk-docker/vars/main.yml new file mode 100644 index 0000000000..53e0885226 --- /dev/null +++ b/roles/dsk-docker/vars/main.yml @@ -0,0 +1,2 @@ +--- +application_id: dsk-docker diff --git a/roles/desk-dotlinker/README.md b/roles/dsk-dotlinker/README.md similarity index 96% rename from roles/desk-dotlinker/README.md rename to roles/dsk-dotlinker/README.md index 384e8e5616..1aa30b0acc 100644 --- a/roles/desk-dotlinker/README.md +++ b/roles/dsk-dotlinker/README.md @@ -18,7 +18,7 @@ The diagram places Dotlinker in the Infinito.Nexus cosmos: the components it dep ```mermaid flowchart LR - subgraph role [desk-dotlinker 💻] + subgraph role [dsk-dotlinker 💻] svc_dotlinker["dotlinker"] end ``` @@ -39,7 +39,7 @@ Clone, set up the workstation, and deploy Dotlinker onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-dotlinker full_cycle=false +make compose-deploy mode=reinstall apps=dsk-dotlinker full_cycle=false ``` ### Production @@ -53,7 +53,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-dotlinker +APP=dsk-dotlinker DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-dotlinker/defaults/main.yml b/roles/dsk-dotlinker/defaults/main.yml similarity index 100% rename from roles/desk-dotlinker/defaults/main.yml rename to roles/dsk-dotlinker/defaults/main.yml diff --git a/roles/desk-dotlinker/meta/info.yml b/roles/dsk-dotlinker/meta/info.yml similarity index 100% rename from roles/desk-dotlinker/meta/info.yml rename to roles/dsk-dotlinker/meta/info.yml diff --git a/roles/desk-dotlinker/meta/main.yml b/roles/dsk-dotlinker/meta/main.yml similarity index 100% rename from roles/desk-dotlinker/meta/main.yml rename to roles/dsk-dotlinker/meta/main.yml diff --git a/roles/desk-dotlinker/meta/services.yml b/roles/dsk-dotlinker/meta/services.yml similarity index 100% rename from roles/desk-dotlinker/meta/services.yml rename to roles/dsk-dotlinker/meta/services.yml diff --git a/roles/desk-dotlinker/tasks/01_install.yml b/roles/dsk-dotlinker/tasks/01_install.yml similarity index 100% rename from roles/desk-dotlinker/tasks/01_install.yml rename to roles/dsk-dotlinker/tasks/01_install.yml diff --git a/roles/desk-dotlinker/tasks/02_apply.yml b/roles/dsk-dotlinker/tasks/02_apply.yml similarity index 100% rename from roles/desk-dotlinker/tasks/02_apply.yml rename to roles/dsk-dotlinker/tasks/02_apply.yml diff --git a/roles/desk-dotlinker/tasks/main.yml b/roles/dsk-dotlinker/tasks/main.yml similarity index 76% rename from roles/desk-dotlinker/tasks/main.yml rename to roles/dsk-dotlinker/tasks/main.yml index 1b96503aff..52b5d61ba9 100644 --- a/roles/desk-dotlinker/tasks/main.yml +++ b/roles/dsk-dotlinker/tasks/main.yml @@ -1,6 +1,6 @@ --- # Install ONCE per play/host, but allow apply multiple times with different mappings - include_tasks: 01_install.yml - when: run_once_desk_dotlinker is not defined + when: run_once_dsk_dotlinker is not defined - include_tasks: 02_apply.yml diff --git a/roles/dsk-dotlinker/vars/main.yml b/roles/dsk-dotlinker/vars/main.yml new file mode 100644 index 0000000000..455ef343e8 --- /dev/null +++ b/roles/dsk-dotlinker/vars/main.yml @@ -0,0 +1,2 @@ +--- +application_id: dsk-dotlinker diff --git a/roles/desk-firefox/README.md b/roles/dsk-firefox/README.md similarity index 97% rename from roles/desk-firefox/README.md rename to roles/dsk-firefox/README.md index a97d6303c6..fe17500981 100644 --- a/roles/desk-firefox/README.md +++ b/roles/dsk-firefox/README.md @@ -14,7 +14,7 @@ The diagram places Firefox in the Infinito.Nexus cosmos: the components it deplo ```mermaid flowchart LR - subgraph role [desk-firefox 💻] + subgraph role [dsk-firefox 💻] svc_firefox["firefox"] end ``` @@ -42,7 +42,7 @@ Clone, set up the workstation, and deploy Firefox onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-firefox full_cycle=false +make compose-deploy mode=reinstall apps=dsk-firefox full_cycle=false ``` ### Production @@ -56,7 +56,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-firefox +APP=dsk-firefox DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-firefox/meta/addons/keepassxc-browser.yml b/roles/dsk-firefox/meta/addons/keepassxc-browser.yml similarity index 100% rename from roles/desk-firefox/meta/addons/keepassxc-browser.yml rename to roles/dsk-firefox/meta/addons/keepassxc-browser.yml diff --git a/roles/desk-firefox/meta/addons/ublock-origin.yml b/roles/dsk-firefox/meta/addons/ublock-origin.yml similarity index 100% rename from roles/desk-firefox/meta/addons/ublock-origin.yml rename to roles/dsk-firefox/meta/addons/ublock-origin.yml diff --git a/roles/desk-firefox/meta/info.yml b/roles/dsk-firefox/meta/info.yml similarity index 100% rename from roles/desk-firefox/meta/info.yml rename to roles/dsk-firefox/meta/info.yml diff --git a/roles/desk-firefox/meta/main.yml b/roles/dsk-firefox/meta/main.yml similarity index 100% rename from roles/desk-firefox/meta/main.yml rename to roles/dsk-firefox/meta/main.yml diff --git a/roles/desk-firefox/meta/packages.yml b/roles/dsk-firefox/meta/packages.yml similarity index 100% rename from roles/desk-firefox/meta/packages.yml rename to roles/dsk-firefox/meta/packages.yml diff --git a/roles/desk-firefox/meta/services.yml b/roles/dsk-firefox/meta/services.yml similarity index 100% rename from roles/desk-firefox/meta/services.yml rename to roles/dsk-firefox/meta/services.yml diff --git a/roles/desk-firefox/tasks/main.yml b/roles/dsk-firefox/tasks/main.yml similarity index 100% rename from roles/desk-firefox/tasks/main.yml rename to roles/dsk-firefox/tasks/main.yml diff --git a/roles/desk-firefox/templates/policies.json.j2 b/roles/dsk-firefox/templates/policies.json.j2 similarity index 100% rename from roles/desk-firefox/templates/policies.json.j2 rename to roles/dsk-firefox/templates/policies.json.j2 diff --git a/roles/desk-firefox/vars/main.yml b/roles/dsk-firefox/vars/main.yml similarity index 84% rename from roles/desk-firefox/vars/main.yml rename to roles/dsk-firefox/vars/main.yml index 362573d2c9..0db68a0b18 100644 --- a/roles/desk-firefox/vars/main.yml +++ b/roles/dsk-firefox/vars/main.yml @@ -1,7 +1,7 @@ --- # Default variables for the pc-firefox role -application_id: "desk-firefox" +application_id: "dsk-firefox" firefox_policy_dir: "/usr/lib/firefox/distribution" diff --git a/roles/desk-git/README.md b/roles/dsk-git/README.md similarity index 94% rename from roles/desk-git/README.md rename to roles/dsk-git/README.md index e3bc8c7ab4..450e35a0c3 100644 --- a/roles/desk-git/README.md +++ b/roles/dsk-git/README.md @@ -14,13 +14,13 @@ The diagram places Git in the Infinito.Nexus cosmos: the components it deploys ( ```mermaid flowchart LR - subgraph role [desk-git 💻] + subgraph role [dsk-git 💻] svc_git["git"] end subgraph dependents [Dependents] - dpt_desk_ssh["desk-ssh 💻 ⚙️"] + dpt_dsk_ssh["dsk-ssh 💻 ⚙️"] end - svc_git -- "1:1" --> dpt_desk_ssh + svc_git -- "1:1" --> dpt_dsk_ssh ``` Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. @@ -46,7 +46,7 @@ Clone, set up the workstation, and deploy Git onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-git full_cycle=false +make compose-deploy mode=reinstall apps=dsk-git full_cycle=false ``` ### Production @@ -60,7 +60,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-git +APP=dsk-git DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-git/meta/info.yml b/roles/dsk-git/meta/info.yml similarity index 100% rename from roles/desk-git/meta/info.yml rename to roles/dsk-git/meta/info.yml diff --git a/roles/desk-git/meta/main.yml b/roles/dsk-git/meta/main.yml similarity index 100% rename from roles/desk-git/meta/main.yml rename to roles/dsk-git/meta/main.yml diff --git a/roles/desk-git/meta/services.yml b/roles/dsk-git/meta/services.yml similarity index 100% rename from roles/desk-git/meta/services.yml rename to roles/dsk-git/meta/services.yml diff --git a/roles/desk-git/tasks/main.yml b/roles/dsk-git/tasks/main.yml similarity index 96% rename from roles/desk-git/tasks/main.yml rename to roles/dsk-git/tasks/main.yml index 011b2ff645..1cc468972f 100644 --- a/roles/desk-git/tasks/main.yml +++ b/roles/dsk-git/tasks/main.yml @@ -10,7 +10,7 @@ package_name: gitcon global_install: true -- when: run_once_desk_git is not defined +- when: run_once_dsk_git is not defined block: - name: 🔧 setup git (as workstation user) ansible.builtin.command: diff --git a/roles/dsk-git/vars/main.yml b/roles/dsk-git/vars/main.yml new file mode 100644 index 0000000000..5c9c9819e9 --- /dev/null +++ b/roles/dsk-git/vars/main.yml @@ -0,0 +1,2 @@ +--- +application_id: dsk-git diff --git a/roles/desk-gnome-caffeine/README.md b/roles/dsk-gnome-caffeine/README.md similarity index 95% rename from roles/desk-gnome-caffeine/README.md rename to roles/dsk-gnome-caffeine/README.md index 2bc12068da..e2b7edfa02 100644 --- a/roles/desk-gnome-caffeine/README.md +++ b/roles/dsk-gnome-caffeine/README.md @@ -14,7 +14,7 @@ The diagram places GNOME Caffeine in the Infinito.Nexus cosmos: the components i ```mermaid flowchart LR - subgraph role [desk-gnome-caffeine 💻] + subgraph role [dsk-gnome-caffeine 💻] svc_gnome_caffeine["gnome-caffeine"] end ``` @@ -42,7 +42,7 @@ Clone, set up the workstation, and deploy GNOME Caffeine onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-gnome-caffeine full_cycle=false +make compose-deploy mode=reinstall apps=dsk-gnome-caffeine full_cycle=false ``` ### Production @@ -56,7 +56,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-gnome-caffeine +APP=dsk-gnome-caffeine DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-gnome-caffeine/meta/info.yml b/roles/dsk-gnome-caffeine/meta/info.yml similarity index 100% rename from roles/desk-gnome-caffeine/meta/info.yml rename to roles/dsk-gnome-caffeine/meta/info.yml diff --git a/roles/desk-gnome-caffeine/meta/main.yml b/roles/dsk-gnome-caffeine/meta/main.yml similarity index 100% rename from roles/desk-gnome-caffeine/meta/main.yml rename to roles/dsk-gnome-caffeine/meta/main.yml diff --git a/roles/desk-gnome-caffeine/meta/packages.yml b/roles/dsk-gnome-caffeine/meta/packages.yml similarity index 100% rename from roles/desk-gnome-caffeine/meta/packages.yml rename to roles/dsk-gnome-caffeine/meta/packages.yml diff --git a/roles/desk-gnome-caffeine/meta/services.yml b/roles/dsk-gnome-caffeine/meta/services.yml similarity index 100% rename from roles/desk-gnome-caffeine/meta/services.yml rename to roles/dsk-gnome-caffeine/meta/services.yml diff --git a/roles/desk-gnome-caffeine/tasks/00_core.yml b/roles/dsk-gnome-caffeine/tasks/00_core.yml similarity index 100% rename from roles/desk-gnome-caffeine/tasks/00_core.yml rename to roles/dsk-gnome-caffeine/tasks/00_core.yml diff --git a/roles/dsk-gnome-caffeine/tasks/main.yml b/roles/dsk-gnome-caffeine/tasks/main.yml new file mode 100644 index 0000000000..0470a802af --- /dev/null +++ b/roles/dsk-gnome-caffeine/tasks/main.yml @@ -0,0 +1,3 @@ +--- +- include_tasks: 00_core.yml + when: run_once_dsk_gnome_caffeine is not defined diff --git a/roles/desk-gnome-caffeine/templates/caffeine.desktop.j2 b/roles/dsk-gnome-caffeine/templates/caffeine.desktop.j2 similarity index 100% rename from roles/desk-gnome-caffeine/templates/caffeine.desktop.j2 rename to roles/dsk-gnome-caffeine/templates/caffeine.desktop.j2 diff --git a/roles/desk-gnome-caffeine/vars/main.yml b/roles/dsk-gnome-caffeine/vars/main.yml similarity index 73% rename from roles/desk-gnome-caffeine/vars/main.yml rename to roles/dsk-gnome-caffeine/vars/main.yml index 54a1786e12..913f46605b 100644 --- a/roles/desk-gnome-caffeine/vars/main.yml +++ b/roles/dsk-gnome-caffeine/vars/main.yml @@ -1,3 +1,3 @@ --- auto_start_directory: /home/{{ lookup('users', WORKSTATION_USER).username }}/.config/autostart/ -application_id: desk-gnome-caffeine +application_id: dsk-gnome-caffeine diff --git a/roles/desk-gnome-extensions/README.md b/roles/dsk-gnome-extensions/README.md similarity index 89% rename from roles/desk-gnome-extensions/README.md rename to roles/dsk-gnome-extensions/README.md index 9fb409b7e9..502ebdb459 100644 --- a/roles/desk-gnome-extensions/README.md +++ b/roles/dsk-gnome-extensions/README.md @@ -16,7 +16,7 @@ The diagram places GNOME Extensions Manager in the Infinito.Nexus cosmos: the co ```mermaid flowchart LR - subgraph role [desk-gnome-extensions 💻] + subgraph role [dsk-gnome-extensions 💻] svc_gnome_extensions["gnome-extensions"] end ``` @@ -44,7 +44,7 @@ Clone, set up the workstation, and deploy GNOME Extensions Manager onto the loca git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-gnome-extensions full_cycle=false +make compose-deploy mode=reinstall apps=dsk-gnome-extensions full_cycle=false ``` ### Production @@ -58,7 +58,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-gnome-extensions +APP=dsk-gnome-extensions DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" @@ -77,7 +77,7 @@ infinito administration deploy dedicated "$INVENTORY/devices.yml" \ This role is a generic GNOME-extension installer driven by `services.gnome-extensions.plugins`, a 3-tuple list (`action`, `uuid`, `url`) consumed by `cli-gnome-extension-manager`. It ships no concrete extensions of its own and reads no `meta/addons/` files. -Roles that ship concrete GNOME extensions, such as [desk-gnome](../desk-gnome/), declare them through the unified addon contract under `meta/addons/` (requirement 026), with one file per extension (`mechanism: extension`, `source: upstream`, `config: { uuid, url }`); see that role for the per-file example. +Roles that ship concrete GNOME extensions, such as [dsk-gnome](../dsk-gnome/), declare them through the unified addon contract under `meta/addons/` (requirement 026), with one file per extension (`mechanism: extension`, `source: upstream`, `config: { uuid, url }`); see that role for the per-file example. Playwright exemption: GNOME Shell extensions are desktop-only and have no in-app web surface to drive, so they carry no Playwright spec (Confirmed Decision 11). diff --git a/roles/desk-gnome-extensions/meta/info.yml b/roles/dsk-gnome-extensions/meta/info.yml similarity index 100% rename from roles/desk-gnome-extensions/meta/info.yml rename to roles/dsk-gnome-extensions/meta/info.yml diff --git a/roles/desk-gnome-extensions/meta/main.yml b/roles/dsk-gnome-extensions/meta/main.yml similarity index 100% rename from roles/desk-gnome-extensions/meta/main.yml rename to roles/dsk-gnome-extensions/meta/main.yml diff --git a/roles/desk-gnome-extensions/meta/services.yml b/roles/dsk-gnome-extensions/meta/services.yml similarity index 100% rename from roles/desk-gnome-extensions/meta/services.yml rename to roles/dsk-gnome-extensions/meta/services.yml diff --git a/roles/desk-gnome-extensions/tasks/00_core.yml b/roles/dsk-gnome-extensions/tasks/00_core.yml similarity index 100% rename from roles/desk-gnome-extensions/tasks/00_core.yml rename to roles/dsk-gnome-extensions/tasks/00_core.yml diff --git a/roles/dsk-gnome-extensions/tasks/main.yml b/roles/dsk-gnome-extensions/tasks/main.yml new file mode 100644 index 0000000000..99fa902338 --- /dev/null +++ b/roles/dsk-gnome-extensions/tasks/main.yml @@ -0,0 +1,3 @@ +--- +- include_tasks: 00_core.yml + when: run_once_dsk_gnome_extensions is not defined diff --git a/roles/desk-gnome-extensions/vars/main.yml b/roles/dsk-gnome-extensions/vars/main.yml similarity index 85% rename from roles/desk-gnome-extensions/vars/main.yml rename to roles/dsk-gnome-extensions/vars/main.yml index 534de78ceb..10137ef241 100644 --- a/roles/desk-gnome-extensions/vars/main.yml +++ b/roles/dsk-gnome-extensions/vars/main.yml @@ -1,4 +1,4 @@ --- -application_id: desk-gnome-extensions +application_id: dsk-gnome-extensions DESK_GNOME_EXTENSIONS_MANAGER_REPO: "{{ lookup('config', application_id, 'services.gnome-extensions.repository') }}" DESK_GNOME_EXTENSIONS_MANAGER_REF: "{{ lookup('config', application_id, 'services.gnome-extensions.ref') }}" diff --git a/roles/desk-gnome-terminal/README.md b/roles/dsk-gnome-terminal/README.md similarity index 95% rename from roles/desk-gnome-terminal/README.md rename to roles/dsk-gnome-terminal/README.md index 9ba2b18e6d..4c3bcfdc5a 100644 --- a/roles/desk-gnome-terminal/README.md +++ b/roles/dsk-gnome-terminal/README.md @@ -16,7 +16,7 @@ The diagram places GNOME Terminal in the Infinito.Nexus cosmos: the components i ```mermaid flowchart LR - subgraph role [desk-gnome-terminal 💻] + subgraph role [dsk-gnome-terminal 💻] svc_gnome_terminal["gnome-terminal"] end ``` @@ -44,7 +44,7 @@ Clone, set up the workstation, and deploy GNOME Terminal onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-gnome-terminal full_cycle=false +make compose-deploy mode=reinstall apps=dsk-gnome-terminal full_cycle=false ``` ### Production @@ -58,7 +58,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-gnome-terminal +APP=dsk-gnome-terminal DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-gnome-terminal/meta/info.yml b/roles/dsk-gnome-terminal/meta/info.yml similarity index 100% rename from roles/desk-gnome-terminal/meta/info.yml rename to roles/dsk-gnome-terminal/meta/info.yml diff --git a/roles/desk-gnome-terminal/meta/main.yml b/roles/dsk-gnome-terminal/meta/main.yml similarity index 100% rename from roles/desk-gnome-terminal/meta/main.yml rename to roles/dsk-gnome-terminal/meta/main.yml diff --git a/roles/desk-gnome-terminal/meta/packages.yml b/roles/dsk-gnome-terminal/meta/packages.yml similarity index 100% rename from roles/desk-gnome-terminal/meta/packages.yml rename to roles/dsk-gnome-terminal/meta/packages.yml diff --git a/roles/desk-gnome-terminal/meta/services.yml b/roles/dsk-gnome-terminal/meta/services.yml similarity index 100% rename from roles/desk-gnome-terminal/meta/services.yml rename to roles/dsk-gnome-terminal/meta/services.yml diff --git a/roles/desk-gnome-terminal/tasks/00_core.yml b/roles/dsk-gnome-terminal/tasks/00_core.yml similarity index 100% rename from roles/desk-gnome-terminal/tasks/00_core.yml rename to roles/dsk-gnome-terminal/tasks/00_core.yml diff --git a/roles/dsk-gnome-terminal/tasks/main.yml b/roles/dsk-gnome-terminal/tasks/main.yml new file mode 100644 index 0000000000..3431ce89a9 --- /dev/null +++ b/roles/dsk-gnome-terminal/tasks/main.yml @@ -0,0 +1,3 @@ +--- +- include_tasks: 00_core.yml + when: run_once_dsk_gnome_terminal is not defined diff --git a/roles/dsk-gnome-terminal/vars/main.yml b/roles/dsk-gnome-terminal/vars/main.yml new file mode 100644 index 0000000000..8af0044e12 --- /dev/null +++ b/roles/dsk-gnome-terminal/vars/main.yml @@ -0,0 +1,2 @@ +--- +application_id: dsk-gnome-terminal diff --git a/roles/desk-gnome/README.md b/roles/dsk-gnome/README.md similarity index 90% rename from roles/desk-gnome/README.md rename to roles/dsk-gnome/README.md index e6e2747459..5b817f8d30 100644 --- a/roles/desk-gnome/README.md +++ b/roles/dsk-gnome/README.md @@ -4,9 +4,9 @@ This role aggregates various GNOME desktop components to ensure a cohesive and fully functional GNOME environment on Arch Linux. It includes the installation and configuration of several sub-roles: -- **desk-gnome-caffeine:** Prevents the system from sleeping or locking automatically. -- **desk-gnome-extensions:** Manages GNOME Shell extensions and installs the CLI GNOME Extension Manager. -- **desk-gnome-terminal:** Installs GNOME Terminal, the official terminal emulator for GNOME. +- **dsk-gnome-caffeine:** Prevents the system from sleeping or locking automatically. +- **dsk-gnome-extensions:** Manages GNOME Shell extensions and installs the CLI GNOME Extension Manager. +- **dsk-gnome-terminal:** Installs GNOME Terminal, the official terminal emulator for GNOME. ## Overview @@ -18,7 +18,7 @@ The diagram places GNOME Desktop in the Infinito.Nexus cosmos: the components it ```mermaid flowchart LR - subgraph role [desk-gnome 💻] + subgraph role [dsk-gnome 💻] svc_gnome["gnome"] end ``` @@ -47,7 +47,7 @@ Clone, set up the workstation, and deploy GNOME Desktop onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-gnome full_cycle=false +make compose-deploy mode=reinstall apps=dsk-gnome full_cycle=false ``` ### Production @@ -61,7 +61,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-gnome +APP=dsk-gnome DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-gnome/meta/addons/dash-to-dock.yml b/roles/dsk-gnome/meta/addons/dash-to-dock.yml similarity index 100% rename from roles/desk-gnome/meta/addons/dash-to-dock.yml rename to roles/dsk-gnome/meta/addons/dash-to-dock.yml diff --git a/roles/desk-gnome/meta/addons/dash-to-panel.yml b/roles/dsk-gnome/meta/addons/dash-to-panel.yml similarity index 100% rename from roles/desk-gnome/meta/addons/dash-to-panel.yml rename to roles/dsk-gnome/meta/addons/dash-to-panel.yml diff --git a/roles/desk-gnome/meta/addons/nasa-apod.yml b/roles/dsk-gnome/meta/addons/nasa-apod.yml similarity index 100% rename from roles/desk-gnome/meta/addons/nasa-apod.yml rename to roles/dsk-gnome/meta/addons/nasa-apod.yml diff --git a/roles/desk-gnome/meta/info.yml b/roles/dsk-gnome/meta/info.yml similarity index 100% rename from roles/desk-gnome/meta/info.yml rename to roles/dsk-gnome/meta/info.yml diff --git a/roles/desk-gnome/meta/main.yml b/roles/dsk-gnome/meta/main.yml similarity index 100% rename from roles/desk-gnome/meta/main.yml rename to roles/dsk-gnome/meta/main.yml diff --git a/roles/desk-gnome/meta/packages.yml b/roles/dsk-gnome/meta/packages.yml similarity index 100% rename from roles/desk-gnome/meta/packages.yml rename to roles/dsk-gnome/meta/packages.yml diff --git a/roles/desk-gnome/meta/services.yml b/roles/dsk-gnome/meta/services.yml similarity index 100% rename from roles/desk-gnome/meta/services.yml rename to roles/dsk-gnome/meta/services.yml diff --git a/roles/desk-gnome/meta/variants.yml b/roles/dsk-gnome/meta/variants.yml similarity index 100% rename from roles/desk-gnome/meta/variants.yml rename to roles/dsk-gnome/meta/variants.yml diff --git a/roles/desk-gnome/tasks/00_core.yml b/roles/dsk-gnome/tasks/00_core.yml similarity index 81% rename from roles/desk-gnome/tasks/00_core.yml rename to roles/dsk-gnome/tasks/00_core.yml index 053ed6688f..80b85c118a 100644 --- a/roles/desk-gnome/tasks/00_core.yml +++ b/roles/dsk-gnome/tasks/00_core.yml @@ -8,9 +8,9 @@ include_role: name: "{{ dependency_role }}" loop: - - desk-gnome-caffeine - - desk-gnome-extensions - - desk-gnome-terminal + - dsk-gnome-caffeine + - dsk-gnome-extensions + - dsk-gnome-terminal loop_control: loop_var: dependency_role diff --git a/roles/desk-gnome/tasks/01_extensions.yml b/roles/dsk-gnome/tasks/01_extensions.yml similarity index 100% rename from roles/desk-gnome/tasks/01_extensions.yml rename to roles/dsk-gnome/tasks/01_extensions.yml diff --git a/roles/dsk-gnome/tasks/main.yml b/roles/dsk-gnome/tasks/main.yml new file mode 100644 index 0000000000..6b9e5ab0f7 --- /dev/null +++ b/roles/dsk-gnome/tasks/main.yml @@ -0,0 +1,3 @@ +--- +- include_tasks: 00_core.yml + when: run_once_dsk_gnome is not defined diff --git a/roles/desk-gnome/vars/main.yml b/roles/dsk-gnome/vars/main.yml similarity index 88% rename from roles/desk-gnome/vars/main.yml rename to roles/dsk-gnome/vars/main.yml index e25b4c33d7..19394a3c11 100644 --- a/roles/desk-gnome/vars/main.yml +++ b/roles/dsk-gnome/vars/main.yml @@ -1,4 +1,4 @@ --- -application_id: desk-gnome +application_id: dsk-gnome DESK_GNOME_EXTENSION_MANAGER_REPO: "{{ lookup('config', application_id, 'services.gnome.repository') }}" DESK_GNOME_EXTENSION_MANAGER_REF: "{{ lookup('config', application_id, 'services.gnome.ref') }}" diff --git a/roles/dsk-gnt-claude/README.md b/roles/dsk-gnt-claude/README.md new file mode 100644 index 0000000000..9cb2d0c4bf --- /dev/null +++ b/roles/dsk-gnt-claude/README.md @@ -0,0 +1,85 @@ +# Claude Code + +## Description + +[Claude Code](https://code.claude.com/) is Anthropic's terminal coding agent. This role installs it on a workstation and points it at the platform's own model gateway instead of the vendor API. + +## Overview + +The agent speaks the Anthropic Messages API, which the LiteLLM gateway serves alongside its OpenAI routes. The role writes `~/.claude/settings.json` with an `env` block carrying the gateway URL, the workstation's key and the model aliases to use, so every request leaves the machine through the gateway and is accounted there. + +The gateway mints its virtual keys from the inventory it is deployed from, and a workstation is deployed from a different one. The key is therefore an operator-supplied credential on the workstation side: mint it on the server, then put it into the workstation inventory. + +## Cosmos + +The diagram places Claude Code in the Infinito.Nexus cosmos: the components it deploys (capabilities), the central services it consumes (dependencies), and its outward reach (federation and bridged external networks). + +```mermaid +flowchart LR + subgraph deps [Dependencies] + dep_svc_ai_litellm["svc-ai-litellm 🐳🐝"] + end + subgraph role [dsk-gnt-claude 💻] + svc_claude["claude"] + svc_litellm["litellm"] + end + subgraph dependents [Dependents] + dpt_dsk_code["dsk-code 💻"] + end + dep_svc_ai_litellm -. "0..1" .-> svc_litellm + svc_claude -. "0..1" .-> dpt_dsk_code +``` + +Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. + +## Features + +- **Gateway-backed:** `ANTHROPIC_BASE_URL` and `ANTHROPIC_AUTH_TOKEN` point the agent at the platform gateway. +- **Explicit models:** `ANTHROPIC_MODEL` and `ANTHROPIC_DEFAULT_HAIKU_MODEL` name gateway aliases, because the vendor's own model ids do not exist there. +- **No configuration without a key:** the settings file is written only once both the gateway URL and the key are set. +- **User-owned:** the file belongs to the workstation user and is readable only by them. + +## Quick Setup + +### Development + +Clone, set up the workstation, and deploy Claude Code onto the local stack: + +```bash +git clone https://github.com/infinito-nexus/core.git +cd core +make onboard +make compose-deploy mode=reinstall apps=dsk-gnt-claude full_cycle=false +``` + +### Production + +Install Claude Code directly onto the target machine: clone the repository, install the OS prerequisites and the repository toolchain, then deploy against localhost over a local connection (no SSH, no container): + +```bash +git clone https://github.com/infinito-nexus/core.git +cd core +bash scripts/install/package.sh +make install +source scripts/meta/env/load.sh + +APP=dsk-gnt-claude +DOMAIN= +TLS_MODE=self_signed +SSH_PUBLIC_KEY="" +INVENTORY=inventories/production +infinito administration inventory provision "$INVENTORY" \ + --inventory-file "$INVENTORY/devices.yml" \ + --host localhost \ + --include "$APP" \ + --vars "{\"TLS_MODE\": \"$TLS_MODE\", \"DOMAIN_PRIMARY\": \"$DOMAIN\", \"users\": {\"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}" +infinito administration deploy dedicated "$INVENTORY/devices.yml" \ + --password-file "$INVENTORY/.password" \ + --diff -vv +``` + +## Credits + +Implemented by **[Kevin Veen-Birkenbach](https://www.veen.world)**. +Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world). +Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license). diff --git a/roles/dsk-gnt-claude/files/test/.nocheck b/roles/dsk-gnt-claude/files/test/.nocheck new file mode 100644 index 0000000000..a0fdb0bf8c --- /dev/null +++ b/roles/dsk-gnt-claude/files/test/.nocheck @@ -0,0 +1 @@ +# nocheck: role-files-layout - Role test harness with test.sh as its entry point; the CLI runner stages roles//files/test/ by path. diff --git a/roles/dsk-gnt-claude/files/test/test.sh b/roles/dsk-gnt-claude/files/test/test.sh new file mode 100644 index 0000000000..28f990c0f5 --- /dev/null +++ b/roles/dsk-gnt-claude/files/test/test.sh @@ -0,0 +1,6 @@ +#!/usr/bin/env bash +# shellcheck shell=bash +set -euo pipefail + +here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +bash "${here}/shared/gateway/run.sh" diff --git a/roles/dsk-gnt-claude/meta/info.yml b/roles/dsk-gnt-claude/meta/info.yml new file mode 100644 index 0000000000..3000e23e60 --- /dev/null +++ b/roles/dsk-gnt-claude/meta/info.yml @@ -0,0 +1,5 @@ +--- +# nocheck: info-media no official demo video +logo: + class: fas fa-terminal +homepage: https://code.claude.com/ diff --git a/roles/dsk-gnt-claude/meta/main.yml b/roles/dsk-gnt-claude/meta/main.yml new file mode 100644 index 0000000000..9a218e1031 --- /dev/null +++ b/roles/dsk-gnt-claude/meta/main.yml @@ -0,0 +1,32 @@ +--- +galaxy_info: + author: Kevin Veen-Birkenbach + description: Terminal coding agent from Anthropic, pointed at the platform's own model gateway instead of the vendor API + license: Infinito.Nexus Community License (Non-Commercial) + company: | + Kevin Veen-Birkenbach + https://www.veen.world + galaxy_tags: + - ai + - agent + - cli + - claude + issue_tracker_url: https://s.infinito.nexus/issues + min_ansible_version: "2.9" + platforms: + - name: ArchLinux + versions: + - all + - name: Debian + versions: + - all + - name: EL + versions: + - all + - name: Fedora + versions: + - all + - name: Ubuntu + versions: + - all +dependencies: [] diff --git a/roles/dsk-gnt-claude/meta/secrets.yml b/roles/dsk-gnt-claude/meta/secrets.yml new file mode 100644 index 0000000000..ea8704904d --- /dev/null +++ b/roles/dsk-gnt-claude/meta/secrets.yml @@ -0,0 +1,12 @@ +--- +credentials: + gateway_key: + description: Virtual key the gateway minted for this workstation. The gateway deploys from a different inventory than the workstation, so the operator + carries the key over by hand. A workstation that runs the gateway itself leaves this empty and uses litellm_api_key instead. + algorithm: plain + validation: ^.*$ + litellm_api_key: + description: Per-consumer virtual key for the shared svc-ai-litellm gateway, minted with the deployment when the workstation runs the gateway itself + algorithm: sk_prefixed + validation: + min_length: 50 diff --git a/roles/dsk-gnt-claude/meta/services.yml b/roles/dsk-gnt-claude/meta/services.yml new file mode 100644 index 0000000000..87b1b63bb3 --- /dev/null +++ b/roles/dsk-gnt-claude/meta/services.yml @@ -0,0 +1,22 @@ +--- +claude: + modes: + host: + enabled: true + lifecycle: alpha + bond: 1 + enabled: true + shared: true + gateway_url: "" + model: "" + background_model: "" + min_storage: "500MB" + cpus: "1.0" + mem_reservation: "256m" + mem_limit: "1g" + pids_limit: 512 +# nocheck: playwright-service-flag, dynamic-flag AI backend, infrastructural, and a single shared instance +litellm: + bond: 1 + enabled: "{{ LITELLM_USABLE }}" + shared: true diff --git a/roles/dsk-gnt-claude/meta/tests.yml b/roles/dsk-gnt-claude/meta/tests.yml new file mode 100644 index 0000000000..3cda2dd425 --- /dev/null +++ b/roles/dsk-gnt-claude/meta/tests.yml @@ -0,0 +1,3 @@ +--- +cli: + timeout: 600 diff --git a/roles/dsk-gnt-claude/tasks/00_core.yml b/roles/dsk-gnt-claude/tasks/00_core.yml new file mode 100644 index 0000000000..72f6be7f3c --- /dev/null +++ b/roles/dsk-gnt-claude/tasks/00_core.yml @@ -0,0 +1,20 @@ +--- +- include_tasks: utils/once/flag.yml + +- name: ✅ Ensure npm is installed + package_install: + id: npm + +- name: 📦 install the claude coding agent + ansible.builtin.command: + cmd: npm install -g {{ agnt_claude_package }} + register: agnt_claude_install + changed_when: "'added' in agnt_claude_install.stdout or 'changed' in agnt_claude_install.stdout" + +- name: "🔐 Verify the gateway contract for {{ application_id }}" + ansible.builtin.include_tasks: "{{ lookup('path_absolute', 'roles/svc-ai-litellm/tasks/utils/consumer_contract.yml') }}" + when: lookup('config', application_id, 'services.litellm.enabled', false) | bool + +- name: 🔑 Point the agent at the model gateway + ansible.builtin.include_tasks: 01_gateway.yml + when: agnt_claude_configured | bool diff --git a/roles/dsk-gnt-claude/tasks/01_gateway.yml b/roles/dsk-gnt-claude/tasks/01_gateway.yml new file mode 100644 index 0000000000..7b35f884a4 --- /dev/null +++ b/roles/dsk-gnt-claude/tasks/01_gateway.yml @@ -0,0 +1,34 @@ +--- +- name: 📁 Ensure the agent's configuration directory exists + ansible.builtin.file: + path: "{{ agnt_claude_settings_path | dirname }}" + state: directory + owner: "{{ agnt_claude_user }}" + group: "{{ agnt_claude_user }}" + mode: "0700" + +- name: 🔎 Look for settings the user already keeps + ansible.builtin.stat: + path: "{{ agnt_claude_settings_path }}" + register: agnt_claude_settings_stat + +- name: 📖 Read the settings the user already keeps + ansible.builtin.slurp: + src: "{{ agnt_claude_settings_path }}" + register: agnt_claude_settings_slurp + no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" + when: agnt_claude_settings_stat.stat.exists + +- name: 🧊 Carry the user's settings into the template + ansible.builtin.set_fact: + agnt_claude_existing_settings: "{{ (agnt_claude_settings_slurp.content | b64decode | from_json) if agnt_claude_settings_stat.stat.exists else {} }}" + no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" + +- name: 🔐 Write the merged settings + ansible.builtin.template: # nocheck: stack-host-template (workstation role; the file belongs to the local user and there is no compose stack to gate on) + src: settings.json.j2 + dest: "{{ agnt_claude_settings_path }}" + owner: "{{ agnt_claude_user }}" + group: "{{ agnt_claude_user }}" + mode: "0600" + no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" diff --git a/roles/dsk-gnt-claude/tasks/main.yml b/roles/dsk-gnt-claude/tasks/main.yml new file mode 100644 index 0000000000..bef398a0f7 --- /dev/null +++ b/roles/dsk-gnt-claude/tasks/main.yml @@ -0,0 +1,3 @@ +--- +- include_tasks: 00_core.yml + when: run_once_dsk_gnt_claude is not defined diff --git a/roles/dsk-gnt-claude/templates/settings.json.j2 b/roles/dsk-gnt-claude/templates/settings.json.j2 new file mode 100644 index 0000000000..d23f45a9da --- /dev/null +++ b/roles/dsk-gnt-claude/templates/settings.json.j2 @@ -0,0 +1,4 @@ +{% set env = agnt_claude_existing_settings.get('env', {}) | combine({'ANTHROPIC_BASE_URL': agnt_claude_gateway_url, 'ANTHROPIC_AUTH_TOKEN': agnt_claude_gateway_key}) %} +{% if agnt_claude_model | length > 0 %}{% set env = env | combine({'ANTHROPIC_MODEL': agnt_claude_model}) %}{% endif %} +{% if agnt_claude_background_model | length > 0 %}{% set env = env | combine({'ANTHROPIC_DEFAULT_HAIKU_MODEL': agnt_claude_background_model}) %}{% endif %} +{{ agnt_claude_existing_settings | combine({'env': env}) | to_nice_json }} diff --git a/roles/dsk-gnt-claude/templates/test.env.j2 b/roles/dsk-gnt-claude/templates/test.env.j2 new file mode 100644 index 0000000000..a9a1b55667 --- /dev/null +++ b/roles/dsk-gnt-claude/templates/test.env.j2 @@ -0,0 +1,4 @@ +GATEWAY_CONFIG_PATH={{ agnt_claude_settings_path }} +GATEWAY_KEY={{ agnt_claude_gateway_key | dotenv_quote }} +GATEWAY_MODEL={{ agnt_claude_model }} +GATEWAY_URL={{ agnt_claude_gateway_url }} diff --git a/roles/dsk-gnt-claude/vars/main.yml b/roles/dsk-gnt-claude/vars/main.yml new file mode 100644 index 0000000000..c14c679709 --- /dev/null +++ b/roles/dsk-gnt-claude/vars/main.yml @@ -0,0 +1,13 @@ +--- +application_id: dsk-gnt-claude +agnt_claude_package: "@anthropic-ai/claude-code" +agnt_claude_user: "{{ lookup('users', WORKSTATION_USER).username }}" +agnt_claude_home: "/home/{{ agnt_claude_user }}" +agnt_claude_settings_path: "{{ [agnt_claude_home, '.claude', 'settings.json'] | path_join }}" +agnt_claude_gateway_url: "{{ lookup('config', application_id, 'services.claude.gateway_url') or (LITELLM_OPENAI_BASE_LOOPBACK_URL if LITELLM_USABLE | bool else '') + }}" +agnt_claude_model: "{{ lookup('config', application_id, 'services.claude.model') or (LITELLM_CHAT_MODEL if LITELLM_USABLE | bool else '') }}" +agnt_claude_background_model: "{{ lookup('config', application_id, 'services.claude.background_model') }}" +agnt_claude_gateway_key: "{{ lookup('config', application_id, 'secrets.credentials.gateway_key') or (lookup('config', application_id, 'secrets.credentials.litellm_api_key') + if LITELLM_USABLE | bool else '') }}" +agnt_claude_configured: "{{ (agnt_claude_gateway_url | length > 0) and (agnt_claude_gateway_key | length > 0) }}" diff --git a/roles/dsk-gnt-codex/README.md b/roles/dsk-gnt-codex/README.md new file mode 100644 index 0000000000..054a44173b --- /dev/null +++ b/roles/dsk-gnt-codex/README.md @@ -0,0 +1,82 @@ +# Codex + +## Description + +[Codex](https://github.com/openai/codex) is OpenAI's terminal coding agent, installed on a workstation and pointed at this deployment's own model gateway instead of the vendor API. + +## Overview + +The role installs the `@openai/codex` npm package and, when the workstation also runs [`svc-ai-litellm`](../svc-ai-litellm/), writes a provider profile that sends every completion to the gateway over the loopback address the engine publishes it on. Nothing leaves the machine: the gateway answers on `127.0.0.1`, so the workstation's virtual key never crosses a network. + +Codex reads its key from an environment variable rather than from its config file, so the role writes the key to a `0600` file under `~/.config/environment.d/` and names that variable in `config.toml`. A workstation whose gateway lives in another inventory leaves `services.codex.gateway_url` and `credentials.gateway_key` set by hand instead, and the loopback wiring stays out of the way. + +## Cosmos + +The diagram places Codex in the Infinito.Nexus cosmos: the components it deploys (capabilities), the central services it consumes (dependencies), and its outward reach (federation and bridged external networks). + +```mermaid +flowchart LR + subgraph deps [Dependencies] + dep_svc_ai_litellm["svc-ai-litellm 🐳🐝"] + end + subgraph role [dsk-gnt-codex 💻] + svc_codex["codex"] + svc_litellm["litellm"] + end + subgraph dependents [Dependents] + dpt_dsk_code["dsk-code 💻"] + end + dep_svc_ai_litellm -. "0..1" .-> svc_litellm + svc_codex -. "0..1" .-> dpt_dsk_code +``` + +Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. + +## Features + +- **Feature:** Describe a capability. + +## Quick Setup + +### Development + +Clone, set up the workstation, and deploy Codex onto the local stack: + +```bash +git clone https://github.com/infinito-nexus/core.git +cd core +make onboard +make compose-deploy mode=reinstall apps=dsk-gnt-codex full_cycle=false +``` + +### Production + +Install Codex directly onto the target machine: clone the repository, install the OS prerequisites and the repository toolchain, then deploy against localhost over a local connection (no SSH, no container): + +```bash +git clone https://github.com/infinito-nexus/core.git +cd core +bash scripts/install/package.sh +make install +source scripts/meta/env/load.sh + +APP=dsk-gnt-codex +DOMAIN= +TLS_MODE=self_signed +SSH_PUBLIC_KEY="" +INVENTORY=inventories/production +infinito administration inventory provision "$INVENTORY" \ + --inventory-file "$INVENTORY/devices.yml" \ + --host localhost \ + --include "$APP" \ + --vars "{\"TLS_MODE\": \"$TLS_MODE\", \"DOMAIN_PRIMARY\": \"$DOMAIN\", \"users\": {\"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}" +infinito administration deploy dedicated "$INVENTORY/devices.yml" \ + --password-file "$INVENTORY/.password" \ + --diff -vv +``` + +## Credits + +Implemented by **[Kevin Veen-Birkenbach](https://www.veen.world)**. +Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world). +Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license). diff --git a/roles/dsk-gnt-codex/files/test/.nocheck b/roles/dsk-gnt-codex/files/test/.nocheck new file mode 100644 index 0000000000..a0fdb0bf8c --- /dev/null +++ b/roles/dsk-gnt-codex/files/test/.nocheck @@ -0,0 +1 @@ +# nocheck: role-files-layout - Role test harness with test.sh as its entry point; the CLI runner stages roles//files/test/ by path. diff --git a/roles/dsk-gnt-codex/files/test/test.sh b/roles/dsk-gnt-codex/files/test/test.sh new file mode 100644 index 0000000000..28f990c0f5 --- /dev/null +++ b/roles/dsk-gnt-codex/files/test/test.sh @@ -0,0 +1,6 @@ +#!/usr/bin/env bash +# shellcheck shell=bash +set -euo pipefail + +here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +bash "${here}/shared/gateway/run.sh" diff --git a/roles/dsk-gnt-codex/meta/info.yml b/roles/dsk-gnt-codex/meta/info.yml new file mode 100644 index 0000000000..89a3244643 --- /dev/null +++ b/roles/dsk-gnt-codex/meta/info.yml @@ -0,0 +1,5 @@ +--- +# nocheck: info-media no official demo video +logo: + class: fas fa-terminal +homepage: https://github.com/openai/codex diff --git a/roles/dsk-gnt-codex/meta/main.yml b/roles/dsk-gnt-codex/meta/main.yml new file mode 100644 index 0000000000..a40550c1f5 --- /dev/null +++ b/roles/dsk-gnt-codex/meta/main.yml @@ -0,0 +1,32 @@ +--- +galaxy_info: + author: Kevin Veen-Birkenbach + description: Terminal coding agent from OpenAI, pointed at the platform's own model gateway instead of the vendor API + license: Infinito.Nexus Community License (Non-Commercial) + company: | + Kevin Veen-Birkenbach + https://www.veen.world + galaxy_tags: + - ai + - agent + - cli + - codex + issue_tracker_url: https://s.infinito.nexus/issues + min_ansible_version: "2.9" + platforms: + - name: ArchLinux + versions: + - all + - name: Debian + versions: + - all + - name: EL + versions: + - all + - name: Fedora + versions: + - all + - name: Ubuntu + versions: + - all +dependencies: [] diff --git a/roles/dsk-gnt-codex/meta/secrets.yml b/roles/dsk-gnt-codex/meta/secrets.yml new file mode 100644 index 0000000000..ea8704904d --- /dev/null +++ b/roles/dsk-gnt-codex/meta/secrets.yml @@ -0,0 +1,12 @@ +--- +credentials: + gateway_key: + description: Virtual key the gateway minted for this workstation. The gateway deploys from a different inventory than the workstation, so the operator + carries the key over by hand. A workstation that runs the gateway itself leaves this empty and uses litellm_api_key instead. + algorithm: plain + validation: ^.*$ + litellm_api_key: + description: Per-consumer virtual key for the shared svc-ai-litellm gateway, minted with the deployment when the workstation runs the gateway itself + algorithm: sk_prefixed + validation: + min_length: 50 diff --git a/roles/dsk-gnt-codex/meta/services.yml b/roles/dsk-gnt-codex/meta/services.yml new file mode 100644 index 0000000000..ed39eb8c18 --- /dev/null +++ b/roles/dsk-gnt-codex/meta/services.yml @@ -0,0 +1,21 @@ +--- +codex: + modes: + host: + enabled: true + lifecycle: alpha + bond: 1 + enabled: true + shared: true + gateway_url: "" + model: "" + min_storage: "500MB" + cpus: "1.0" + mem_reservation: "256m" + mem_limit: "1g" + pids_limit: 512 +# nocheck: playwright-service-flag, dynamic-flag AI backend, infrastructural, and a single shared instance +litellm: + bond: 1 + enabled: "{{ LITELLM_USABLE }}" + shared: true diff --git a/roles/dsk-gnt-codex/meta/tests.yml b/roles/dsk-gnt-codex/meta/tests.yml new file mode 100644 index 0000000000..3cda2dd425 --- /dev/null +++ b/roles/dsk-gnt-codex/meta/tests.yml @@ -0,0 +1,3 @@ +--- +cli: + timeout: 600 diff --git a/roles/dsk-gnt-codex/tasks/00_core.yml b/roles/dsk-gnt-codex/tasks/00_core.yml new file mode 100644 index 0000000000..4fe24cbfc6 --- /dev/null +++ b/roles/dsk-gnt-codex/tasks/00_core.yml @@ -0,0 +1,20 @@ +--- +- include_tasks: utils/once/flag.yml + +- name: ✅ Ensure npm is installed + package_install: + id: npm + +- name: 📦 install the codex coding agent + ansible.builtin.command: + cmd: npm install -g {{ agnt_codex_package }} + register: agnt_codex_install + changed_when: "'added' in agnt_codex_install.stdout or 'changed' in agnt_codex_install.stdout" + +- name: "🔐 Verify the gateway contract for {{ application_id }}" + ansible.builtin.include_tasks: "{{ lookup('path_absolute', 'roles/svc-ai-litellm/tasks/utils/consumer_contract.yml') }}" + when: lookup('config', application_id, 'services.litellm.enabled', false) | bool + +- name: 🔑 Point the agent at the model gateway + ansible.builtin.include_tasks: 01_gateway.yml + when: agnt_codex_configured | bool diff --git a/roles/dsk-gnt-codex/tasks/01_gateway.yml b/roles/dsk-gnt-codex/tasks/01_gateway.yml new file mode 100644 index 0000000000..80eb573c3f --- /dev/null +++ b/roles/dsk-gnt-codex/tasks/01_gateway.yml @@ -0,0 +1,28 @@ +--- +- name: 📁 Ensure the directories the agent reads from exist + ansible.builtin.file: + path: "{{ item | dirname }}" + state: directory + owner: "{{ agnt_codex_user }}" + group: "{{ agnt_codex_user }}" + mode: "0700" + loop: + - "{{ agnt_codex_config_path }}" + - "{{ agnt_codex_env_path }}" + +- name: 🔐 Write the gateway profile + ansible.builtin.template: # nocheck: stack-host-template (workstation role; the file belongs to the local user and there is no compose stack to gate on) + src: config.toml.j2 + dest: "{{ agnt_codex_config_path }}" + owner: "{{ agnt_codex_user }}" + group: "{{ agnt_codex_user }}" + mode: "0600" + +- name: "🔑 Write the key the agent reads from {{ agnt_codex_key_env }}" + ansible.builtin.template: # nocheck: stack-host-template (workstation role; the file belongs to the local user and there is no compose stack to gate on) + src: gateway.conf.j2 + dest: "{{ agnt_codex_env_path }}" + owner: "{{ agnt_codex_user }}" + group: "{{ agnt_codex_user }}" + mode: "0600" + no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" diff --git a/roles/dsk-gnt-codex/tasks/main.yml b/roles/dsk-gnt-codex/tasks/main.yml new file mode 100644 index 0000000000..228399fa04 --- /dev/null +++ b/roles/dsk-gnt-codex/tasks/main.yml @@ -0,0 +1,3 @@ +--- +- include_tasks: 00_core.yml + when: run_once_dsk_gnt_codex is not defined diff --git a/roles/dsk-gnt-codex/templates/config.toml.j2 b/roles/dsk-gnt-codex/templates/config.toml.j2 new file mode 100644 index 0000000000..fcd6b9a442 --- /dev/null +++ b/roles/dsk-gnt-codex/templates/config.toml.j2 @@ -0,0 +1,8 @@ +model = "{{ agnt_codex_model }}" +model_provider = "infinito" + +[model_providers.infinito] +name = "{{ SOFTWARE_NAME }} gateway" +base_url = "{{ agnt_codex_gateway_url }}" +env_key = "{{ agnt_codex_key_env }}" +wire_api = "chat" diff --git a/roles/dsk-gnt-codex/templates/gateway.conf.j2 b/roles/dsk-gnt-codex/templates/gateway.conf.j2 new file mode 100644 index 0000000000..d1ee37553d --- /dev/null +++ b/roles/dsk-gnt-codex/templates/gateway.conf.j2 @@ -0,0 +1 @@ +{{ agnt_codex_key_env }}={{ agnt_codex_gateway_key }} diff --git a/roles/dsk-gnt-codex/templates/test.env.j2 b/roles/dsk-gnt-codex/templates/test.env.j2 new file mode 100644 index 0000000000..947fd43fc4 --- /dev/null +++ b/roles/dsk-gnt-codex/templates/test.env.j2 @@ -0,0 +1,4 @@ +GATEWAY_CONFIG_PATH={{ agnt_codex_config_path }} +GATEWAY_KEY={{ agnt_codex_gateway_key | dotenv_quote }} +GATEWAY_MODEL={{ agnt_codex_model }} +GATEWAY_URL={{ agnt_codex_gateway_url }} diff --git a/roles/dsk-gnt-codex/vars/main.yml b/roles/dsk-gnt-codex/vars/main.yml new file mode 100644 index 0000000000..e219e29d77 --- /dev/null +++ b/roles/dsk-gnt-codex/vars/main.yml @@ -0,0 +1,14 @@ +--- +application_id: dsk-gnt-codex +agnt_codex_package: "@openai/codex" +agnt_codex_user: "{{ lookup('users', WORKSTATION_USER).username }}" +agnt_codex_home: "/home/{{ agnt_codex_user }}" +agnt_codex_config_path: "{{ [agnt_codex_home, '.codex', 'config.toml'] | path_join }}" +agnt_codex_key_env: INFINITO_GATEWAY_KEY +agnt_codex_env_path: "{{ [agnt_codex_home, '.config', 'environment.d', '50-infinito-codex.conf'] | path_join }}" +agnt_codex_gateway_url: "{{ lookup('config', application_id, 'services.codex.gateway_url') or (LITELLM_OPENAI_BASE_LOOPBACK_URL if LITELLM_USABLE | bool else '') + }}" +agnt_codex_model: "{{ lookup('config', application_id, 'services.codex.model') or (LITELLM_CHAT_MODEL if LITELLM_USABLE | bool else '') }}" +agnt_codex_gateway_key: "{{ lookup('config', application_id, 'secrets.credentials.gateway_key') or (lookup('config', application_id, 'secrets.credentials.litellm_api_key') + if LITELLM_USABLE | bool else '') }}" +agnt_codex_configured: "{{ (agnt_codex_gateway_url | length > 0) and (agnt_codex_gateway_key | length > 0) and (agnt_codex_model | length > 0) }}" diff --git a/roles/dsk-gnt-cursor/README.md b/roles/dsk-gnt-cursor/README.md new file mode 100644 index 0000000000..fec1cc9758 --- /dev/null +++ b/roles/dsk-gnt-cursor/README.md @@ -0,0 +1,74 @@ +# Cursor + +## Description + +[Cursor](https://cursor.com/) is an AI-first code editor. This role installs it on a workstation. + +## Overview + +Cursor's CLI cannot be pointed at a custom OpenAI-compatible endpoint: it exchanges the key for Cursor session tokens and talks its own protocol afterwards. The editor does carry an "Override OpenAI Base URL" field, but it lives in Cursor's own state database rather than in a file this role could render. + +The role therefore installs the editor and leaves the endpoint override to the operator. Nothing here pretends to configure a gateway that the product does not accept from a file. + +## Cosmos + +The diagram places Cursor in the Infinito.Nexus cosmos: the components it deploys (capabilities), the central services it consumes (dependencies), and its outward reach (federation and bridged external networks). + +```mermaid +flowchart LR + subgraph role [dsk-gnt-cursor 💻] + svc_cursor["cursor"] + end +``` + +Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. + +## Features + +- **Installed, not faked:** the editor is provisioned; the endpoint override stays a documented operator step. +- **Arch today:** the vendor ships neither a Debian nor an RPM repository, so only the Arch build is wired up. + +## Quick Setup + +### Development + +Clone, set up the workstation, and deploy Cursor onto the local stack: + +```bash +git clone https://github.com/infinito-nexus/core.git +cd core +make onboard +make compose-deploy mode=reinstall apps=dsk-gnt-cursor full_cycle=false +``` + +### Production + +Install Cursor directly onto the target machine: clone the repository, install the OS prerequisites and the repository toolchain, then deploy against localhost over a local connection (no SSH, no container): + +```bash +git clone https://github.com/infinito-nexus/core.git +cd core +bash scripts/install/package.sh +make install +source scripts/meta/env/load.sh + +APP=dsk-gnt-cursor +DOMAIN= +TLS_MODE=self_signed +SSH_PUBLIC_KEY="" +INVENTORY=inventories/production +infinito administration inventory provision "$INVENTORY" \ + --inventory-file "$INVENTORY/devices.yml" \ + --host localhost \ + --include "$APP" \ + --vars "{\"TLS_MODE\": \"$TLS_MODE\", \"DOMAIN_PRIMARY\": \"$DOMAIN\", \"users\": {\"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}" +infinito administration deploy dedicated "$INVENTORY/devices.yml" \ + --password-file "$INVENTORY/.password" \ + --diff -vv +``` + +## Credits + +Implemented by **[Kevin Veen-Birkenbach](https://www.veen.world)**. +Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world). +Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license). diff --git a/roles/dsk-gnt-cursor/meta/info.yml b/roles/dsk-gnt-cursor/meta/info.yml new file mode 100644 index 0000000000..3aea5b7868 --- /dev/null +++ b/roles/dsk-gnt-cursor/meta/info.yml @@ -0,0 +1,5 @@ +--- +# nocheck: info-media no official demo video +logo: + class: fas fa-i-cursor +homepage: https://cursor.com/ diff --git a/roles/dsk-gnt-cursor/meta/main.yml b/roles/dsk-gnt-cursor/meta/main.yml new file mode 100644 index 0000000000..b31c8547a3 --- /dev/null +++ b/roles/dsk-gnt-cursor/meta/main.yml @@ -0,0 +1,31 @@ +--- +galaxy_info: + author: Kevin Veen-Birkenbach + description: AI-first code editor installed on the workstation, whose endpoint override stays an operator step + license: Infinito.Nexus Community License (Non-Commercial) + company: | + Kevin Veen-Birkenbach + https://www.veen.world + galaxy_tags: + - ai + - editor + - cursor + issue_tracker_url: https://s.infinito.nexus/issues + min_ansible_version: "2.9" + platforms: + - name: ArchLinux + versions: + - all + - name: Debian + versions: + - all + - name: EL + versions: + - all + - name: Fedora + versions: + - all + - name: Ubuntu + versions: + - all +dependencies: [] diff --git a/roles/dsk-gnt-cursor/meta/packages.yml b/roles/dsk-gnt-cursor/meta/packages.yml new file mode 100644 index 0000000000..3a2011f4d8 --- /dev/null +++ b/roles/dsk-gnt-cursor/meta/packages.yml @@ -0,0 +1,8 @@ +--- +cursor: + Archlinux: + source: aur + names: + - cursor-bin + Debian: [] # nocheck: empty - Cursor ships only as a vendor AppImage and deb outside any Debian or Ubuntu archive; TODO: install the vendor build through a build source once its signature is pinned + RedHat: [] # nocheck: empty - Cursor ships no rpm and is carried by neither Fedora nor EPEL; TODO: install the vendor AppImage through a build source once its signature is pinned diff --git a/roles/dsk-gnt-cursor/meta/services.yml b/roles/dsk-gnt-cursor/meta/services.yml new file mode 100644 index 0000000000..4862551ea4 --- /dev/null +++ b/roles/dsk-gnt-cursor/meta/services.yml @@ -0,0 +1,11 @@ +--- +cursor: + modes: + host: + enabled: true + lifecycle: alpha + min_storage: "1GB" + cpus: "2.0" + mem_reservation: "512m" + mem_limit: "4g" + pids_limit: 1024 diff --git a/roles/dsk-gnt-cursor/tasks/main.yml b/roles/dsk-gnt-cursor/tasks/main.yml new file mode 100644 index 0000000000..4c87f36676 --- /dev/null +++ b/roles/dsk-gnt-cursor/tasks/main.yml @@ -0,0 +1,8 @@ +--- +- block: + - name: 📦 install cursor + package_install: + id: cursor + + - include_tasks: utils/once/flag.yml + when: run_once_dsk_gnt_cursor is not defined diff --git a/roles/dsk-gnt-cursor/vars/main.yml b/roles/dsk-gnt-cursor/vars/main.yml new file mode 100644 index 0000000000..9ecffd93b5 --- /dev/null +++ b/roles/dsk-gnt-cursor/vars/main.yml @@ -0,0 +1,2 @@ +--- +application_id: dsk-gnt-cursor diff --git a/roles/dsk-gnt-pi/README.md b/roles/dsk-gnt-pi/README.md new file mode 100644 index 0000000000..93cca7f735 --- /dev/null +++ b/roles/dsk-gnt-pi/README.md @@ -0,0 +1,81 @@ +# Pi Coding Agent + +## Description + +[Pi](https://github.com/earendil-works/pi) is a minimal terminal coding agent with a lazy-loading skill system. This role installs it on a workstation and registers the platform's model gateway as its provider. + +## Overview + +Pi reads its providers from `~/.pi/agent/models.json`. The role renders that file with one provider of type `openai-completions` pointing at the gateway's `/v1` base, the workstation's key, and the model aliases the operator listed. + +As with every workstation agent here, the key is operator-supplied: the gateway mints it from the server inventory, and the workstation inventory carries it. + +## Cosmos + +The diagram places Pi Coding Agent in the Infinito.Nexus cosmos: the components it deploys (capabilities), the central services it consumes (dependencies), and its outward reach (federation and bridged external networks). + +```mermaid +flowchart LR + subgraph deps [Dependencies] + dep_svc_ai_litellm["svc-ai-litellm 🐳🐝"] + end + subgraph role [dsk-gnt-pi 💻] + svc_pi["pi"] + svc_litellm["litellm"] + end + dep_svc_ai_litellm -. "0..1" .-> svc_litellm +``` + +Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. + +## Features + +- **Gateway-backed:** one provider entry, no vendor credentials on the machine. +- **Explicit model list:** the aliases come from the inventory, so the agent offers exactly what the gateway serves. +- **No configuration without a key:** the file is written only once URL, key and model list are set. +- **User-owned:** the file belongs to the workstation user and is readable only by them. + +## Quick Setup + +### Development + +Clone, set up the workstation, and deploy Pi Coding Agent onto the local stack: + +```bash +git clone https://github.com/infinito-nexus/core.git +cd core +make onboard +make compose-deploy mode=reinstall apps=dsk-gnt-pi full_cycle=false +``` + +### Production + +Install Pi Coding Agent directly onto the target machine: clone the repository, install the OS prerequisites and the repository toolchain, then deploy against localhost over a local connection (no SSH, no container): + +```bash +git clone https://github.com/infinito-nexus/core.git +cd core +bash scripts/install/package.sh +make install +source scripts/meta/env/load.sh + +APP=dsk-gnt-pi +DOMAIN= +TLS_MODE=self_signed +SSH_PUBLIC_KEY="" +INVENTORY=inventories/production +infinito administration inventory provision "$INVENTORY" \ + --inventory-file "$INVENTORY/devices.yml" \ + --host localhost \ + --include "$APP" \ + --vars "{\"TLS_MODE\": \"$TLS_MODE\", \"DOMAIN_PRIMARY\": \"$DOMAIN\", \"users\": {\"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}" +infinito administration deploy dedicated "$INVENTORY/devices.yml" \ + --password-file "$INVENTORY/.password" \ + --diff -vv +``` + +## Credits + +Implemented by **[Kevin Veen-Birkenbach](https://www.veen.world)**. +Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world). +Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license). diff --git a/roles/dsk-gnt-pi/files/test/.nocheck b/roles/dsk-gnt-pi/files/test/.nocheck new file mode 100644 index 0000000000..a0fdb0bf8c --- /dev/null +++ b/roles/dsk-gnt-pi/files/test/.nocheck @@ -0,0 +1 @@ +# nocheck: role-files-layout - Role test harness with test.sh as its entry point; the CLI runner stages roles//files/test/ by path. diff --git a/roles/dsk-gnt-pi/files/test/test.sh b/roles/dsk-gnt-pi/files/test/test.sh new file mode 100644 index 0000000000..28f990c0f5 --- /dev/null +++ b/roles/dsk-gnt-pi/files/test/test.sh @@ -0,0 +1,6 @@ +#!/usr/bin/env bash +# shellcheck shell=bash +set -euo pipefail + +here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +bash "${here}/shared/gateway/run.sh" diff --git a/roles/dsk-gnt-pi/meta/info.yml b/roles/dsk-gnt-pi/meta/info.yml new file mode 100644 index 0000000000..fb962ff471 --- /dev/null +++ b/roles/dsk-gnt-pi/meta/info.yml @@ -0,0 +1,5 @@ +--- +# nocheck: info-media no official demo video +logo: + class: fas fa-robot +homepage: https://github.com/earendil-works/pi diff --git a/roles/dsk-gnt-pi/meta/main.yml b/roles/dsk-gnt-pi/meta/main.yml new file mode 100644 index 0000000000..ca84cbb184 --- /dev/null +++ b/roles/dsk-gnt-pi/meta/main.yml @@ -0,0 +1,32 @@ +--- +galaxy_info: + author: Kevin Veen-Birkenbach + description: Minimal terminal coding agent with a lazy-loading skill system, wired to the platform's own model gateway + license: Infinito.Nexus Community License (Non-Commercial) + company: | + Kevin Veen-Birkenbach + https://www.veen.world + galaxy_tags: + - ai + - agent + - cli + - pi + issue_tracker_url: https://s.infinito.nexus/issues + min_ansible_version: "2.9" + platforms: + - name: ArchLinux + versions: + - all + - name: Debian + versions: + - all + - name: EL + versions: + - all + - name: Fedora + versions: + - all + - name: Ubuntu + versions: + - all +dependencies: [] diff --git a/roles/dsk-gnt-pi/meta/secrets.yml b/roles/dsk-gnt-pi/meta/secrets.yml new file mode 100644 index 0000000000..ea8704904d --- /dev/null +++ b/roles/dsk-gnt-pi/meta/secrets.yml @@ -0,0 +1,12 @@ +--- +credentials: + gateway_key: + description: Virtual key the gateway minted for this workstation. The gateway deploys from a different inventory than the workstation, so the operator + carries the key over by hand. A workstation that runs the gateway itself leaves this empty and uses litellm_api_key instead. + algorithm: plain + validation: ^.*$ + litellm_api_key: + description: Per-consumer virtual key for the shared svc-ai-litellm gateway, minted with the deployment when the workstation runs the gateway itself + algorithm: sk_prefixed + validation: + min_length: 50 diff --git a/roles/dsk-gnt-pi/meta/services.yml b/roles/dsk-gnt-pi/meta/services.yml new file mode 100644 index 0000000000..58349dbbec --- /dev/null +++ b/roles/dsk-gnt-pi/meta/services.yml @@ -0,0 +1,18 @@ +--- +pi: + modes: + host: + enabled: true + lifecycle: alpha + gateway_url: "" + models: [] + min_storage: "500MB" + cpus: "1.0" + mem_reservation: "256m" + mem_limit: "1g" + pids_limit: 512 +# nocheck: playwright-service-flag, dynamic-flag AI backend, infrastructural, and a single shared instance +litellm: + bond: 1 + enabled: "{{ LITELLM_USABLE }}" + shared: true diff --git a/roles/dsk-gnt-pi/meta/tests.yml b/roles/dsk-gnt-pi/meta/tests.yml new file mode 100644 index 0000000000..3cda2dd425 --- /dev/null +++ b/roles/dsk-gnt-pi/meta/tests.yml @@ -0,0 +1,3 @@ +--- +cli: + timeout: 600 diff --git a/roles/dsk-gnt-pi/tasks/00_core.yml b/roles/dsk-gnt-pi/tasks/00_core.yml new file mode 100644 index 0000000000..aa2b86732e --- /dev/null +++ b/roles/dsk-gnt-pi/tasks/00_core.yml @@ -0,0 +1,20 @@ +--- +- name: ✅ Ensure npm is installed + package_install: + id: npm + +- name: 📦 install the pi coding agent + ansible.builtin.command: + cmd: npm install -g --ignore-scripts {{ agnt_pi_package }} + register: agnt_pi_install + changed_when: "'added' in agnt_pi_install.stdout or 'changed' in agnt_pi_install.stdout" + +- name: "🔐 Verify the gateway contract for {{ application_id }}" + ansible.builtin.include_tasks: "{{ lookup('path_absolute', 'roles/svc-ai-litellm/tasks/utils/consumer_contract.yml') }}" + when: lookup('config', application_id, 'services.litellm.enabled', false) | bool + +- name: 🔑 Point the agent at the model gateway + ansible.builtin.include_tasks: 01_gateway.yml + when: agnt_pi_configured | bool + +- include_tasks: utils/once/flag.yml diff --git a/roles/dsk-gnt-pi/tasks/01_gateway.yml b/roles/dsk-gnt-pi/tasks/01_gateway.yml new file mode 100644 index 0000000000..c0aac8fe46 --- /dev/null +++ b/roles/dsk-gnt-pi/tasks/01_gateway.yml @@ -0,0 +1,34 @@ +--- +- name: 📁 Ensure the agent's configuration directory exists + ansible.builtin.file: + path: "{{ agnt_pi_models_path | dirname }}" + state: directory + owner: "{{ agnt_pi_user }}" + group: "{{ agnt_pi_user }}" + mode: "0700" + +- name: 🔎 Look for models the user already keeps + ansible.builtin.stat: + path: "{{ agnt_pi_models_path }}" + register: agnt_pi_models_stat + +- name: 📖 Read the models the user already keeps + ansible.builtin.slurp: + src: "{{ agnt_pi_models_path }}" + register: agnt_pi_models_slurp + no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" + when: agnt_pi_models_stat.stat.exists + +- name: 🧊 Carry the user's models into the template + ansible.builtin.set_fact: + agnt_pi_existing_models: "{{ (agnt_pi_models_slurp.content | b64decode | from_json) if agnt_pi_models_stat.stat.exists else {} }}" + no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" + +- name: 🔐 Write the merged model registry + ansible.builtin.template: # nocheck: stack-host-template (workstation role; the file belongs to the local user and there is no compose stack to gate on) + src: models.json.j2 + dest: "{{ agnt_pi_models_path }}" + owner: "{{ agnt_pi_user }}" + group: "{{ agnt_pi_user }}" + mode: "0600" + no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" diff --git a/roles/dsk-gnt-pi/tasks/main.yml b/roles/dsk-gnt-pi/tasks/main.yml new file mode 100644 index 0000000000..e2a08945c8 --- /dev/null +++ b/roles/dsk-gnt-pi/tasks/main.yml @@ -0,0 +1,3 @@ +--- +- include_tasks: 00_core.yml + when: run_once_dsk_gnt_pi is not defined diff --git a/roles/dsk-gnt-pi/templates/models.json.j2 b/roles/dsk-gnt-pi/templates/models.json.j2 new file mode 100644 index 0000000000..15c4a85186 --- /dev/null +++ b/roles/dsk-gnt-pi/templates/models.json.j2 @@ -0,0 +1,7 @@ +{% set entries = [] %} +{% for alias in agnt_pi_models %} +{% set _ = entries.append({"id": alias}) %} +{% endfor %} +{% set litellm = {"baseUrl": agnt_pi_gateway_url ~ "/v1", "api": "openai-completions", "apiKey": agnt_pi_gateway_key, "models": entries} %} +{% set providers = agnt_pi_existing_models.get('providers', {}) | combine({"litellm": litellm}) %} +{{ agnt_pi_existing_models | combine({"providers": providers}) | to_nice_json }} diff --git a/roles/dsk-gnt-pi/templates/test.env.j2 b/roles/dsk-gnt-pi/templates/test.env.j2 new file mode 100644 index 0000000000..1a36717904 --- /dev/null +++ b/roles/dsk-gnt-pi/templates/test.env.j2 @@ -0,0 +1,4 @@ +GATEWAY_CONFIG_PATH={{ agnt_pi_models_path }} +GATEWAY_KEY={{ agnt_pi_gateway_key | dotenv_quote }} +GATEWAY_MODEL={{ agnt_pi_models | first | default('', true) }} +GATEWAY_URL={{ agnt_pi_gateway_url }} diff --git a/roles/dsk-gnt-pi/vars/main.yml b/roles/dsk-gnt-pi/vars/main.yml new file mode 100644 index 0000000000..5767a1effe --- /dev/null +++ b/roles/dsk-gnt-pi/vars/main.yml @@ -0,0 +1,11 @@ +--- +application_id: dsk-gnt-pi +agnt_pi_package: "@earendil-works/pi-coding-agent" +agnt_pi_user: "{{ lookup('users', WORKSTATION_USER).username }}" +agnt_pi_home: "/home/{{ agnt_pi_user }}" +agnt_pi_models_path: "{{ [agnt_pi_home, '.pi', 'agent', 'models.json'] | path_join }}" +agnt_pi_gateway_url: "{{ lookup('config', application_id, 'services.pi.gateway_url') or (LITELLM_OPENAI_BASE_LOOPBACK_URL if LITELLM_USABLE | bool else '') }}" +agnt_pi_models: "{{ lookup('config', application_id, 'services.pi.models') or ([LITELLM_CHAT_MODEL] if LITELLM_USABLE | bool else []) }}" +agnt_pi_gateway_key: "{{ lookup('config', application_id, 'secrets.credentials.gateway_key') or (lookup('config', application_id, 'secrets.credentials.litellm_api_key') + if LITELLM_USABLE | bool else '') }}" +agnt_pi_configured: "{{ (agnt_pi_gateway_url | length > 0) and (agnt_pi_gateway_key | length > 0) and (agnt_pi_models | length > 0) }}" diff --git a/roles/desk-gnucash/README.md b/roles/dsk-gnucash/README.md similarity index 91% rename from roles/desk-gnucash/README.md rename to roles/dsk-gnucash/README.md index f0209271af..0cb93dcba7 100644 --- a/roles/desk-gnucash/README.md +++ b/roles/dsk-gnucash/README.md @@ -14,7 +14,7 @@ The diagram places GnuCash in the Infinito.Nexus cosmos: the components it deplo ```mermaid flowchart LR - subgraph role [desk-gnucash 💻] + subgraph role [dsk-gnucash 💻] svc_gnucash["gnucash"] end ``` @@ -35,7 +35,7 @@ Clone, set up the workstation, and deploy GnuCash onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-gnucash full_cycle=false +make compose-deploy mode=reinstall apps=dsk-gnucash full_cycle=false ``` ### Production @@ -49,7 +49,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-gnucash +APP=dsk-gnucash DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" @@ -64,9 +64,9 @@ infinito administration deploy dedicated "$INVENTORY/devices.yml" \ --diff -vv ``` -## Role: desk-gnucash +## Role: dsk-gnucash -The `desk-gnucash` role ensures that GnuCash is installed and maintained at its latest available version in the Pacman repositories. +The `dsk-gnucash` role ensures that GnuCash is installed and maintained at its latest available version in the Pacman repositories. ## Requirements @@ -92,7 +92,7 @@ An example of how to use this role in your playbook: ```yaml - hosts: your_target_group roles: - - desk-gnucash + - dsk-gnucash ``` ## Credits diff --git a/roles/desk-gnucash/meta/info.yml b/roles/dsk-gnucash/meta/info.yml similarity index 100% rename from roles/desk-gnucash/meta/info.yml rename to roles/dsk-gnucash/meta/info.yml diff --git a/roles/desk-gnucash/meta/main.yml b/roles/dsk-gnucash/meta/main.yml similarity index 100% rename from roles/desk-gnucash/meta/main.yml rename to roles/dsk-gnucash/meta/main.yml diff --git a/roles/desk-gnucash/meta/packages.yml b/roles/dsk-gnucash/meta/packages.yml similarity index 100% rename from roles/desk-gnucash/meta/packages.yml rename to roles/dsk-gnucash/meta/packages.yml diff --git a/roles/desk-gnucash/meta/services.yml b/roles/dsk-gnucash/meta/services.yml similarity index 100% rename from roles/desk-gnucash/meta/services.yml rename to roles/dsk-gnucash/meta/services.yml diff --git a/roles/desk-gnucash/tasks/main.yml b/roles/dsk-gnucash/tasks/main.yml similarity index 100% rename from roles/desk-gnucash/tasks/main.yml rename to roles/dsk-gnucash/tasks/main.yml diff --git a/roles/dsk-gnucash/vars/main.yml b/roles/dsk-gnucash/vars/main.yml new file mode 100644 index 0000000000..a725bd3b15 --- /dev/null +++ b/roles/dsk-gnucash/vars/main.yml @@ -0,0 +1,2 @@ +--- +application_id: dsk-gnucash diff --git a/roles/desk-jrnl/README.md b/roles/dsk-jrnl/README.md similarity index 95% rename from roles/desk-jrnl/README.md rename to roles/dsk-jrnl/README.md index 9e6f255c6b..e6301852ff 100644 --- a/roles/desk-jrnl/README.md +++ b/roles/dsk-jrnl/README.md @@ -14,7 +14,7 @@ The diagram places Jrnl in the Infinito.Nexus cosmos: the components it deploys ```mermaid flowchart LR - subgraph role [desk-jrnl 💻] + subgraph role [dsk-jrnl 💻] svc_jrnl["jrnl"] end ``` @@ -35,7 +35,7 @@ Clone, set up the workstation, and deploy Jrnl onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-jrnl full_cycle=false +make compose-deploy mode=reinstall apps=dsk-jrnl full_cycle=false ``` ### Production @@ -49,7 +49,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-jrnl +APP=dsk-jrnl DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-jrnl/meta/info.yml b/roles/dsk-jrnl/meta/info.yml similarity index 100% rename from roles/desk-jrnl/meta/info.yml rename to roles/dsk-jrnl/meta/info.yml diff --git a/roles/desk-jrnl/meta/main.yml b/roles/dsk-jrnl/meta/main.yml similarity index 100% rename from roles/desk-jrnl/meta/main.yml rename to roles/dsk-jrnl/meta/main.yml diff --git a/roles/desk-jrnl/meta/packages.yml b/roles/dsk-jrnl/meta/packages.yml similarity index 100% rename from roles/desk-jrnl/meta/packages.yml rename to roles/dsk-jrnl/meta/packages.yml diff --git a/roles/desk-jrnl/meta/services.yml b/roles/dsk-jrnl/meta/services.yml similarity index 100% rename from roles/desk-jrnl/meta/services.yml rename to roles/dsk-jrnl/meta/services.yml diff --git a/roles/desk-jrnl/tasks/main.yml b/roles/dsk-jrnl/tasks/main.yml similarity index 100% rename from roles/desk-jrnl/tasks/main.yml rename to roles/dsk-jrnl/tasks/main.yml diff --git a/roles/dsk-jrnl/vars/main.yml b/roles/dsk-jrnl/vars/main.yml new file mode 100644 index 0000000000..a30c72b325 --- /dev/null +++ b/roles/dsk-jrnl/vars/main.yml @@ -0,0 +1,2 @@ +--- +application_id: dsk-jrnl diff --git a/roles/desk-keepassxc/README.md b/roles/dsk-keepassxc/README.md similarity index 92% rename from roles/desk-keepassxc/README.md rename to roles/dsk-keepassxc/README.md index 8c05e5002b..62f1dfbb22 100644 --- a/roles/desk-keepassxc/README.md +++ b/roles/dsk-keepassxc/README.md @@ -15,7 +15,7 @@ The diagram places KeePassXC in the Infinito.Nexus cosmos: the components it dep ```mermaid flowchart LR - subgraph role [desk-keepassxc 💻] + subgraph role [dsk-keepassxc 💻] svc_keepassxc["keepassxc"] end ``` @@ -26,7 +26,7 @@ Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional ( - **Local-first vault:** Keeps the password database on the workstation, with no mandatory cloud sync. - **Pacman integration:** Installs the upstream `keepassxc` package via the standard system package manager. -- **Workstation scope:** Targets the desktop tier (`desk-*`) and stays out of server inventories. +- **Workstation scope:** Targets the desktop tier (`dsk-*`) and stays out of server inventories. - **Minimal footprint:** Does not enable services or autostart entries beyond what the package itself provides. ## Quick Setup @@ -39,7 +39,7 @@ Clone, set up the workstation, and deploy KeePassXC onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-keepassxc full_cycle=false +make compose-deploy mode=reinstall apps=dsk-keepassxc full_cycle=false ``` ### Production @@ -53,7 +53,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-keepassxc +APP=dsk-keepassxc DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-keepassxc/meta/info.yml b/roles/dsk-keepassxc/meta/info.yml similarity index 100% rename from roles/desk-keepassxc/meta/info.yml rename to roles/dsk-keepassxc/meta/info.yml diff --git a/roles/desk-keepassxc/meta/main.yml b/roles/dsk-keepassxc/meta/main.yml similarity index 100% rename from roles/desk-keepassxc/meta/main.yml rename to roles/dsk-keepassxc/meta/main.yml diff --git a/roles/desk-keepassxc/meta/packages.yml b/roles/dsk-keepassxc/meta/packages.yml similarity index 100% rename from roles/desk-keepassxc/meta/packages.yml rename to roles/dsk-keepassxc/meta/packages.yml diff --git a/roles/desk-keepassxc/meta/services.yml b/roles/dsk-keepassxc/meta/services.yml similarity index 100% rename from roles/desk-keepassxc/meta/services.yml rename to roles/dsk-keepassxc/meta/services.yml diff --git a/roles/desk-keepassxc/tasks/main.yml b/roles/dsk-keepassxc/tasks/main.yml similarity index 74% rename from roles/desk-keepassxc/tasks/main.yml rename to roles/dsk-keepassxc/tasks/main.yml index 31cc7abdfc..7bda10646d 100644 --- a/roles/desk-keepassxc/tasks/main.yml +++ b/roles/dsk-keepassxc/tasks/main.yml @@ -5,4 +5,4 @@ id: keepassxc - include_tasks: utils/once/flag.yml - when: run_once_desk_keepassxc is not defined + when: run_once_dsk_keepassxc is not defined diff --git a/roles/dsk-keepassxc/vars/main.yml b/roles/dsk-keepassxc/vars/main.yml new file mode 100644 index 0000000000..d32b9b7bad --- /dev/null +++ b/roles/dsk-keepassxc/vars/main.yml @@ -0,0 +1,2 @@ +--- +application_id: dsk-keepassxc diff --git a/roles/dsk-laya/README.md b/roles/dsk-laya/README.md new file mode 100644 index 0000000000..2fe31af81c --- /dev/null +++ b/roles/dsk-laya/README.md @@ -0,0 +1,78 @@ +# Laya + +## Description + +[Laya](https://laya.aay.sh/) is a local-first notification command center that aggregates Slack, mail, issue trackers and calendars into approvable action cards. + +## Overview + +The role installs the pinned upstream AppImage and puts it on `PATH`. Upstream signs its assets with the Tauri updater key and publishes no digest, so the role pins a sha256 taken from the released build itself and the download is verified against it. A rebuilt or replaced asset fails the deploy instead of being installed. + +The pinned build is the amd64 AppImage, and the role refuses a host of another architecture rather than linking a binary that cannot run there. + +Laya keeps its own settings in `~/.laya/`, including the model providers it talks to. This role does not render that file: upstream documents the provider types but not the on-disk schema, so the provider is an operator step in the application's own settings. Pointing it at the platform gateway means adding a provider of type `openai_compatible`. + +## Cosmos + +The diagram places Laya in the Infinito.Nexus cosmos: the components it deploys (capabilities), the central services it consumes (dependencies), and its outward reach (federation and bridged external networks). + +```mermaid +flowchart LR + subgraph role [dsk-laya 💻] + svc_laya["laya"] + end +``` + +Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. + +## Features + +- **Pinned and verified:** the AppImage version and its sha256 live in the role, and a mismatch aborts the deploy. +- **Architecture-guarded:** a host that is not amd64 aborts before the download. +- **Distribution-independent:** the AppImage runs on every supported family, so no vendor repository is needed. +- **No invented configuration:** the provider setup stays in the application, because its file schema is not documented upstream. + +## Quick Setup + +### Development + +Clone, set up the workstation, and deploy Laya onto the local stack: + +```bash +git clone https://github.com/infinito-nexus/core.git +cd core +make onboard +make compose-deploy mode=reinstall apps=dsk-laya full_cycle=false +``` + +### Production + +Install Laya directly onto the target machine: clone the repository, install the OS prerequisites and the repository toolchain, then deploy against localhost over a local connection (no SSH, no container): + +```bash +git clone https://github.com/infinito-nexus/core.git +cd core +bash scripts/install/package.sh +make install +source scripts/meta/env/load.sh + +APP=dsk-laya +DOMAIN= +TLS_MODE=self_signed +SSH_PUBLIC_KEY="" +INVENTORY=inventories/production +infinito administration inventory provision "$INVENTORY" \ + --inventory-file "$INVENTORY/devices.yml" \ + --host localhost \ + --include "$APP" \ + --vars "{\"TLS_MODE\": \"$TLS_MODE\", \"DOMAIN_PRIMARY\": \"$DOMAIN\", \"users\": {\"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}" +infinito administration deploy dedicated "$INVENTORY/devices.yml" \ + --password-file "$INVENTORY/.password" \ + --diff -vv +``` + +## Credits + +Implemented by **[Kevin Veen-Birkenbach](https://www.veen.world)**. +Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world). +Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license). diff --git a/roles/dsk-laya/meta/info.yml b/roles/dsk-laya/meta/info.yml new file mode 100644 index 0000000000..42cd96af04 --- /dev/null +++ b/roles/dsk-laya/meta/info.yml @@ -0,0 +1,5 @@ +--- +# nocheck: info-media no official demo video +logo: + class: fas fa-bell +homepage: https://laya.aay.sh/ diff --git a/roles/dsk-laya/meta/main.yml b/roles/dsk-laya/meta/main.yml new file mode 100644 index 0000000000..b6cd523ab6 --- /dev/null +++ b/roles/dsk-laya/meta/main.yml @@ -0,0 +1,32 @@ +--- +galaxy_info: + author: Kevin Veen-Birkenbach + description: Local-first notification command center that aggregates Slack, mail and issue trackers into approvable action cards + license: Infinito.Nexus Community License (Non-Commercial) + company: | + Kevin Veen-Birkenbach + https://www.veen.world + galaxy_tags: + - ai + - notifications + - workstation + - laya + issue_tracker_url: https://s.infinito.nexus/issues + min_ansible_version: "2.9" + platforms: + - name: ArchLinux + versions: + - all + - name: Debian + versions: + - all + - name: EL + versions: + - all + - name: Fedora + versions: + - all + - name: Ubuntu + versions: + - all +dependencies: [] diff --git a/roles/dsk-laya/meta/services.yml b/roles/dsk-laya/meta/services.yml new file mode 100644 index 0000000000..804695dd44 --- /dev/null +++ b/roles/dsk-laya/meta/services.yml @@ -0,0 +1,11 @@ +--- +laya: + modes: + host: + enabled: true + lifecycle: pre-alpha + min_storage: "1GB" + cpus: "2.0" + mem_reservation: "512m" + mem_limit: "4g" + pids_limit: 1024 diff --git a/roles/dsk-laya/tasks/00_core.yml b/roles/dsk-laya/tasks/00_core.yml new file mode 100644 index 0000000000..7a16b221a1 --- /dev/null +++ b/roles/dsk-laya/tasks/00_core.yml @@ -0,0 +1,28 @@ +--- +- name: "🛑 Refuse an architecture the pinned build does not carry" + ansible.builtin.assert: + that: + - ansible_facts['architecture'] == "x86_64" + fail_msg: "dsk-laya pins the amd64 AppImage; this host reports {{ ansible_facts['architecture'] }}." + quiet: true + +- name: 📁 Ensure the application directory exists + ansible.builtin.file: + path: "{{ laya_appimage_path | dirname }}" + state: directory + mode: "0755" + +- name: ⬇️ Fetch the pinned Laya AppImage + get_url_retry: + url: "{{ laya_appimage_url }}" + dest: "{{ laya_appimage_path }}" + checksum: "{{ laya_appimage_checksum }}" + mode: "0755" + +- name: 🔗 Put the pinned build on PATH + ansible.builtin.file: + src: "{{ laya_appimage_path }}" + dest: "{{ laya_binary_path }}" + state: link + +- include_tasks: utils/once/flag.yml diff --git a/roles/dsk-laya/tasks/main.yml b/roles/dsk-laya/tasks/main.yml new file mode 100644 index 0000000000..47fdcda22f --- /dev/null +++ b/roles/dsk-laya/tasks/main.yml @@ -0,0 +1,3 @@ +--- +- include_tasks: 00_core.yml + when: run_once_dsk_laya is not defined diff --git a/roles/dsk-laya/vars/main.yml b/roles/dsk-laya/vars/main.yml new file mode 100644 index 0000000000..57de6313b1 --- /dev/null +++ b/roles/dsk-laya/vars/main.yml @@ -0,0 +1,7 @@ +--- +application_id: dsk-laya +laya_version: "1.6.0" +laya_appimage_url: "https://github.com/aayushch/laya/releases/download/v{{ laya_version }}/Laya_{{ laya_version }}_amd64.AppImage" +laya_appimage_checksum: "sha256:acdae347736bbeff540b227f131de7c837cb9e2839d7fc93dd1c96d04dafae11" +laya_appimage_path: "/opt/laya/Laya-{{ laya_version }}.AppImage" +laya_binary_path: /usr/local/bin/laya diff --git a/roles/desk-libreoffice/README.md b/roles/dsk-libreoffice/README.md similarity index 96% rename from roles/desk-libreoffice/README.md rename to roles/dsk-libreoffice/README.md index 6ac7535d54..4fe0384b3d 100644 --- a/roles/desk-libreoffice/README.md +++ b/roles/dsk-libreoffice/README.md @@ -19,7 +19,7 @@ flowchart LR subgraph deps [Dependencies] dep_gen_hunspell["gen-hunspell 💻 ⚙️"] end - subgraph role [desk-libreoffice 💻] + subgraph role [dsk-libreoffice 💻] svc_libreoffice["libreoffice"] end dep_gen_hunspell -- "1:1" --> svc_libreoffice @@ -48,7 +48,7 @@ Clone, set up the workstation, and deploy LibreOffice onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-libreoffice full_cycle=false +make compose-deploy mode=reinstall apps=dsk-libreoffice full_cycle=false ``` ### Production @@ -62,7 +62,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-libreoffice +APP=dsk-libreoffice DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-libreoffice/meta/info.yml b/roles/dsk-libreoffice/meta/info.yml similarity index 100% rename from roles/desk-libreoffice/meta/info.yml rename to roles/dsk-libreoffice/meta/info.yml diff --git a/roles/desk-libreoffice/meta/main.yml b/roles/dsk-libreoffice/meta/main.yml similarity index 100% rename from roles/desk-libreoffice/meta/main.yml rename to roles/dsk-libreoffice/meta/main.yml diff --git a/roles/desk-libreoffice/meta/packages.yml b/roles/dsk-libreoffice/meta/packages.yml similarity index 100% rename from roles/desk-libreoffice/meta/packages.yml rename to roles/dsk-libreoffice/meta/packages.yml diff --git a/roles/desk-libreoffice/meta/services.yml b/roles/dsk-libreoffice/meta/services.yml similarity index 100% rename from roles/desk-libreoffice/meta/services.yml rename to roles/dsk-libreoffice/meta/services.yml diff --git a/roles/desk-libreoffice/tasks/main.yml b/roles/dsk-libreoffice/tasks/main.yml similarity index 77% rename from roles/desk-libreoffice/tasks/main.yml rename to roles/dsk-libreoffice/tasks/main.yml index 52cfdc9b61..9e2fd86237 100644 --- a/roles/desk-libreoffice/tasks/main.yml +++ b/roles/dsk-libreoffice/tasks/main.yml @@ -5,6 +5,6 @@ - name: 📦 install libreoffice language packages package: # nocheck: dynamic-package (the language pack name is built from the configured flavor and the loop variable) - name: "libreoffice-{{ lookup('applications', 'desk-libreoffice').flavor }}-{{ item }}" + name: "libreoffice-{{ lookup('applications', 'dsk-libreoffice').flavor }}-{{ item }}" state: present loop: "{{ libreoffice_languages }}" diff --git a/roles/desk-libreoffice/vars/main.yml b/roles/dsk-libreoffice/vars/main.yml similarity index 74% rename from roles/desk-libreoffice/vars/main.yml rename to roles/dsk-libreoffice/vars/main.yml index b7c6f80acb..3dbbf6b5c8 100644 --- a/roles/desk-libreoffice/vars/main.yml +++ b/roles/dsk-libreoffice/vars/main.yml @@ -1,3 +1,3 @@ --- -application_id: "desk-libreoffice" +application_id: "dsk-libreoffice" libreoffice_languages: "{{ lookup('config', application_id, 'services.libreoffice.languages') }}" diff --git a/roles/desk-micro/README.md b/roles/dsk-micro/README.md similarity index 95% rename from roles/desk-micro/README.md rename to roles/dsk-micro/README.md index 5df1c7ac5c..0bf1ff8453 100644 --- a/roles/desk-micro/README.md +++ b/roles/dsk-micro/README.md @@ -14,7 +14,7 @@ The diagram places micro in the Infinito.Nexus cosmos: the components it deploys ```mermaid flowchart LR - subgraph role [desk-micro 💻] + subgraph role [dsk-micro 💻] svc_micro["micro"] end ``` @@ -35,7 +35,7 @@ Clone, set up the workstation, and deploy micro onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-micro full_cycle=false +make compose-deploy mode=reinstall apps=dsk-micro full_cycle=false ``` ### Production @@ -49,7 +49,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-micro +APP=dsk-micro DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" @@ -82,7 +82,7 @@ None. ```yaml - hosts: all roles: - - desk-micro + - dsk-micro ``` ## Further Resources diff --git a/roles/desk-micro/meta/info.yml b/roles/dsk-micro/meta/info.yml similarity index 100% rename from roles/desk-micro/meta/info.yml rename to roles/dsk-micro/meta/info.yml diff --git a/roles/desk-micro/meta/main.yml b/roles/dsk-micro/meta/main.yml similarity index 100% rename from roles/desk-micro/meta/main.yml rename to roles/dsk-micro/meta/main.yml diff --git a/roles/desk-micro/meta/packages.yml b/roles/dsk-micro/meta/packages.yml similarity index 100% rename from roles/desk-micro/meta/packages.yml rename to roles/dsk-micro/meta/packages.yml diff --git a/roles/desk-micro/meta/services.yml b/roles/dsk-micro/meta/services.yml similarity index 100% rename from roles/desk-micro/meta/services.yml rename to roles/dsk-micro/meta/services.yml diff --git a/roles/desk-micro/tasks/main.yml b/roles/dsk-micro/tasks/main.yml similarity index 100% rename from roles/desk-micro/tasks/main.yml rename to roles/dsk-micro/tasks/main.yml diff --git a/roles/desk-micro/vars/main.yml b/roles/dsk-micro/vars/main.yml similarity index 51% rename from roles/desk-micro/vars/main.yml rename to roles/dsk-micro/vars/main.yml index 6113e7f6e9..552e909f20 100644 --- a/roles/desk-micro/vars/main.yml +++ b/roles/dsk-micro/vars/main.yml @@ -1,3 +1,3 @@ --- # Internal configuration -application_id: desk-micro +application_id: dsk-micro diff --git a/roles/desk-neovim/README.md b/roles/dsk-neovim/README.md similarity index 95% rename from roles/desk-neovim/README.md rename to roles/dsk-neovim/README.md index ae9ed570e2..e55f0b7c0c 100644 --- a/roles/desk-neovim/README.md +++ b/roles/dsk-neovim/README.md @@ -14,7 +14,7 @@ The diagram places Neovim in the Infinito.Nexus cosmos: the components it deploy ```mermaid flowchart LR - subgraph role [desk-neovim 💻] + subgraph role [dsk-neovim 💻] svc_neovim["neovim"] end ``` @@ -35,7 +35,7 @@ Clone, set up the workstation, and deploy Neovim onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-neovim full_cycle=false +make compose-deploy mode=reinstall apps=dsk-neovim full_cycle=false ``` ### Production @@ -49,7 +49,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-neovim +APP=dsk-neovim DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" @@ -82,7 +82,7 @@ None. ```yaml - hosts: all roles: - - desk-neovim + - dsk-neovim ``` ## Further Resources diff --git a/roles/desk-neovim/meta/info.yml b/roles/dsk-neovim/meta/info.yml similarity index 100% rename from roles/desk-neovim/meta/info.yml rename to roles/dsk-neovim/meta/info.yml diff --git a/roles/desk-neovim/meta/main.yml b/roles/dsk-neovim/meta/main.yml similarity index 100% rename from roles/desk-neovim/meta/main.yml rename to roles/dsk-neovim/meta/main.yml diff --git a/roles/desk-neovim/meta/packages.yml b/roles/dsk-neovim/meta/packages.yml similarity index 100% rename from roles/desk-neovim/meta/packages.yml rename to roles/dsk-neovim/meta/packages.yml diff --git a/roles/desk-neovim/meta/services.yml b/roles/dsk-neovim/meta/services.yml similarity index 100% rename from roles/desk-neovim/meta/services.yml rename to roles/dsk-neovim/meta/services.yml diff --git a/roles/desk-neovim/tasks/main.yml b/roles/dsk-neovim/tasks/main.yml similarity index 100% rename from roles/desk-neovim/tasks/main.yml rename to roles/dsk-neovim/tasks/main.yml diff --git a/roles/desk-neovim/vars/main.yml b/roles/dsk-neovim/vars/main.yml similarity index 50% rename from roles/desk-neovim/vars/main.yml rename to roles/dsk-neovim/vars/main.yml index e2d8e3d7d0..8137dd59e7 100644 --- a/roles/desk-neovim/vars/main.yml +++ b/roles/dsk-neovim/vars/main.yml @@ -1,3 +1,3 @@ --- # Internal configuration -application_id: desk-neovim +application_id: dsk-neovim diff --git a/roles/desk-nextcloud/README.md b/roles/dsk-nextcloud/README.md similarity index 96% rename from roles/desk-nextcloud/README.md rename to roles/dsk-nextcloud/README.md index 9bf3f7c7dc..88f914f8c3 100644 --- a/roles/desk-nextcloud/README.md +++ b/roles/dsk-nextcloud/README.md @@ -14,7 +14,7 @@ The diagram places Nextcloud Client ️ in the Infinito.Nexus cosmos: the compon ```mermaid flowchart LR - subgraph role [desk-nextcloud 💻] + subgraph role [dsk-nextcloud 💻] svc_nextcloud["nextcloud"] end ``` @@ -42,7 +42,7 @@ Clone, set up the workstation, and deploy Nextcloud Client ️ onto the local st git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-nextcloud full_cycle=false +make compose-deploy mode=reinstall apps=dsk-nextcloud full_cycle=false ``` ### Production @@ -56,7 +56,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-nextcloud +APP=dsk-nextcloud DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-nextcloud/TODO.md b/roles/dsk-nextcloud/TODO.md similarity index 100% rename from roles/desk-nextcloud/TODO.md rename to roles/dsk-nextcloud/TODO.md diff --git a/roles/desk-nextcloud/meta/info.yml b/roles/dsk-nextcloud/meta/info.yml similarity index 100% rename from roles/desk-nextcloud/meta/info.yml rename to roles/dsk-nextcloud/meta/info.yml diff --git a/roles/desk-nextcloud/meta/main.yml b/roles/dsk-nextcloud/meta/main.yml similarity index 100% rename from roles/desk-nextcloud/meta/main.yml rename to roles/dsk-nextcloud/meta/main.yml diff --git a/roles/desk-nextcloud/meta/packages.yml b/roles/dsk-nextcloud/meta/packages.yml similarity index 100% rename from roles/desk-nextcloud/meta/packages.yml rename to roles/dsk-nextcloud/meta/packages.yml diff --git a/roles/desk-nextcloud/meta/secrets.yml b/roles/dsk-nextcloud/meta/secrets.yml similarity index 100% rename from roles/desk-nextcloud/meta/secrets.yml rename to roles/dsk-nextcloud/meta/secrets.yml diff --git a/roles/desk-nextcloud/meta/services.yml b/roles/dsk-nextcloud/meta/services.yml similarity index 100% rename from roles/desk-nextcloud/meta/services.yml rename to roles/dsk-nextcloud/meta/services.yml diff --git a/roles/desk-nextcloud/tasks/main.yml b/roles/dsk-nextcloud/tasks/main.yml similarity index 98% rename from roles/desk-nextcloud/tasks/main.yml rename to roles/dsk-nextcloud/tasks/main.yml index a630b7975f..2e0bb69682 100644 --- a/roles/desk-nextcloud/tasks/main.yml +++ b/roles/dsk-nextcloud/tasks/main.yml @@ -38,7 +38,7 @@ - name: 📌 Apply Nextcloud folder mappings via dotlinker ansible.builtin.include_role: - name: desk-dotlinker + name: dsk-dotlinker vars: dotlinker_user: "{{ lookup('users', WORKSTATION_USER).username }}" dotlinker_config_path: "{{ nextcloud_user_home_directory }}.config/dotlinker/config.yaml" diff --git a/roles/desk-nextcloud/vars/main.yml b/roles/dsk-nextcloud/vars/main.yml similarity index 94% rename from roles/desk-nextcloud/vars/main.yml rename to roles/dsk-nextcloud/vars/main.yml index 9ef09b53b9..d506f6eb58 100644 --- a/roles/desk-nextcloud/vars/main.yml +++ b/roles/dsk-nextcloud/vars/main.yml @@ -1,5 +1,5 @@ --- -application_id: desk-nextcloud +application_id: dsk-nextcloud nextcloud_user_home_directory: "/home/{{ lookup('users', WORKSTATION_USER).username }}/" nextcloud_cloud_fqdn: "{{ lookup('config', application_id, 'secrets.credentials.cloud_fqdn') }}" nextcloud_cloud_directory: "{{ nextcloud_user_home_directory }}Clouds/{{ nextcloud_cloud_fqdn }}/{{ lookup('users', WORKSTATION_USER).username }}/" diff --git a/roles/desk-obs/README.md b/roles/dsk-obs/README.md similarity index 93% rename from roles/desk-obs/README.md rename to roles/dsk-obs/README.md index 835d7d7700..77fbf73b40 100644 --- a/roles/desk-obs/README.md +++ b/roles/dsk-obs/README.md @@ -15,7 +15,7 @@ The diagram places OBS Studio in the Infinito.Nexus cosmos: the components it de ```mermaid flowchart LR - subgraph role [desk-obs 💻] + subgraph role [dsk-obs 💻] svc_obs["obs"] end ``` @@ -26,7 +26,7 @@ Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional ( - **Streaming and recording:** Provides the upstream OBS Studio binary for both live broadcasting and local recording. - **Pacman integration:** Installs the `obs-studio` package via the standard system package manager. -- **Workstation scope:** Targets the desktop tier (`desk-*`) and stays out of server inventories. +- **Workstation scope:** Targets the desktop tier (`dsk-*`) and stays out of server inventories. - **No state changes beyond install:** Does not enable services or write per-user OBS configuration. ## Quick Setup @@ -39,7 +39,7 @@ Clone, set up the workstation, and deploy OBS Studio onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-obs full_cycle=false +make compose-deploy mode=reinstall apps=dsk-obs full_cycle=false ``` ### Production @@ -53,7 +53,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-obs +APP=dsk-obs DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-obs/meta/info.yml b/roles/dsk-obs/meta/info.yml similarity index 100% rename from roles/desk-obs/meta/info.yml rename to roles/dsk-obs/meta/info.yml diff --git a/roles/desk-obs/meta/main.yml b/roles/dsk-obs/meta/main.yml similarity index 100% rename from roles/desk-obs/meta/main.yml rename to roles/dsk-obs/meta/main.yml diff --git a/roles/desk-obs/meta/packages.yml b/roles/dsk-obs/meta/packages.yml similarity index 100% rename from roles/desk-obs/meta/packages.yml rename to roles/dsk-obs/meta/packages.yml diff --git a/roles/desk-obs/meta/services.yml b/roles/dsk-obs/meta/services.yml similarity index 100% rename from roles/desk-obs/meta/services.yml rename to roles/dsk-obs/meta/services.yml diff --git a/roles/desk-obs/tasks/main.yml b/roles/dsk-obs/tasks/main.yml similarity index 100% rename from roles/desk-obs/tasks/main.yml rename to roles/dsk-obs/tasks/main.yml diff --git a/roles/dsk-obs/vars/main.yml b/roles/dsk-obs/vars/main.yml new file mode 100644 index 0000000000..7123b3d04b --- /dev/null +++ b/roles/dsk-obs/vars/main.yml @@ -0,0 +1,2 @@ +--- +application_id: dsk-obs diff --git a/roles/desk-qbittorrent/README.md b/roles/dsk-qbittorrent/README.md similarity index 82% rename from roles/desk-qbittorrent/README.md rename to roles/dsk-qbittorrent/README.md index 4f037b9810..7afb094a1b 100644 --- a/roles/desk-qbittorrent/README.md +++ b/roles/dsk-qbittorrent/README.md @@ -6,7 +6,7 @@ Installs the qBittorrent torrent client from the distribution repositories. ## Overview -This README is for the `desk-qbittorrent` role within the `infinito` repository. This role is specifically crafted for installing qBittorrent, a popular open-source torrent client, on personal computers. +This README is for the `dsk-qbittorrent` role within the `infinito` repository. This role is specifically crafted for installing qBittorrent, a popular open-source torrent client, on personal computers. ## Cosmos @@ -14,7 +14,7 @@ The diagram places QBittorrent in the Infinito.Nexus cosmos: the components it d ```mermaid flowchart LR - subgraph role [desk-qbittorrent 💻] + subgraph role [dsk-qbittorrent 💻] svc_qbittorrent["qbittorrent"] end ``` @@ -35,7 +35,7 @@ Clone, set up the workstation, and deploy QBittorrent onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-qbittorrent full_cycle=false +make compose-deploy mode=reinstall apps=dsk-qbittorrent full_cycle=false ``` ### Production @@ -49,7 +49,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-qbittorrent +APP=dsk-qbittorrent DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" @@ -66,14 +66,14 @@ infinito administration deploy dedicated "$INVENTORY/devices.yml" \ ## Role Tasks -The `main.yml` file in the `desk-qbittorrent` role includes the following task: +The `main.yml` file in the `dsk-qbittorrent` role includes the following task: 1. **Install Torrent Software**: - This task calls `package_install` with the `qbittorrent` id; the per-distribution names are declared in [`meta/packages.yml`](meta/packages.yml). ## Purpose and Usage -The `desk-qbittorrent` role is tailored for users who require a reliable and user-friendly torrent client for downloading and sharing files via the BitTorrent protocol. qBittorrent is known for its balance of features, simplicity, and minimal impact on system resources. +The `dsk-qbittorrent` role is tailored for users who require a reliable and user-friendly torrent client for downloading and sharing files via the BitTorrent protocol. qBittorrent is known for its balance of features, simplicity, and minimal impact on system resources. ## Prerequisites @@ -85,7 +85,7 @@ The `desk-qbittorrent` role is tailored for users who require a reliable and use To utilize this role: 1. Clone the `infinito` repository. -2. Navigate to the `roles/desk-qbittorrent` directory. +2. Navigate to the `roles/dsk-qbittorrent` directory. 3. Execute the role using Ansible, ensuring you have the required system permissions for package installation. ## Customization diff --git a/roles/desk-qbittorrent/meta/info.yml b/roles/dsk-qbittorrent/meta/info.yml similarity index 100% rename from roles/desk-qbittorrent/meta/info.yml rename to roles/dsk-qbittorrent/meta/info.yml diff --git a/roles/desk-qbittorrent/meta/main.yml b/roles/dsk-qbittorrent/meta/main.yml similarity index 100% rename from roles/desk-qbittorrent/meta/main.yml rename to roles/dsk-qbittorrent/meta/main.yml diff --git a/roles/desk-qbittorrent/meta/packages.yml b/roles/dsk-qbittorrent/meta/packages.yml similarity index 100% rename from roles/desk-qbittorrent/meta/packages.yml rename to roles/dsk-qbittorrent/meta/packages.yml diff --git a/roles/desk-qbittorrent/meta/services.yml b/roles/dsk-qbittorrent/meta/services.yml similarity index 100% rename from roles/desk-qbittorrent/meta/services.yml rename to roles/dsk-qbittorrent/meta/services.yml diff --git a/roles/desk-qbittorrent/tasks/main.yml b/roles/dsk-qbittorrent/tasks/main.yml similarity index 74% rename from roles/desk-qbittorrent/tasks/main.yml rename to roles/dsk-qbittorrent/tasks/main.yml index d56b09ceb5..98656b4643 100644 --- a/roles/desk-qbittorrent/tasks/main.yml +++ b/roles/dsk-qbittorrent/tasks/main.yml @@ -5,4 +5,4 @@ id: qbittorrent - include_tasks: utils/once/flag.yml - when: run_once_desk_qbittorrent is not defined + when: run_once_dsk_qbittorrent is not defined diff --git a/roles/dsk-qbittorrent/vars/main.yml b/roles/dsk-qbittorrent/vars/main.yml new file mode 100644 index 0000000000..77a68a9ace --- /dev/null +++ b/roles/dsk-qbittorrent/vars/main.yml @@ -0,0 +1,2 @@ +--- +application_id: dsk-qbittorrent diff --git a/roles/desk-retroarch/README.md b/roles/dsk-retroarch/README.md similarity index 96% rename from roles/desk-retroarch/README.md rename to roles/dsk-retroarch/README.md index b097399486..f652088d66 100644 --- a/roles/desk-retroarch/README.md +++ b/roles/dsk-retroarch/README.md @@ -14,7 +14,7 @@ The diagram places RetroArch in the Infinito.Nexus cosmos: the components it dep ```mermaid flowchart LR - subgraph role [desk-retroarch 💻] + subgraph role [dsk-retroarch 💻] svc_retroarch["retroarch"] end ``` @@ -40,7 +40,7 @@ Clone, set up the workstation, and deploy RetroArch onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-retroarch full_cycle=false +make compose-deploy mode=reinstall apps=dsk-retroarch full_cycle=false ``` ### Production @@ -54,7 +54,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-retroarch +APP=dsk-retroarch DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-retroarch/meta/info.yml b/roles/dsk-retroarch/meta/info.yml similarity index 100% rename from roles/desk-retroarch/meta/info.yml rename to roles/dsk-retroarch/meta/info.yml diff --git a/roles/desk-retroarch/meta/main.yml b/roles/dsk-retroarch/meta/main.yml similarity index 100% rename from roles/desk-retroarch/meta/main.yml rename to roles/dsk-retroarch/meta/main.yml diff --git a/roles/desk-retroarch/meta/packages.yml b/roles/dsk-retroarch/meta/packages.yml similarity index 100% rename from roles/desk-retroarch/meta/packages.yml rename to roles/dsk-retroarch/meta/packages.yml diff --git a/roles/desk-retroarch/meta/services.yml b/roles/dsk-retroarch/meta/services.yml similarity index 100% rename from roles/desk-retroarch/meta/services.yml rename to roles/dsk-retroarch/meta/services.yml diff --git a/roles/desk-retroarch/tasks/main.yml b/roles/dsk-retroarch/tasks/main.yml similarity index 100% rename from roles/desk-retroarch/tasks/main.yml rename to roles/dsk-retroarch/tasks/main.yml diff --git a/roles/dsk-retroarch/vars/main.yml b/roles/dsk-retroarch/vars/main.yml new file mode 100644 index 0000000000..f275f84fa7 --- /dev/null +++ b/roles/dsk-retroarch/vars/main.yml @@ -0,0 +1,2 @@ +--- +application_id: dsk-retroarch diff --git a/roles/desk-spotify/README.md b/roles/dsk-spotify/README.md similarity index 96% rename from roles/desk-spotify/README.md rename to roles/dsk-spotify/README.md index aa597baad4..54c7df6e9e 100644 --- a/roles/desk-spotify/README.md +++ b/roles/dsk-spotify/README.md @@ -14,7 +14,7 @@ The diagram places Spotify in the Infinito.Nexus cosmos: the components it deplo ```mermaid flowchart LR - subgraph role [desk-spotify 💻] + subgraph role [dsk-spotify 💻] svc_spotify["spotify"] end ``` @@ -40,7 +40,7 @@ Clone, set up the workstation, and deploy Spotify onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-spotify full_cycle=false +make compose-deploy mode=reinstall apps=dsk-spotify full_cycle=false ``` ### Production @@ -54,7 +54,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-spotify +APP=dsk-spotify DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-spotify/meta/info.yml b/roles/dsk-spotify/meta/info.yml similarity index 100% rename from roles/desk-spotify/meta/info.yml rename to roles/dsk-spotify/meta/info.yml diff --git a/roles/desk-spotify/meta/main.yml b/roles/dsk-spotify/meta/main.yml similarity index 100% rename from roles/desk-spotify/meta/main.yml rename to roles/dsk-spotify/meta/main.yml diff --git a/roles/desk-spotify/meta/packages.yml b/roles/dsk-spotify/meta/packages.yml similarity index 100% rename from roles/desk-spotify/meta/packages.yml rename to roles/dsk-spotify/meta/packages.yml diff --git a/roles/desk-spotify/meta/services.yml b/roles/dsk-spotify/meta/services.yml similarity index 100% rename from roles/desk-spotify/meta/services.yml rename to roles/dsk-spotify/meta/services.yml diff --git a/roles/desk-spotify/tasks/main.yml b/roles/dsk-spotify/tasks/main.yml similarity index 74% rename from roles/desk-spotify/tasks/main.yml rename to roles/dsk-spotify/tasks/main.yml index 94410d3c08..6fa170f1e1 100644 --- a/roles/desk-spotify/tasks/main.yml +++ b/roles/dsk-spotify/tasks/main.yml @@ -5,4 +5,4 @@ id: spotify - include_tasks: utils/once/flag.yml - when: run_once_desk_spotify is not defined + when: run_once_dsk_spotify is not defined diff --git a/roles/dsk-spotify/vars/main.yml b/roles/dsk-spotify/vars/main.yml new file mode 100644 index 0000000000..77facbeb50 --- /dev/null +++ b/roles/dsk-spotify/vars/main.yml @@ -0,0 +1,2 @@ +--- +application_id: dsk-spotify diff --git a/roles/desk-ssh/README.md b/roles/dsk-ssh/README.md similarity index 94% rename from roles/desk-ssh/README.md rename to roles/dsk-ssh/README.md index 3d22d65f34..89ea96b909 100644 --- a/roles/desk-ssh/README.md +++ b/roles/dsk-ssh/README.md @@ -15,12 +15,12 @@ The diagram places SSH Agent in the Infinito.Nexus cosmos: the components it dep ```mermaid flowchart LR subgraph deps [Dependencies] - dep_desk_git["desk-git 💻 ⚙️"] + dep_dsk_git["dsk-git 💻 ⚙️"] end - subgraph role [desk-ssh 💻] + subgraph role [dsk-ssh 💻] svc_ssh["ssh"] end - dep_desk_git -- "1:1" --> svc_ssh + dep_dsk_git -- "1:1" --> svc_ssh ``` Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. @@ -31,7 +31,7 @@ The purpose of this role is to automate the provisioning of SSH agent capabiliti ## Features -- **Clones a remote SSH config repository** into `~/.ssh` using the `desk-git` role. +- **Clones a remote SSH config repository** into `~/.ssh` using the `dsk-git` role. - **Deploys and enables a systemd user service** for `ssh-agent`. - **Ensures environment compatibility** by injecting the `SSH_AUTH_SOCK` variable into either `.bash_profile` or `.profile`. - **Fails gracefully** with an optional debug message if the Git repository is unreachable. @@ -47,7 +47,7 @@ Clone, set up the workstation, and deploy SSH Agent onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-ssh full_cycle=false +make compose-deploy mode=reinstall apps=dsk-ssh full_cycle=false ``` ### Production @@ -61,7 +61,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-ssh +APP=dsk-ssh DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-ssh/meta/info.yml b/roles/dsk-ssh/meta/info.yml similarity index 100% rename from roles/desk-ssh/meta/info.yml rename to roles/dsk-ssh/meta/info.yml diff --git a/roles/desk-ssh/meta/main.yml b/roles/dsk-ssh/meta/main.yml similarity index 98% rename from roles/desk-ssh/meta/main.yml rename to roles/dsk-ssh/meta/main.yml index ab6a1c5594..b5e4213bd9 100644 --- a/roles/desk-ssh/meta/main.yml +++ b/roles/dsk-ssh/meta/main.yml @@ -33,4 +33,4 @@ galaxy_info: - keepassxc issue_tracker_url: https://s.infinito.nexus/issues dependencies: - - desk-git + - dsk-git diff --git a/roles/desk-ssh/meta/services.yml b/roles/dsk-ssh/meta/services.yml similarity index 100% rename from roles/desk-ssh/meta/services.yml rename to roles/dsk-ssh/meta/services.yml diff --git a/roles/desk-ssh/tasks/00_core.yml b/roles/dsk-ssh/tasks/00_core.yml similarity index 100% rename from roles/desk-ssh/tasks/00_core.yml rename to roles/dsk-ssh/tasks/00_core.yml diff --git a/roles/dsk-ssh/tasks/main.yml b/roles/dsk-ssh/tasks/main.yml new file mode 100644 index 0000000000..6aaa37d257 --- /dev/null +++ b/roles/dsk-ssh/tasks/main.yml @@ -0,0 +1,3 @@ +--- +- include_tasks: 00_core.yml + when: run_once_dsk_ssh is not defined diff --git a/roles/desk-ssh/vars/main.yml b/roles/dsk-ssh/vars/main.yml similarity index 90% rename from roles/desk-ssh/vars/main.yml rename to roles/dsk-ssh/vars/main.yml index d8bb7879e0..3208ab4b8f 100644 --- a/roles/desk-ssh/vars/main.yml +++ b/roles/dsk-ssh/vars/main.yml @@ -1,5 +1,5 @@ --- -application_id: desk-ssh +application_id: dsk-ssh system_service_id: "{{ application_id }}" desk_ssh_repository: "{{ lookup('config', application_id, 'services.ssh.repository') }}" desk_ssh_legacy_unit: "{{ [ DIR_SYSTEM_SERVICE_USER, 'ssh-agent.service' ] | path_join }}" diff --git a/roles/desk-torbrowser/README.md b/roles/dsk-torbrowser/README.md similarity index 93% rename from roles/desk-torbrowser/README.md rename to roles/dsk-torbrowser/README.md index 96a3fadf7c..caed0f6a36 100644 --- a/roles/desk-torbrowser/README.md +++ b/roles/dsk-torbrowser/README.md @@ -6,7 +6,7 @@ This Ansible role installs and configures the Tor service and the Tor Browser La ## Overview -The `desk-torbrowser` role uses the `community.general.pacman` module to: +The `dsk-torbrowser` role uses the `community.general.pacman` module to: 1. Install **tor** (the core Tor network service) 2. Install **torbrowser-launcher** (the launcher for Tor Browser) @@ -17,7 +17,7 @@ The diagram places Torbrowser in the Infinito.Nexus cosmos: the components it de ```mermaid flowchart LR - subgraph role [desk-torbrowser 💻] + subgraph role [dsk-torbrowser 💻] svc_torbrowser["torbrowser"] end ``` @@ -40,7 +40,7 @@ Clone, set up the workstation, and deploy Torbrowser onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-torbrowser full_cycle=false +make compose-deploy mode=reinstall apps=dsk-torbrowser full_cycle=false ``` ### Production @@ -54,7 +54,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-torbrowser +APP=dsk-torbrowser DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-torbrowser/meta/info.yml b/roles/dsk-torbrowser/meta/info.yml similarity index 100% rename from roles/desk-torbrowser/meta/info.yml rename to roles/dsk-torbrowser/meta/info.yml diff --git a/roles/desk-torbrowser/meta/main.yml b/roles/dsk-torbrowser/meta/main.yml similarity index 100% rename from roles/desk-torbrowser/meta/main.yml rename to roles/dsk-torbrowser/meta/main.yml diff --git a/roles/desk-torbrowser/meta/packages.yml b/roles/dsk-torbrowser/meta/packages.yml similarity index 100% rename from roles/desk-torbrowser/meta/packages.yml rename to roles/dsk-torbrowser/meta/packages.yml diff --git a/roles/desk-torbrowser/meta/services.yml b/roles/dsk-torbrowser/meta/services.yml similarity index 100% rename from roles/desk-torbrowser/meta/services.yml rename to roles/dsk-torbrowser/meta/services.yml diff --git a/roles/desk-torbrowser/tasks/main.yml b/roles/dsk-torbrowser/tasks/main.yml similarity index 100% rename from roles/desk-torbrowser/tasks/main.yml rename to roles/dsk-torbrowser/tasks/main.yml diff --git a/roles/dsk-torbrowser/vars/main.yml b/roles/dsk-torbrowser/vars/main.yml new file mode 100644 index 0000000000..909408233d --- /dev/null +++ b/roles/dsk-torbrowser/vars/main.yml @@ -0,0 +1,2 @@ +--- +application_id: dsk-torbrowser diff --git a/roles/desk-virtualbox/README.md b/roles/dsk-virtualbox/README.md similarity index 95% rename from roles/desk-virtualbox/README.md rename to roles/dsk-virtualbox/README.md index 0f6e7c5159..254d12ce4f 100644 --- a/roles/desk-virtualbox/README.md +++ b/roles/dsk-virtualbox/README.md @@ -20,7 +20,7 @@ The diagram places Virtual Box in the Infinito.Nexus cosmos: the components it d ```mermaid flowchart LR - subgraph role [desk-virtualbox 💻] + subgraph role [dsk-virtualbox 💻] svc_virtualbox["virtualbox"] end ``` @@ -41,7 +41,7 @@ Clone, set up the workstation, and deploy Virtual Box onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-virtualbox full_cycle=false +make compose-deploy mode=reinstall apps=dsk-virtualbox full_cycle=false ``` ### Production @@ -55,7 +55,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-virtualbox +APP=dsk-virtualbox DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-virtualbox/meta/info.yml b/roles/dsk-virtualbox/meta/info.yml similarity index 100% rename from roles/desk-virtualbox/meta/info.yml rename to roles/dsk-virtualbox/meta/info.yml diff --git a/roles/desk-virtualbox/meta/main.yml b/roles/dsk-virtualbox/meta/main.yml similarity index 100% rename from roles/desk-virtualbox/meta/main.yml rename to roles/dsk-virtualbox/meta/main.yml diff --git a/roles/desk-virtualbox/meta/services.yml b/roles/dsk-virtualbox/meta/services.yml similarity index 100% rename from roles/desk-virtualbox/meta/services.yml rename to roles/dsk-virtualbox/meta/services.yml diff --git a/roles/desk-virtualbox/tasks/00_core.yml b/roles/dsk-virtualbox/tasks/00_core.yml similarity index 100% rename from roles/desk-virtualbox/tasks/00_core.yml rename to roles/dsk-virtualbox/tasks/00_core.yml diff --git a/roles/desk-virtualbox/tasks/main.yml b/roles/dsk-virtualbox/tasks/main.yml similarity index 91% rename from roles/desk-virtualbox/tasks/main.yml rename to roles/dsk-virtualbox/tasks/main.yml index 1b4ed34933..6160b53c76 100644 --- a/roles/desk-virtualbox/tasks/main.yml +++ b/roles/dsk-virtualbox/tasks/main.yml @@ -1,7 +1,7 @@ --- - include_tasks: 00_core.yml when: - - run_once_desk_virtualbox is not defined + - run_once_dsk_virtualbox is not defined - ansible_facts['os_family'] == "Archlinux" - name: 🎭 WARNING - VirtualBox role is Arch-only, skipping on this host diff --git a/roles/dsk-virtualbox/vars/main.yml b/roles/dsk-virtualbox/vars/main.yml new file mode 100644 index 0000000000..62eaeab9a5 --- /dev/null +++ b/roles/dsk-virtualbox/vars/main.yml @@ -0,0 +1,2 @@ +--- +application_id: dsk-virtualbox diff --git a/roles/desk-zoom/README.md b/roles/dsk-zoom/README.md similarity index 95% rename from roles/desk-zoom/README.md rename to roles/dsk-zoom/README.md index 61c8095dc4..73a8a04138 100644 --- a/roles/desk-zoom/README.md +++ b/roles/dsk-zoom/README.md @@ -14,7 +14,7 @@ The diagram places Zoom in the Infinito.Nexus cosmos: the components it deploys ```mermaid flowchart LR - subgraph role [desk-zoom 💻] + subgraph role [dsk-zoom 💻] svc_zoom["zoom"] end ``` @@ -35,7 +35,7 @@ Clone, set up the workstation, and deploy Zoom onto the local stack: git clone https://github.com/infinito-nexus/core.git cd core make onboard -make compose-deploy mode=reinstall apps=desk-zoom full_cycle=false +make compose-deploy mode=reinstall apps=dsk-zoom full_cycle=false ``` ### Production @@ -49,7 +49,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP=desk-zoom +APP=dsk-zoom DOMAIN= TLS_MODE=self_signed SSH_PUBLIC_KEY="" diff --git a/roles/desk-zoom/meta/info.yml b/roles/dsk-zoom/meta/info.yml similarity index 100% rename from roles/desk-zoom/meta/info.yml rename to roles/dsk-zoom/meta/info.yml diff --git a/roles/desk-zoom/meta/main.yml b/roles/dsk-zoom/meta/main.yml similarity index 100% rename from roles/desk-zoom/meta/main.yml rename to roles/dsk-zoom/meta/main.yml diff --git a/roles/desk-zoom/meta/packages.yml b/roles/dsk-zoom/meta/packages.yml similarity index 100% rename from roles/desk-zoom/meta/packages.yml rename to roles/dsk-zoom/meta/packages.yml diff --git a/roles/desk-zoom/meta/services.yml b/roles/dsk-zoom/meta/services.yml similarity index 100% rename from roles/desk-zoom/meta/services.yml rename to roles/dsk-zoom/meta/services.yml diff --git a/roles/desk-zoom/tasks/main.yml b/roles/dsk-zoom/tasks/main.yml similarity index 77% rename from roles/desk-zoom/tasks/main.yml rename to roles/dsk-zoom/tasks/main.yml index bae4ad0ba4..7370f9fe29 100644 --- a/roles/desk-zoom/tasks/main.yml +++ b/roles/dsk-zoom/tasks/main.yml @@ -5,4 +5,4 @@ id: zoom - include_tasks: utils/once/flag.yml - when: run_once_desk_zoom is not defined + when: run_once_dsk_zoom is not defined diff --git a/roles/dsk-zoom/vars/main.yml b/roles/dsk-zoom/vars/main.yml new file mode 100644 index 0000000000..b02a86835b --- /dev/null +++ b/roles/dsk-zoom/vars/main.yml @@ -0,0 +1,2 @@ +--- +application_id: dsk-zoom diff --git a/roles/gen-hunspell/README.md b/roles/gen-hunspell/README.md index 30e5aee231..60717e736e 100644 --- a/roles/gen-hunspell/README.md +++ b/roles/gen-hunspell/README.md @@ -18,9 +18,9 @@ flowchart LR svc_hunspell["hunspell"] end subgraph dependents [Dependents] - dpt_desk_libreoffice["desk-libreoffice 💻 ⚙️"] + dpt_dsk_libreoffice["dsk-libreoffice 💻 ⚙️"] end - svc_hunspell -- "1:1" --> dpt_desk_libreoffice + svc_hunspell -- "1:1" --> dpt_dsk_libreoffice ``` Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. diff --git a/roles/svc-ai-agent-broker/README.md b/roles/svc-ai-agent-broker/README.md new file mode 100644 index 0000000000..ddcc301c4c --- /dev/null +++ b/roles/svc-ai-agent-broker/README.md @@ -0,0 +1,96 @@ +# Agent Broker + +## Description + +Agent Broker is an OpenAI-compatible endpoint that gives every entitled user a personal Hermes or OpenClaw agent, started on the first request and addressed by the model name `hermes` or `openclaw`. + +## Overview + +This role deploys the broker together with a container-API socket proxy in Compose and Swarm deployments. Open WebUI reaches the broker as a second OpenAI connection and forwards the caller's identity. The broker checks the caller's Keycloak group, creates or starts that caller's agent under the isolating runtime, forwards the request to it and streams the answer back. Agents reach their model only through the broker, which relays each call to the shared LiteLLM gateway with its own key and tags it with the owner. + +## Cosmos + +The diagram places Agent Broker in the Infinito.Nexus cosmos: the components it deploys (capabilities), the central services it consumes (dependencies), and its outward reach (federation and bridged external networks). + +```mermaid +flowchart LR + subgraph deps [Dependencies] + dep_svc_ai_litellm["svc-ai-litellm 🐳🐝"] + dep_web_app_hermes["web-app-hermes 🐳🐝"] + dep_web_app_keycloak["web-app-keycloak 🐳🐝"] + dep_web_app_openclaw["web-app-openclaw 🐳🐝"] + end + subgraph role [svc-ai-agent-broker 🐳🐝] + svc_agent_broker["agent-broker"] + svc_socket_proxy["socket-proxy"] + svc_litellm["litellm"] + svc_sso["sso"] + svc_hermes["hermes"] + svc_openclaw["openclaw"] + end + subgraph dependents [Dependents] + dpt_web_app_openwebui["web-app-openwebui 🐳🐝"] + end + dep_svc_ai_litellm -- "1:1" --> svc_litellm + dep_web_app_hermes -- "1:1" --> svc_hermes + dep_web_app_keycloak -- "1:1" --> svc_sso + dep_web_app_openclaw -- "1:1" --> svc_openclaw + svc_agent_broker -. "0..1" .-> dpt_web_app_openwebui +``` + +Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. + +## Features + +- **Agents on demand:** The first completion of a user for `hermes` or `openclaw` creates that user's agent; later requests reuse it, and a stopped agent is started again with its state volume re-attached. +- **RBAC gate:** A user needs the `agent-user` role of `web-app-hermes` or `web-app-openclaw`. Open WebUI lists each agent model only to that group, and the broker refuses any other caller with HTTP 403 and stops a running agent of a user who lost the role. +- **Isolation:** Every agent runs in its own container or single-replica service under the isolating runtime, with its own volume, its own network whose only other member is the broker, and its own bearer key; an agent that would start under `runc` is stopped and refused. +- **Restricted engine access:** The broker reaches the container engine only through a filtered unix socket in a volume shared with the socket proxy, which publishes no port and runs with `network_mode: none` in compose. The proxy admits only the container, service, task, network, volume and image calls the broker makes, and refuses every other method and path, including exec and delete. It filters methods and paths, not request bodies: `volumes/create` stays open for the per-agent volume, so the isolation holds against a compromised agent, which reaches no socket at all, not against a compromised broker. +- **Lifecycle settings:** `agents.idle_stop`, `agents.idle_minutes`, `agents.max_running`, `agents.start_timeout` and `agents.access_cache_seconds` in `meta/services.yml` control idle stops, the running-agent bound, the start wait and the membership cache; an inventory overrides them under `applications.svc-ai-agent-broker.services.agent-broker.agents`. An agent serving a request is never swept. +- **Model and context:** `agents.model` names the gateway alias every agent runs on and defaults to the gateway's chat model, so agents can run on a larger model than Open WebUI's default. The context window of that alias comes from the model's `context` in the inventory and is written into the agent config as Hermes `model.context_length` and OpenClaw `contextWindow`. An agent prompt runs to tens of thousands of tokens, so a CPU-only local model spends minutes per turn on it; point `agents.model` at a remote model for interactive use. Hermes refuses a window below 64000 tokens and answers HTTP 500 on every prompt, so a platform whose minimum the selected alias misses is not offered at all: it is left out of `/v1/models` and answers 404, and the deploy says which platform it dropped and why. The rest of the broker keeps working, which is what makes a small model and OpenClaw a valid pairing. An alias that declares no window is reported instead of filtered, because an agent that does not know its window runs. +- **Model relay:** Agents call `http://agent-broker:/llm/v1` with their own key; the broker forwards the call to LiteLLM with the broker's virtual key, sets the OpenAI `user` field to the owner and logs a JSON `relay` event with owner, platform, path and upstream status. + +## Quick Setup + +### Development + +Clone, set up the workstation, and deploy Agent Broker onto the local stack: + +```bash +git clone https://github.com/infinito-nexus/core.git +cd core +make onboard +make compose-deploy mode=reinstall apps=svc-ai-agent-broker,svc-ai-ollama full_cycle=false +``` + +### Production + +Run the published image to provision the inventory and deploy Agent Broker to a managed server (the mounted volume persists the inventory): + +```bash +APP=svc-ai-agent-broker,svc-ai-ollama +HOST= +DOMAIN= +TLS_MODE=self_signed +SSH_PUBLIC_KEY="" + +docker run --rm -it \ + -v "$PWD/inventories:/etc/infinito.nexus/inventories" \ + -e APP="$APP" -e HOST="$HOST" -e DOMAIN="$DOMAIN" -e TLS_MODE="$TLS_MODE" -e SSH_PUBLIC_KEY="$SSH_PUBLIC_KEY" \ + ghcr.io/infinito-nexus/core/debian bash -c ' + INVENTORY=/etc/infinito.nexus/inventories/production + infinito administration inventory provision "$INVENTORY" \ + --inventory-file "$INVENTORY/devices.yml" \ + --host "$HOST" \ + --include "$APP" \ + --vars "{\"TLS_MODE\": \"$TLS_MODE\", \"DOMAIN_PRIMARY\": \"$DOMAIN\", \"users\": {\"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}" && + infinito administration deploy dedicated "$INVENTORY/devices.yml" \ + --password-file "$INVENTORY/.password" \ + --diff -vv' +``` + +## Credits + +Implemented by **[Kevin Veen-Birkenbach](https://www.veen.world)**. +Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world). +Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license). diff --git a/roles/svc-ai-agent-broker/files/Dockerfile b/roles/svc-ai-agent-broker/files/Dockerfile new file mode 100644 index 0000000000..cbfb0f8263 --- /dev/null +++ b/roles/svc-ai-agent-broker/files/Dockerfile @@ -0,0 +1,11 @@ +ARG AGENT_BROKER_BASE_IMAGE +ARG AGENT_BROKER_BASE_VERSION +FROM ${AGENT_BROKER_BASE_IMAGE}:${AGENT_BROKER_BASE_VERSION} + +WORKDIR /opt/broker +COPY access.py agents.py engine.py server.py /opt/broker/ + +RUN adduser --system --no-create-home --disabled-login broker +USER broker + +ENTRYPOINT ["python3", "/opt/broker/server.py"] diff --git a/roles/svc-ai-agent-broker/files/python/access.py b/roles/svc-ai-agent-broker/files/python/access.py new file mode 100644 index 0000000000..477ad1cff5 --- /dev/null +++ b/roles/svc-ai-agent-broker/files/python/access.py @@ -0,0 +1,104 @@ +from __future__ import annotations + +import json +import threading +import time +import urllib.error +import urllib.parse +import urllib.request + + +class AccessError(RuntimeError): + pass + + +class Keycloak: + """ + Args: + base_url: Keycloak base URL. + realm: realm holding the platform users. + username: master-realm administrator. + password: its password. + cache_seconds: how long a membership answer is reused. + timeout: seconds per HTTP call. + """ + + def __init__(self, base_url, realm, username, password, cache_seconds, timeout): + self.base_url = base_url.rstrip("/") + self.realm = realm + self.username = username + self.password = password + self.cache_seconds = cache_seconds + self.timeout = timeout + self._token = "" + self._token_expiry = 0.0 + self._cache = {} + self._lock = threading.Lock() + + def _request(self, method, url, data=None, headers=None): + request = urllib.request.Request(url, data=data, method=method) # noqa: S310 - https:// base fixed by KEYCLOAK_URL + for key, value in (headers or {}).items(): + request.add_header(key, value) + try: + with urllib.request.urlopen(request, timeout=self.timeout) as response: # noqa: S310 - https:// base fixed by KEYCLOAK_URL + return response.status, json.loads(response.read() or b"null") + except urllib.error.HTTPError as error: + return error.code, error.read().decode(errors="replace") + except urllib.error.URLError as error: + raise AccessError( + f"Keycloak unreachable at {url}: {error.reason}" + ) from error + + def _admin_token(self): + if self._token and time.monotonic() < self._token_expiry: + return self._token + status, body = self._request( + "POST", + f"{self.base_url}/realms/master/protocol/openid-connect/token", + data=urllib.parse.urlencode( + { + "client_id": "admin-cli", + "grant_type": "password", + "username": self.username, + "password": self.password, + } + ).encode(), + headers={"Content-Type": "application/x-www-form-urlencoded"}, + ) + if status != 200: + raise AccessError(f"Keycloak admin token refused: {status}") + self._token = body["access_token"] + self._token_expiry = time.monotonic() + max(int(body["expires_in"]) - 30, 10) + return self._token + + def _get(self, path, query): + url = f"{self.base_url}/admin/realms/{self.realm}{path}?{urllib.parse.urlencode(query)}" + status, body = self._request( + "GET", url, headers={"Authorization": f"Bearer {self._admin_token()}"} + ) + if status != 200: + raise AccessError(f"Keycloak GET {path} -> {status}") + return body + + def groups_of(self, email): + key = email.lower() + with self._lock: + cached = self._cache.get(key) + if cached and time.monotonic() - cached[0] < self.cache_seconds: + return cached[1] + users = self._get("/users", {"email": key, "exact": "true"}) + if len(users) != 1: + groups = frozenset() + else: + groups = frozenset( + group["path"] + for group in self._get( + f"/users/{users[0]['id']}/groups", + {"briefRepresentation": "true", "max": "1000"}, + ) + ) + self._cache[key] = (time.monotonic(), groups) + return groups + + def allows(self, email, group_path): + return bool(email) and group_path in self.groups_of(email) diff --git a/roles/svc-ai-agent-broker/files/python/agents.py b/roles/svc-ai-agent-broker/files/python/agents.py new file mode 100644 index 0000000000..857ea7b2a5 --- /dev/null +++ b/roles/svc-ai-agent-broker/files/python/agents.py @@ -0,0 +1,315 @@ +from __future__ import annotations + +import hashlib +import json +import secrets +import threading +import time +import urllib.error +import urllib.request +from datetime import datetime + +from engine import LABEL_OWNER, LABEL_PLATFORM + +KEY_ENV = "AGENT_BROKER_KEY" +CONFIG_ENV = "AGENT_CONFIG" +CONFIG_PATH_ENV = "AGENT_CONFIG_PATH" +WRITE_CONFIG = ( + 'mkdir -p "$(dirname "$AGENT_CONFIG_PATH")" && ' + 'printf "%s" "$AGENT_CONFIG" > "$AGENT_CONFIG_PATH" && exec "$@"' +) + + +class CapacityError(RuntimeError): + pass + + +def agent_name(platform, owner): + digest = hashlib.sha256(owner.encode()).hexdigest()[:16] + return f"agent-{platform}-{digest}" + + +def hermes_config(model, relay_url, key, context): + lines = [ + "model:", + f" default: {json.dumps(model)}", + " provider: custom", + f" base_url: {json.dumps(relay_url)}", + f" api_key: {json.dumps(key)}", + ] + if context: + lines.append(f" context_length: {context}") + lines += ["mcp_servers: {}", ""] + return "\n".join(lines) + + +def openclaw_config(model, relay_url, key, context): + entry = {"id": model, "name": model} + if context: + entry["contextWindow"] = context + return json.dumps( + { + "gateway": { + "mode": "local", + "http": {"endpoints": {"chatCompletions": {"enabled": True}}}, + }, + "agents": {"defaults": {"model": {"primary": f"broker/{model}"}}}, + "models": { + "providers": { + "broker": { + "baseUrl": relay_url, + "apiKey": key, + "api": "openai-completions", + "models": [entry], + } + } + }, + } + ) + + +CONFIG_RENDERERS = {"hermes": hermes_config, "openclaw": openclaw_config} + + +def _env_value(env, name): + prefix = f"{name}=" + return next((item[len(prefix) :] for item in env if item.startswith(prefix)), "") + + +def _parse_time(value): + try: + return datetime.fromisoformat(value[:19] + "+00:00").timestamp() + except ValueError: + return 0.0 + + +class Agents: + """ + Args: + backend: :class:`engine.ComposeBackend` or :class:`engine.SwarmBackend`. + engine: the :class:`engine.Engine` behind it. + platforms: platform name -> spec from ``AGENT_PLATFORMS``. + self_container: the broker's own container, joined to each agent network. + broker_alias: name agents resolve the broker by. + relay_url: base URL agents send model calls to. + model: model alias every agent asks the relay for. + context: context window of that model in tokens; 0 leaves it to the agent. + idle_stop: whether idle agents are stopped. + idle_seconds: idle time before a stop. + max_running: bound on concurrently running agents. + start_timeout: seconds an agent gets to answer its health endpoint. + """ + + def __init__( + self, + backend, + engine, + platforms, + self_container, + broker_alias, + relay_url, + model, + context, + idle_stop, + idle_seconds, + max_running, + start_timeout, + ): + self.backend = backend + self.engine = engine + self.platforms = platforms + self.self_container = self_container + self.broker_alias = broker_alias + self.relay_url = relay_url + self.model = model + self.context = context + self.idle_stop = idle_stop + self.idle_seconds = idle_seconds + self.max_running = max_running + self.start_timeout = start_timeout + self._locks = {} + self._locks_guard = threading.Lock() + self._busy = {} + self._last_used = {} + self._keys = {} + self._agent_models = {} + + def _lock_for(self, name): + with self._locks_guard: + return self._locks.setdefault(name, threading.Lock()) + + def _spec(self, platform, owner, key): + platform_spec = self.platforms[platform] + name = agent_name(platform, owner) + entrypoint, cmd = self.engine.image_config(platform_spec["image"]) + command = list(platform_spec["command"]) or cmd + config = CONFIG_RENDERERS[platform]( + self.model, self.relay_url, key, self.context + ) + env = [f"{k}={v}" for k, v in sorted(platform_spec["env"].items())] + env += [ + f"{platform_spec['key_env']}={key}", + f"{KEY_ENV}={key}", + f"{CONFIG_ENV}={config}", + f"{CONFIG_PATH_ENV}={platform_spec['config_path']}", + ] + return { + "name": name, + "network": name, + "volume": name, + "image": platform_spec["image"], + "entrypoint": ["/bin/sh", "-c", WRITE_CONFIG, "agent"], + "cmd": [str(part) for part in entrypoint + command], + "env": env, + "labels": {LABEL_PLATFORM: platform, LABEL_OWNER: owner}, + "data": platform_spec["data"], + "user": platform_spec["user"], + "nano_cpus": int(float(platform_spec["cpus"]) * 1_000_000_000), + "memory": int(platform_spec["mem_limit"]), + "memory_reservation": int(platform_spec["mem_reservation"]), + "pids": int(platform_spec["pids_limit"]), + } + + def _running_count(self): + return sum( + 1 + for detail in self.backend.list_agents() + if self.backend.is_running(detail) + ) + + def _remember(self, detail): + labels = self.backend.labels_of(detail) + key = _env_value(self.backend.env_of(detail), KEY_ENV) + if key: + self._keys[key] = ( + labels.get(LABEL_OWNER, ""), + labels.get(LABEL_PLATFORM, ""), + ) + return key + + def _wait_healthy(self, platform, name, key): + spec = self.platforms[platform] + url = f"http://{name}:{spec['port']}{spec['health']}" + deadline = time.monotonic() + self.start_timeout + while time.monotonic() < deadline: + request = urllib.request.Request(url) + request.add_header("Authorization", f"Bearer {key}") + try: + with urllib.request.urlopen(request, timeout=5) as response: # noqa: S310 - http:// URL built from the agent name and port + if response.status < 400: + return + except (urllib.error.URLError, OSError): + pass + time.sleep(2) + raise TimeoutError( + f"{name} did not answer {spec['health']} within {self.start_timeout}s" + ) + + def ensure(self, platform, owner): + """Return ``(address, key)`` of the caller's running agent. + + Args: + platform: ``hermes`` or ``openclaw``. + owner: Open WebUI user id of the caller. + """ + name = agent_name(platform, owner) + with self._lock_for(name): + detail = self.backend.find(name) + if detail is None: + if self._running_count() >= self.max_running: + raise CapacityError(f"{self.max_running} agents already running") + key = secrets.token_urlsafe(32) + network_id = self.engine.ensure_network( + name, + self.backend.network_driver, + {LABEL_PLATFORM: platform, LABEL_OWNER: owner}, + ) + self.engine.connect(network_id, self.self_container, self.broker_alias) + detail = self.backend.create( + self._spec(platform, owner, key), network_id + ) + else: + network_id = self.engine.ensure_network( + name, + self.backend.network_driver, + {LABEL_PLATFORM: platform, LABEL_OWNER: owner}, + ) + self.engine.connect(network_id, self.self_container, self.broker_alias) + key = self._remember(detail) + if not self.backend.is_running(detail): + if self._running_count() >= self.max_running: + raise CapacityError(f"{self.max_running} agents already running") + self.backend.start(detail) + self._wait_healthy(platform, name, key) + self._last_used[name] = time.monotonic() + return f"http://{name}:{self.platforms[platform]['port']}", key + + def agent_model(self, platform, address, key): + if platform in self._agent_models: + return self._agent_models[platform] + request = urllib.request.Request(f"{address}/v1/models") # noqa: S310 - http:// URL built from the agent name and port + request.add_header("Authorization", f"Bearer {key}") + model = self.platforms[platform]["request_model"] + try: + with urllib.request.urlopen(request, timeout=10) as response: # noqa: S310 - http:// URL built from the agent name and port + listed = json.loads(response.read()).get("data") or [] + if listed: + model = listed[0]["id"] + except (urllib.error.URLError, OSError, ValueError, KeyError): + pass + self._agent_models[platform] = model + return model + + def owner_of_key(self, key): + if key not in self._keys: + for detail in self.backend.list_agents(): + self._remember(detail) + return self._keys.get(key) + + def begin(self, platform, owner): + name = agent_name(platform, owner) + with self._locks_guard: + self._busy[name] = self._busy.get(name, 0) + 1 + return name + + def end(self, name): + with self._locks_guard: + self._busy[name] = max(self._busy.get(name, 1) - 1, 0) + self._last_used[name] = time.monotonic() + + def stop_owner(self, platform, owner): + detail = self.backend.find(agent_name(platform, owner)) + if detail is not None and self.backend.is_running(detail): + self.backend.stop(detail) + + def _idle_seconds_of(self, name, detail): + last = self._last_used.get(name) + if last is not None: + return time.monotonic() - last + return time.time() - _parse_time(self.backend.started_at(detail)) + + def reap(self): + if not self.idle_stop: + return + for detail in self.backend.list_agents(): + if not self.backend.is_running(detail): + continue + name = detail.get("Name", "").lstrip("/") or (detail.get("Spec") or {}).get( + "Name", "" + ) + if ( + self._busy.get(name) + or self._idle_seconds_of(name, detail) < self.idle_seconds + ): + continue + with self._lock_for(name): + current = self.backend.find(name) + if current is None or not self.backend.is_running(current): + continue + if ( + self._busy.get(name) + or self._idle_seconds_of(name, current) < self.idle_seconds + ): + continue + self.backend.stop(current) + self._last_used.pop(name, None) diff --git a/roles/svc-ai-agent-broker/files/python/engine.py b/roles/svc-ai-agent-broker/files/python/engine.py new file mode 100644 index 0000000000..18bd7ce585 --- /dev/null +++ b/roles/svc-ai-agent-broker/files/python/engine.py @@ -0,0 +1,379 @@ +from __future__ import annotations + +import http.client +import json +import socket +import time +import urllib.parse + +LABEL_PLATFORM = "infinito.agent.platform" +LABEL_OWNER = "infinito.agent.owner" + + +class EngineError(RuntimeError): + """Args: + message: what the call was and how it answered. + status: HTTP status the engine returned, or None when it never answered. + """ + + def __init__(self, message, status=None): + super().__init__(message) + self.status = status + + +class _UnixConnection(http.client.HTTPConnection): + def __init__(self, path, timeout): + super().__init__("localhost", timeout=timeout) + self._path = path + + def connect(self): + sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + sock.settimeout(self.timeout) + sock.connect(self._path) + self.sock = sock + + +class Engine: + """ + Args: + socket_path: filtered engine socket the socket proxy serves. + timeout: seconds per API call. + """ + + def __init__(self, socket_path: str, timeout: float): + self.socket_path = socket_path + self.timeout = timeout + + def call( + self, method, path, body=None, query=None, expect=(200, 201, 204), timeout=None + ): + target = f"{path}?{urllib.parse.urlencode(query)}" if query else path + data = json.dumps(body).encode() if body is not None else None + headers = {"Content-Type": "application/json"} if data is not None else {} + connection = _UnixConnection(self.socket_path, timeout or self.timeout) + try: + connection.request(method, target, body=data, headers=headers) + response = connection.getresponse() + raw = response.read() + status = response.status + except OSError as error: + raise EngineError(f"{method} {path} failed: {error}") from error + finally: + connection.close() + if status not in expect: + raise EngineError(f"{method} {path} -> {status}: {raw[:400]!r}", status) + if not raw: + return None + try: + return json.loads(raw) + except ValueError: + return raw.decode(errors="replace") + + def image_config(self, ref): + quoted = urllib.parse.quote(ref, safe="") + try: + info = self.call("GET", f"/images/{quoted}/json") + except EngineError: + name, _, tag = ref.rpartition(":") + self.call( + "POST", + "/images/create", + query={"fromImage": name, "tag": tag}, + timeout=1800, + ) + info = self.call("GET", f"/images/{quoted}/json") + config = info.get("Config") or {} + return list(config.get("Entrypoint") or []), list(config.get("Cmd") or []) + + def ensure_network(self, name, driver, labels): + found = self.call( + "GET", + "/networks", + query={"filters": json.dumps({"name": [name]})}, + ) + for network in found or []: + if network.get("Name") == name: + return network["Id"] + created = self.call( + "POST", + "/networks/create", + body={ + "Name": name, + "Driver": driver, + "Attachable": driver == "overlay", + "CheckDuplicate": True, + "Labels": labels, + }, + ) + return created["Id"] + + def connect(self, network_id, container, alias, realize_timeout=120, pause=2): + """Attach ``container`` to ``network_id`` once the network is usable. + + Exception: a swarm-scoped overlay answers GET from the control plane + before its allocator has realized it on any node, while connect is + node-local, so connect returns 404 for a network that does exist. A 404 + that outlives ``realize_timeout`` is raised, because then the network + really is gone. + + Args: + network_id: id ``ensure_network`` returned. + container: container id or name to attach. + alias: name the other members of the network reach it by. + realize_timeout: seconds to allow the swarm allocator. + pause: seconds between attempts. + """ + deadline = time.monotonic() + realize_timeout + while True: + detail = self.call("GET", f"/networks/{network_id}") + attached = detail.get("Containers") or {} + if any(key.startswith(container) for key in attached): + return + try: + self.call( + "POST", + f"/networks/{network_id}/connect", + body={ + "Container": container, + "EndpointConfig": {"Aliases": [alias]}, + }, + expect=(200, 204), + ) + except EngineError as error: + if error.status != 404 or time.monotonic() >= deadline: + raise + time.sleep(pause) + continue + return + + +class ComposeBackend: + """ + Args: + engine: the :class:`Engine`. + runtime: OCI runtime every agent must run under (``runsc``). + """ + + network_driver = "bridge" + + def __init__(self, engine, runtime): + self.engine = engine + self.runtime = runtime + + def find(self, name): + found = self.engine.call( + "GET", + "/containers/json", + query={"all": "true", "filters": json.dumps({"name": [name]})}, + ) + for item in found or []: + if f"/{name}" in (item.get("Names") or []): + return self.engine.call("GET", f"/containers/{item['Id']}/json") + return None + + def list_agents(self): + found = self.engine.call( + "GET", + "/containers/json", + query={"all": "true", "filters": json.dumps({"label": [LABEL_PLATFORM]})}, + ) + return [ + self.engine.call("GET", f"/containers/{item['Id']}/json") + for item in found or [] + ] + + @staticmethod + def env_of(detail): + return list((detail.get("Config") or {}).get("Env") or []) + + @staticmethod + def labels_of(detail): + return dict((detail.get("Config") or {}).get("Labels") or {}) + + @staticmethod + def is_running(detail): + return bool((detail.get("State") or {}).get("Running")) + + @staticmethod + def started_at(detail): + return (detail.get("State") or {}).get("StartedAt") or "" + + def create(self, spec, network_id): + self.engine.call( + "POST", + "/volumes/create", + body={"Name": spec["volume"], "Labels": spec["labels"]}, + ) + body = { + "Image": spec["image"], + "Entrypoint": spec["entrypoint"], + "Cmd": spec["cmd"], + "Env": spec["env"], + "Labels": spec["labels"], + "HostConfig": { + "Runtime": self.runtime, + "Mounts": [ + {"Type": "volume", "Source": spec["volume"], "Target": spec["data"]} + ], + "NanoCpus": spec["nano_cpus"], + "Memory": spec["memory"], + "MemoryReservation": spec["memory_reservation"], + "PidsLimit": spec["pids"], + "RestartPolicy": {"Name": "unless-stopped"}, + "SecurityOpt": ["no-new-privileges:true"], + "NetworkMode": spec["network"], + }, + "NetworkingConfig": { + "EndpointsConfig": {network_id: {"Aliases": [spec["name"]]}} + }, + } + if spec["user"]: + body["User"] = spec["user"] + self.engine.call( + "POST", "/containers/create", body=body, query={"name": spec["name"]} + ) + return self.find(spec["name"]) + + def start(self, detail): + self.engine.call("POST", f"/containers/{detail['Id']}/start", expect=(204, 304)) + started = self.engine.call("GET", f"/containers/{detail['Id']}/json") + runtime = (started.get("HostConfig") or {}).get("Runtime") + if runtime != self.runtime: + self.engine.call( + "POST", f"/containers/{detail['Id']}/stop", expect=(204, 304) + ) + raise EngineError( + f"{started.get('Name')} came up under '{runtime}', not " + f"'{self.runtime}'; stopped it rather than run it on the host kernel" + ) + return started + + def stop(self, detail): + self.engine.call( + "POST", + f"/containers/{detail['Id']}/stop", + query={"t": "30"}, + expect=(204, 304), + ) + + +class SwarmBackend: + """ + Args: + engine: the :class:`Engine`. + constraint: placement constraint of the sandbox nodes. + """ + + network_driver = "overlay" + + def __init__(self, engine, constraint): + self.engine = engine + self.constraint = constraint + + def find(self, name): + found = self.engine.call( + "GET", + "/services", + query={"filters": json.dumps({"name": [name]})}, + ) + for item in found or []: + if (item.get("Spec") or {}).get("Name") == name: + return item + return None + + def list_agents(self): + return ( + self.engine.call( + "GET", + "/services", + query={"filters": json.dumps({"label": [LABEL_PLATFORM]})}, + ) + or [] + ) + + @staticmethod + def env_of(detail): + spec = (detail.get("Spec") or {}).get("TaskTemplate") or {} + return list((spec.get("ContainerSpec") or {}).get("Env") or []) + + @staticmethod + def labels_of(detail): + return dict((detail.get("Spec") or {}).get("Labels") or {}) + + def is_running(self, detail): + mode = (detail.get("Spec") or {}).get("Mode") or {} + if ((mode.get("Replicated") or {}).get("Replicas") or 0) < 1: + return False + tasks = self.engine.call( + "GET", + "/tasks", + query={ + "filters": json.dumps( + {"service": [detail["ID"]], "desired-state": ["running"]} + ) + }, + ) + return any( + (task.get("Status") or {}).get("State") == "running" for task in tasks or [] + ) + + @staticmethod + def started_at(detail): + return detail.get("UpdatedAt") or "" + + def create(self, spec, network_id): + body = { + "Name": spec["name"], + "Labels": spec["labels"], + "TaskTemplate": { + "ContainerSpec": { + "Image": spec["image"], + "Command": spec["entrypoint"], + "Args": spec["cmd"], + "Env": spec["env"], + "Labels": spec["labels"], + "Mounts": [ + { + "Type": "volume", + "Source": spec["volume"], + "Target": spec["data"], + } + ], + "Privileges": {"NoNewPrivileges": True}, + }, + "Resources": { + "Limits": { + "NanoCPUs": spec["nano_cpus"], + "MemoryBytes": spec["memory"], + "Pids": spec["pids"], + }, + "Reservations": {"MemoryBytes": spec["memory_reservation"]}, + }, + "Placement": {"Constraints": [self.constraint]}, + "RestartPolicy": {"Condition": "any"}, + "Networks": [{"Target": network_id, "Aliases": [spec["name"]]}], + }, + "Mode": {"Replicated": {"Replicas": 0}}, + } + if spec["user"]: + body["TaskTemplate"]["ContainerSpec"]["User"] = spec["user"] + self.engine.call("POST", "/services/create", body=body) + return self.find(spec["name"]) + + def _scale(self, detail, replicas): + current = self.engine.call("GET", f"/services/{detail['ID']}") + spec = current["Spec"] + spec["Mode"] = {"Replicated": {"Replicas": replicas}} + self.engine.call( + "POST", + f"/services/{detail['ID']}/update", + body=spec, + query={"version": current["Version"]["Index"]}, + ) + return self.engine.call("GET", f"/services/{detail['ID']}") + + def start(self, detail): + return self._scale(detail, 1) + + def stop(self, detail): + self._scale(detail, 0) diff --git a/roles/svc-ai-agent-broker/files/python/server.py b/roles/svc-ai-agent-broker/files/python/server.py new file mode 100644 index 0000000000..8b81477d90 --- /dev/null +++ b/roles/svc-ai-agent-broker/files/python/server.py @@ -0,0 +1,265 @@ +# nocheck: mirrored-unit-test - builds its engine, agents and access clients from the environment at import and is exercised end to end by files/test/test.sh +""" +Environment: + BROKER_PORT: port to listen on. + BROKER_KEY: bearer Open WebUI presents. + BROKER_ALIAS: name agents resolve the broker by. + ENGINE_SOCKET: filtered engine socket of the socket proxy. + ENGINE_MODE: ``compose`` or ``swarm``. + AGENT_RUNTIME: OCI runtime of compose agents. + AGENT_CONSTRAINT: placement constraint of swarm agents. + LITELLM_URL: gateway base URL. + LITELLM_KEY: the broker's own gateway key. + AGENT_MODEL: model alias agents are configured with. + AGENT_CONTEXT: context window of that model in tokens; empty leaves it to the agent. + AGENT_PLATFORMS: JSON platform name -> spec. + KEYCLOAK_URL: Keycloak base URL. + KEYCLOAK_REALM: realm of the platform users. + KEYCLOAK_ADMIN_USERNAME: master-realm administrator. + KEYCLOAK_ADMIN_PASSWORD: its password. + ACCESS_CACHE_SECONDS: reuse window of a membership answer. + IDLE_STOP: ``true`` stops idle agents. + IDLE_MINUTES: idle time before a stop. + MAX_RUNNING: bound on concurrently running agents. + START_TIMEOUT: seconds an agent gets to become healthy. + REQUEST_TIMEOUT: seconds a proxied request may take. +""" + +from __future__ import annotations + +import http.client +import json +import os +import socket +import sys +import threading +import time +import urllib.parse +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer + +from access import AccessError, Keycloak +from agents import Agents, CapacityError +from engine import ComposeBackend, Engine, EngineError, SwarmBackend + +ENV = os.environ +PORT = int(ENV["BROKER_PORT"]) +BROKER_KEY = ENV["BROKER_KEY"] +LITELLM = urllib.parse.urlsplit(ENV["LITELLM_URL"]) +LITELLM_KEY = ENV["LITELLM_KEY"] +REQUEST_TIMEOUT = float(ENV["REQUEST_TIMEOUT"]) +PLATFORMS = json.loads(ENV["AGENT_PLATFORMS"]) +USER_ID_HEADER = "X-OpenWebUI-User-Id" +USER_EMAIL_HEADER = "X-OpenWebUI-User-Email" +HOP_BY_HOP = {"connection", "transfer-encoding", "keep-alive", "content-length"} + +ENGINE = Engine(ENV["ENGINE_SOCKET"], timeout=120) +BACKEND = ( + SwarmBackend(ENGINE, ENV["AGENT_CONSTRAINT"]) + if ENV["ENGINE_MODE"] == "swarm" + else ComposeBackend(ENGINE, ENV["AGENT_RUNTIME"]) +) +AGENTS = Agents( + backend=BACKEND, + engine=ENGINE, + platforms=PLATFORMS, + self_container=socket.gethostname(), + broker_alias=ENV["BROKER_ALIAS"], + relay_url=f"http://{ENV['BROKER_ALIAS']}:{PORT}/llm/v1", + model=ENV["AGENT_MODEL"], + context=int(ENV["AGENT_CONTEXT"] or 0), + idle_stop=ENV["IDLE_STOP"].lower() == "true", + idle_seconds=int(ENV["IDLE_MINUTES"]) * 60, + max_running=int(ENV["MAX_RUNNING"]), + start_timeout=int(ENV["START_TIMEOUT"]), +) +ACCESS = Keycloak( + ENV["KEYCLOAK_URL"], + ENV["KEYCLOAK_REALM"], + ENV["KEYCLOAK_ADMIN_USERNAME"], + ENV["KEYCLOAK_ADMIN_PASSWORD"], + cache_seconds=int(ENV["ACCESS_CACHE_SECONDS"]), + timeout=15, +) + + +def log(event, **fields): + print(json.dumps({"ts": time.time(), "event": event, **fields}), flush=True) + + +def bearer(headers): + value = headers.get("Authorization") or "" + return value[7:] if value.startswith("Bearer ") else "" + + +class Handler(BaseHTTPRequestHandler): + protocol_version = "HTTP/1.0" + + def log_message(self, _format, *_args): + return + + def _json(self, status, payload): + body = json.dumps(payload).encode() + self.send_response(status) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def _error(self, status, message): + self._json(status, {"error": {"message": message, "type": "agent_broker"}}) + + def _body(self): + length = int(self.headers.get("Content-Length") or 0) + return self.rfile.read(length) if length else b"" + + def _proxy(self, host, port, path, body, headers): + connection = http.client.HTTPConnection(host, port, timeout=REQUEST_TIMEOUT) + try: + connection.request(self.command, path, body=body, headers=headers) + upstream = connection.getresponse() + self.send_response(upstream.status) + for name, value in upstream.getheaders(): + if name.lower() not in HOP_BY_HOP: + self.send_header(name, value) + self.end_headers() + while chunk := upstream.read1(65536): + self.wfile.write(chunk) + self.wfile.flush() + return upstream.status + finally: + connection.close() + + def do_GET(self): + if self.path == "/health": + self._json(200, {"status": "ok"}) + return + if self.path.startswith("/llm/"): + self._relay() + return + if bearer(self.headers) != BROKER_KEY: + self._error(401, "invalid broker key") + return + if self.path == "/v1/models": + self._json( + 200, + { + "object": "list", + "data": [ + {"id": name, "object": "model", "owned_by": "agent-broker"} + for name in sorted(PLATFORMS) + ], + }, + ) + return + self._error(404, "unknown path") + + def do_POST(self): + if self.path.startswith("/llm/"): + self._relay() + return + if bearer(self.headers) != BROKER_KEY: + self._error(401, "invalid broker key") + return + if self.path != "/v1/chat/completions": + self._error(404, "unknown path") + return + self._chat() + + def _chat(self): + owner = (self.headers.get(USER_ID_HEADER) or "").strip() + email = (self.headers.get(USER_EMAIL_HEADER) or "").strip() + if not owner: + self._error(401, f"{USER_ID_HEADER} missing") + return + try: + payload = json.loads(self._body() or b"{}") + except ValueError: + self._error(400, "body is not JSON") + return + platform = payload.get("model") + if platform not in PLATFORMS: + self._error(404, f"unknown agent platform '{platform}'") + return + try: + allowed = ACCESS.allows(email, PLATFORMS[platform]["group"]) + except AccessError as error: + log("access_unavailable", platform=platform, error=str(error)) + self._error(503, "access check unavailable") + return + if not allowed: + log("refused", platform=platform, owner=owner) + AGENTS.stop_owner(platform, owner) + self._error(403, f"not entitled to the {platform} agent") + return + try: + address, key = AGENTS.ensure(platform, owner) + except CapacityError as error: + self._error(503, str(error)) + return + except (EngineError, TimeoutError) as error: + log("agent_failed", platform=platform, owner=owner, error=str(error)) + self._error(502, f"agent could not be started: {error}") + return + payload["model"] = AGENTS.agent_model(platform, address, key) + payload["user"] = owner + target = urllib.parse.urlsplit(address) + log("forward", platform=platform, owner=owner) + name = AGENTS.begin(platform, owner) + try: + self._proxy( + target.hostname, + target.port, + "/v1/chat/completions", + json.dumps(payload).encode(), + {"Content-Type": "application/json", "Authorization": f"Bearer {key}"}, + ) + finally: + AGENTS.end(name) + + def _relay(self): + entry = AGENTS.owner_of_key(bearer(self.headers)) + if entry is None: + self._error(401, "unknown agent key") + return + owner, platform = entry + body = self._body() + if body and self.command == "POST": + try: + payload = json.loads(body) + except ValueError: + payload = None + if isinstance(payload, dict): + payload["user"] = owner + body = json.dumps(payload).encode() + headers = { + "Authorization": f"Bearer {LITELLM_KEY}", + "Content-Type": self.headers.get("Content-Type") or "application/json", + } + path = self.path[len("/llm") :] + status = self._proxy( + LITELLM.hostname, LITELLM.port, path, body or None, headers + ) + log("relay", owner=owner, platform=platform, path=path, status=status) + + +def reaper(): + while True: + time.sleep(60) + try: + AGENTS.reap() + except (EngineError, OSError) as error: + log("reap_failed", error=str(error)) + + +def main(): + threading.Thread(target=reaper, daemon=True).start() + server = ThreadingHTTPServer(("0.0.0.0", PORT), Handler) # noqa: S104 - listens only on the broker's container networks + log("listening", port=PORT, mode=ENV["ENGINE_MODE"]) + try: + server.serve_forever() + except KeyboardInterrupt: + sys.exit(0) + + +if __name__ == "__main__": + main() diff --git a/roles/svc-ai-agent-broker/files/test/.nocheck b/roles/svc-ai-agent-broker/files/test/.nocheck new file mode 100644 index 0000000000..2b9d82d840 --- /dev/null +++ b/roles/svc-ai-agent-broker/files/test/.nocheck @@ -0,0 +1 @@ +# nocheck: role-files-layout - Role test harness with test.sh as its entry point; test-e2e-cli stages files/test/ by that fixed path. diff --git a/roles/svc-ai-agent-broker/files/test/membership.py b/roles/svc-ai-agent-broker/files/test/membership.py new file mode 100644 index 0000000000..85c51eae75 --- /dev/null +++ b/roles/svc-ai-agent-broker/files/test/membership.py @@ -0,0 +1,36 @@ +""" +Environment: + MEMBER: realm username whose membership changes. + METHOD: PUT grants the group, DELETE revokes it. + GROUP_PATH: Keycloak path of the group. + +Runs inside the broker with its own KEYCLOAK_* settings, because curl refuses +to resolve the .onion Keycloak URL of a Tor deployment. +""" + +import os +import sys + +sys.path.insert(0, "/opt/broker") + +from access import Keycloak + +keycloak = Keycloak( + os.environ["KEYCLOAK_URL"], + os.environ["KEYCLOAK_REALM"], + os.environ["KEYCLOAK_ADMIN_USERNAME"], + os.environ["KEYCLOAK_ADMIN_PASSWORD"], + 0, + 60, +) +users = keycloak._get("/users", {"username": os.environ["MEMBER"], "exact": "true"}) +if len(users) != 1: + sys.exit(f"{len(users)} users named {os.environ['MEMBER']}") +group = keycloak._get(f"/group-by-path{os.environ['GROUP_PATH']}", {}) +status, body = keycloak._request( + os.environ["METHOD"], + f"{keycloak.base_url}/admin/realms/{keycloak.realm}/users/{users[0]['id']}/groups/{group['id']}", + headers={"Authorization": f"Bearer {keycloak._admin_token()}"}, +) +if status >= 300 and status != 404: + sys.exit(f"{os.environ['METHOD']} {os.environ['GROUP_PATH']} -> {status} {body}") diff --git a/roles/svc-ai-agent-broker/files/test/probe.py b/roles/svc-ai-agent-broker/files/test/probe.py new file mode 100644 index 0000000000..b0528f65c1 --- /dev/null +++ b/roles/svc-ai-agent-broker/files/test/probe.py @@ -0,0 +1,47 @@ +""" +Environment: + PORT: broker port. + KEY: broker key; empty sends no Authorization header. + OWNER: X-OpenWebUI-User-Id to send. + EMAIL: X-OpenWebUI-User-Email to send. + MODEL: agent platform. + TIMEOUT: seconds the request may take. + +Prints ``STATUS `` and, on 200, ``CONTENT ``. +""" + +import json +import os +import urllib.error +import urllib.request + +request = urllib.request.Request( + f"http://127.0.0.1:{os.environ['PORT']}/v1/chat/completions", + data=json.dumps( + { + "model": os.environ["MODEL"], + "messages": [ + {"role": "user", "content": "Reply with exactly the word: pong"} + ], + "stream": False, + } + ).encode(), + method="POST", +) +request.add_header("Content-Type", "application/json") +if os.environ["KEY"]: + request.add_header("Authorization", f"Bearer {os.environ['KEY']}") +request.add_header("X-OpenWebUI-User-Id", os.environ["OWNER"]) +request.add_header("X-OpenWebUI-User-Email", os.environ["EMAIL"]) +try: + with urllib.request.urlopen( # noqa: S310 - fixed http://127.0.0.1 target + request, timeout=int(os.environ["TIMEOUT"]) + ) as response: + body = json.loads(response.read()) + print(f"STATUS {response.status}") + content = (body.get("choices") or [{}])[0].get("message", {}).get( + "content" + ) or "" + print(f"CONTENT {content.strip()[:80]}") +except urllib.error.HTTPError as error: + print(f"STATUS {error.code}") diff --git a/roles/svc-ai-agent-broker/files/test/proxy.py b/roles/svc-ai-agent-broker/files/test/proxy.py new file mode 100644 index 0000000000..06f3ce7b1c --- /dev/null +++ b/roles/svc-ai-agent-broker/files/test/proxy.py @@ -0,0 +1,52 @@ +""" +Environment: + ENGINE_SOCKET: filtered engine socket the socket proxy serves (broker env). + SELF_NAME: the broker's own container, used as the exec and delete target. + +Prints `` `` per call, or `` allowed`` when the proxy +passed the call through. The image cannot be pulled, so a create the proxy +fails to refuse still leaves no container behind. +""" + +import os +import re +import sys + +sys.path.insert(0, "/opt/broker") + +import engine + +IMAGE = "agent-broker-proxy-probe:absent" +BIND = [{"Type": "bind", "Source": "/", "Target": "/host"}] +PROBES = { + "exec": ("POST", "/containers/{self}/exec", {"Cmd": ["true"]}), + "binds": ( + "POST", + "/containers/create", + {"Image": IMAGE, "HostConfig": {"Binds": ["/:/host"]}}, + ), + "mounts": ( + "POST", + "/containers/create", + {"Image": IMAGE, "HostConfig": {"Mounts": BIND}}, + ), + "service_mounts": ( + "POST", + "/services/create", + { + "Name": "agent-broker-proxy-probe", + "TaskTemplate": {"ContainerSpec": {"Image": IMAGE, "Mounts": BIND}}, + }, + ), + "delete": ("DELETE", "/containers/{self}", None), +} + +api = engine.Engine(os.environ["ENGINE_SOCKET"], timeout=30) +for name, (method, path, body) in PROBES.items(): + target = path.format(self=os.environ["SELF_NAME"]) + try: + api.call(method, target, body=body, expect=()) + print(f"{name} allowed", flush=True) + except engine.EngineError as error: + status = re.search(r"-> (\d{3})", str(error)) + print(f"{name} {status.group(1) if status else error}"[:200], flush=True) diff --git a/roles/svc-ai-agent-broker/files/test/test.sh b/roles/svc-ai-agent-broker/files/test/test.sh new file mode 100644 index 0000000000..fe8920ecab --- /dev/null +++ b/roles/svc-ai-agent-broker/files/test/test.sh @@ -0,0 +1,236 @@ +#!/usr/bin/env bash +# shellcheck shell=bash +# +# Env (rendered into test.env from templates/test.env.j2): +# AGENT_BROKER_CONTAINER resolved broker container (CLI_LOCAL_CID) +# AGENT_BROKER_PORT broker port inside the container +# AGENT_BROKER_KEY_B64 base64 broker key +# AGENT_RUNTIME runtime every agent must run under +# AGENT_DATA_DIR state mount of a hermes agent +# AGENT_PORT API port of a hermes agent +# AGENT_GROUP_PATH Keycloak path of the hermes agent-user group +# AGENT_KEY_ENV env var an agent carries its bearer key in +# AGENT_MODEL model alias the broker configures agents with +# AGENT_OPENCLAW_GROUP_PATH Keycloak path of the openclaw agent-user group +# AGENT_PLATFORMS_OFFERED comma-separated platforms the broker serves +# DEPLOYMENT_MODE compose or swarm +# IDLE_STOP, IDLE_MINUTES lifecycle settings the broker must run with +# MAX_RUNNING bound on concurrently running agents +# USER_A_NAME, USER_A_EMAIL first entitled user +# USER_B_NAME, USER_B_EMAIL second user, refused before being entitled +# REQUEST_TIMEOUT seconds a probe waits for its agent + +set -uo pipefail + +here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +BROKER_KEY="$(printf '%s' "${AGENT_BROKER_KEY_B64}" | base64 -d)" +OWNER_A="cli-isolation-a" +OWNER_B="cli-isolation-b" +FAILED=0 + +for required in hermes openclaw; do + case ",${AGENT_PLATFORMS_OFFERED}," in + *",${required},"*) ;; + *) + echo "[FAIL] ${required} is not offered by this deploy, which serves '${AGENT_PLATFORMS_OFFERED}'." >&2 + echo " The broker withholds a platform whose context minimum '${AGENT_MODEL}' misses," >&2 + echo " so point services.agent-broker.agents.model at a model that satisfies it." >&2 + exit 1 + ;; + esac +done + +fail() { + echo "[FAIL] $*" >&2 + FAILED=1 +} + +agent_name() { + printf 'agent-%s-%s' "$1" "$(printf '%s' "$2" | sha256sum | cut -c1-16)" +} + +kc_membership() { + # nocheck: container-exec-resolver address resolved by the caller and passed in + container exec -i \ + -e "MEMBER=$1" \ + -e "METHOD=$2" \ + -e "GROUP_PATH=$3" \ + "${AGENT_BROKER_CONTAINER}" python3 - <"${here}/membership.py" +} + +probe() { + # nocheck: container-exec-resolver address resolved by the caller and passed in + container exec -i \ + -e "PORT=${AGENT_BROKER_PORT}" \ + -e "KEY=$1" \ + -e "OWNER=$2" \ + -e "EMAIL=$3" \ + -e "MODEL=$4" \ + -e "TIMEOUT=${REQUEST_TIMEOUT}" \ + "${AGENT_BROKER_CONTAINER}" python3 - <"${here}/probe.py" +} + +status_of() { + printf '%s\n' "$1" | sed -n 's/^STATUS //p' +} + +proxy_probe() { + # nocheck: container-exec-resolver address resolved by the caller and passed in + container exec -i -e "SELF_NAME=${AGENT_BROKER_CONTAINER}" \ + "${AGENT_BROKER_CONTAINER}" python3 - <"${here}/proxy.py" +} + +EMPTY_KEY_DIGEST="$(printf '' | sha256sum)" + +agent_key_digest() { + if [ "${DEPLOYMENT_MODE}" = "swarm" ]; then + container docker service inspect -f '{{range .Spec.TaskTemplate.ContainerSpec.Env}}{{println .}}{{end}}' "$1" | + sed -n "s/^${AGENT_KEY_ENV}=//p" | sha256sum + else + container docker inspect --type container -f '{{range .Config.Env}}{{println .}}{{end}}' "$1" | + sed -n "s/^${AGENT_KEY_ENV}=//p" | sha256sum + fi +} + +started_at() { + container docker inspect --type container -f '{{.State.StartedAt}}' "$1" +} + +broker_env() { + container docker inspect --type container -f '{{range .Config.Env}}{{println .}}{{end}}' "${AGENT_BROKER_CONTAINER}" | + sed -n "s/^$1=//p" +} + +relayed() { + container docker logs --since "$1" "${AGENT_BROKER_CONTAINER}" 2>&1 | + grep -qF "\"event\": \"relay\", \"owner\": \"$2\", \"platform\": \"$3\", \"path\": \"/v1/chat/completions\", \"status\": 200" +} + +# shellcheck disable=SC2329,SC2317 # reached through the EXIT trap below, which shellcheck does not follow. +cleanup() { + kc_membership "${USER_A_NAME}" DELETE "${AGENT_GROUP_PATH}" + kc_membership "${USER_B_NAME}" DELETE "${AGENT_GROUP_PATH}" + kc_membership "${USER_A_NAME}" DELETE "${AGENT_OPENCLAW_GROUP_PATH}" + local name + for name in "${NAME_A}" "${NAME_B}" "${NAME_CLAW}"; do + if [ "${DEPLOYMENT_MODE}" = "swarm" ]; then + container docker service rm "${name}" >/dev/null 2>&1 + else + container docker rm -f "${name}" >/dev/null 2>&1 + fi + container docker network disconnect -f "${name}" "${AGENT_BROKER_CONTAINER}" >/dev/null 2>&1 + container docker network rm "${name}" >/dev/null 2>&1 + container docker volume rm "${name}" >/dev/null 2>&1 + done +} + +NAME_A="$(agent_name hermes "${OWNER_A}")" +NAME_B="$(agent_name hermes "${OWNER_B}")" +NAME_CLAW="$(agent_name openclaw "${OWNER_A}")" +trap cleanup EXIT + +output="$(probe "" "${OWNER_A}" "${USER_A_EMAIL}" hermes)" +[ "$(status_of "${output}")" = "401" ] || fail "a request without the broker key answered $(status_of "${output}"), not 401" + +kc_membership "${USER_B_NAME}" DELETE "${AGENT_GROUP_PATH}" +output="$(probe "${BROKER_KEY}" "${OWNER_B}" "${USER_B_EMAIL}" hermes)" +[ "$(status_of "${output}")" = "403" ] || fail "${USER_B_NAME} outside the group answered $(status_of "${output}"), not 403" +if container docker inspect --type container "${NAME_B}" >/dev/null 2>&1 || + container docker service inspect "${NAME_B}" >/dev/null 2>&1; then + fail "the refused request still created ${NAME_B}" +fi + +proxy_output="$(proxy_probe)" +echo "${proxy_output}" +for expected in "exec 403" "binds 403" "mounts 403" "service_mounts 403" "delete 405"; do + printf '%s\n' "${proxy_output}" | grep -qx "${expected}" || + fail "the socket proxy answered '$(printf '%s\n' "${proxy_output}" | grep "^${expected%% *} ")', expected '${expected}'" +done + +for setting in IDLE_STOP:"${IDLE_STOP}" IDLE_MINUTES:"${IDLE_MINUTES}" MAX_RUNNING:"${MAX_RUNNING}" AGENT_MODEL:"${AGENT_MODEL}"; do + [ "$(broker_env "${setting%%:*}")" = "${setting#*:}" ] || + fail "the broker runs with ${setting%%:*}=$(broker_env "${setting%%:*}"), not the inventory's ${setting#*:}" +done + +kc_membership "${USER_A_NAME}" PUT "${AGENT_GROUP_PATH}" || fail "could not grant ${USER_A_NAME} ${AGENT_GROUP_PATH}" +kc_membership "${USER_B_NAME}" PUT "${AGENT_GROUP_PATH}" || fail "could not grant ${USER_B_NAME} ${AGENT_GROUP_PATH}" +kc_membership "${USER_A_NAME}" PUT "${AGENT_OPENCLAW_GROUP_PATH}" || fail "could not grant ${USER_A_NAME} ${AGENT_OPENCLAW_GROUP_PATH}" +sleep 65 + +since="$(date +%s)" +output="$(probe "${BROKER_KEY}" "${OWNER_A}" "${USER_A_EMAIL}" hermes)" +echo "${output}" +[ "$(status_of "${output}")" = "200" ] || fail "${USER_A_NAME}'s agent answered $(status_of "${output}"), not 200" +relayed "${since}" "${OWNER_A}" hermes || fail "${USER_A_NAME}'s hermes agent never got a model answer through the relay" +output="$(probe "${BROKER_KEY}" "${OWNER_B}" "${USER_B_EMAIL}" hermes)" +echo "${output}" +[ "$(status_of "${output}")" = "200" ] || fail "${USER_B_NAME}'s agent answered $(status_of "${output}"), not 200" +relayed "${since}" "${OWNER_B}" hermes || fail "${USER_B_NAME}'s hermes agent never got a model answer through the relay" +output="$(probe "${BROKER_KEY}" "${OWNER_A}" "${USER_A_EMAIL}" openclaw)" +echo "${output}" +[ "$(status_of "${output}")" = "200" ] || fail "${USER_A_NAME}'s openclaw agent answered $(status_of "${output}"), not 200" +relayed "${since}" "${OWNER_A}" openclaw || fail "${USER_A_NAME}'s openclaw agent never got a model answer through the relay" + +for name in "${NAME_A}" "${NAME_B}"; do + [ "$(agent_key_digest "${name}")" != "${EMPTY_KEY_DIGEST}" ] || + fail "${name} carries no ${AGENT_KEY_ENV}, so comparing the two keys proves nothing" +done + +[ "$(agent_key_digest "${NAME_A}")" != "$(agent_key_digest "${NAME_B}")" ] || + fail "both owners' agents carry the same ${AGENT_KEY_ENV}" + +if [ "${DEPLOYMENT_MODE}" = "swarm" ]; then + container docker service inspect "${NAME_A}" "${NAME_B}" >/dev/null || fail "the two agent services are missing" + tasks_before="$(container docker service ps -q "${NAME_A}")" + output="$(probe "${BROKER_KEY}" "${OWNER_A}" "${USER_A_EMAIL}" hermes)" + [ "$(status_of "${output}")" = "200" ] || fail "${USER_A_NAME}'s second prompt answered $(status_of "${output}"), not 200" + [ "$(container docker service ps -q "${NAME_A}")" = "${tasks_before}" ] || + fail "the second prompt replaced the task of ${NAME_A} instead of reusing it" + exit "${FAILED}" +fi + +for name in "${NAME_A}" "${NAME_B}"; do + runtime="$(container docker inspect --type container -f '{{.HostConfig.Runtime}}' "${name}")" + [ "${runtime}" = "${AGENT_RUNTIME}" ] || fail "${name} runs under '${runtime}', not '${AGENT_RUNTIME}'" +done + +[ "$(container docker inspect --type container -f '{{.Id}}' "${NAME_A}")" != "$(container docker inspect --type container -f '{{.Id}}' "${NAME_B}")" ] || + fail "both owners share one container" +volume_a="$(container docker inspect --type container -f '{{range .Mounts}}{{.Name}}{{end}}' "${NAME_A}")" +volume_b="$(container docker inspect --type container -f '{{range .Mounts}}{{.Name}}{{end}}' "${NAME_B}")" +if [ -z "${volume_a}" ] || [ "${volume_a}" = "${volume_b}" ]; then + fail "the owners share a volume (${volume_a} / ${volume_b})" +fi + +# nocheck: container-exec-resolver agent container named by the broker +container exec "${NAME_A}" sh -c "echo isolated > '${AGENT_DATA_DIR}/cli-isolation-marker'" || fail "could not write the marker in ${NAME_A}" +# nocheck: container-exec-resolver agent container named by the broker +if container exec "${NAME_B}" test -e "${AGENT_DATA_DIR}/cli-isolation-marker"; then + fail "${NAME_B} sees the file ${NAME_A} wrote" +fi + +ip_b="$(container docker inspect --type container -f "{{(index .NetworkSettings.Networks \"${NAME_B}\").IPAddress}}" "${NAME_B}")" +[ -n "${ip_b}" ] || fail "${NAME_B} has no address on its own network" +# nocheck: container-exec-resolver agent container named by the broker +if container exec "${NAME_A}" python3 -c "import socket; socket.create_connection(('${ip_b}', ${AGENT_PORT}), 3)" 2>/dev/null; then + fail "${NAME_A} reached ${NAME_B} at ${ip_b}:${AGENT_PORT}" +fi + +started_before="$(started_at "${NAME_A}")" +id_before="$(container docker inspect --type container -f '{{.Id}}' "${NAME_A}")" +output="$(probe "${BROKER_KEY}" "${OWNER_A}" "${USER_A_EMAIL}" hermes)" +[ "$(status_of "${output}")" = "200" ] || fail "${USER_A_NAME}'s second prompt answered $(status_of "${output}"), not 200" +[ "$(started_at "${NAME_A}")" = "${started_before}" ] || + fail "the second prompt restarted ${NAME_A} instead of reusing the running agent" + +container docker stop "${NAME_A}" >/dev/null || fail "could not stop ${NAME_A} for the restart check" +output="$(probe "${BROKER_KEY}" "${OWNER_A}" "${USER_A_EMAIL}" hermes)" +[ "$(status_of "${output}")" = "200" ] || fail "the prompt after a stop answered $(status_of "${output}"), not 200" +[ "$(container docker inspect --type container -f '{{.Id}}' "${NAME_A}")" = "${id_before}" ] || + fail "the stopped agent was replaced instead of started again" +# nocheck: container-exec-resolver agent container named by the broker +container exec "${NAME_A}" test -e "${AGENT_DATA_DIR}/cli-isolation-marker" || + fail "${NAME_A} lost its state across the stop" + +[ "${FAILED}" -eq 0 ] && echo "[OK] two owners got two isolated agents under ${AGENT_RUNTIME}" +exit "${FAILED}" diff --git a/roles/svc-ai-agent-broker/meta/info.yml b/roles/svc-ai-agent-broker/meta/info.yml new file mode 100644 index 0000000000..1f00d37182 --- /dev/null +++ b/roles/svc-ai-agent-broker/meta/info.yml @@ -0,0 +1,4 @@ +--- +# nocheck: info-media no official demo video +logo: + class: fa-solid fa-people-arrows diff --git a/roles/svc-ai-agent-broker/meta/main.yml b/roles/svc-ai-agent-broker/meta/main.yml new file mode 100644 index 0000000000..7f4553c50f --- /dev/null +++ b/roles/svc-ai-agent-broker/meta/main.yml @@ -0,0 +1,34 @@ +--- +galaxy_info: + author: Kevin Veen-Birkenbach + description: Per-user Hermes and OpenClaw agents started on demand behind an OpenAI-compatible endpoint + license: Infinito.Nexus Community License (Non-Commercial) + company: | + Kevin Veen-Birkenbach + https://www.veen.world + galaxy_tags: + - ai + - agent + - broker + - hermes + - openclaw + - self-hosted + issue_tracker_url: https://s.infinito.nexus/issues + min_ansible_version: "2.9" + platforms: + - name: ArchLinux + versions: + - all + - name: Debian + versions: + - all + - name: EL + versions: + - all + - name: Fedora + versions: + - all + - name: Ubuntu + versions: + - all +dependencies: [] diff --git a/roles/svc-ai-agent-broker/meta/networks.yml b/roles/svc-ai-agent-broker/meta/networks.yml new file mode 100644 index 0000000000..f0eb839753 --- /dev/null +++ b/roles/svc-ai-agent-broker/meta/networks.yml @@ -0,0 +1,6 @@ +--- +local: + subnet: 192.168.223.0/24 +overlay: + modes: [compose, swarm] + topology: shared_net diff --git a/roles/svc-ai-agent-broker/meta/secrets.yml b/roles/svc-ai-agent-broker/meta/secrets.yml new file mode 100644 index 0000000000..b5b2f3a11e --- /dev/null +++ b/roles/svc-ai-agent-broker/meta/secrets.yml @@ -0,0 +1,12 @@ +--- +credentials: + api_key: + description: Bearer key Open WebUI presents to the agent broker + algorithm: alphanumeric + validation: + min_length: 32 + litellm_api_key: + description: The broker's own virtual key for the shared svc-ai-litellm gateway, used for every agent call it relays + algorithm: sk_prefixed + validation: + min_length: 50 diff --git a/roles/svc-ai-agent-broker/meta/services.yml b/roles/svc-ai-agent-broker/meta/services.yml new file mode 100644 index 0000000000..53a5bd6c5b --- /dev/null +++ b/roles/svc-ai-agent-broker/meta/services.yml @@ -0,0 +1,78 @@ +--- +agent-broker: + modes: + compose: + enabled: true + swarm: + enabled: true + # nocheck: default-placement-manager joins each agent's attachable overlay through the manager engine the socket proxy fronts + placement: manager + bond: 1 + enabled: true + shared: true + image: python + # nocheck: docker-version base of a self-built stdlib-only image, not the served application + version: "3.13.15-slim" + custom: true + ca_client: true + name: agent-broker + min_storage: 130MB + cpus: "0.5" + mem_reservation: 64m + mem_limit: 256m + pids_limit: 256 + healthcheck: + flavor: http_status + path: health + ports: + internal: + http: 8080 + request_timeout: 1800 + agents: + model: "{{ AI_AGENT_MODEL }}" + idle_stop: true + idle_minutes: 30 + max_running: 20 + start_timeout: 900 + access_cache_seconds: 60 + run_after: + - svc-ai-litellm + guide_companions: + - svc-ai-ollama + lifecycle: beta + +# nocheck: shared +# nocheck: dynamic-flag +socket-proxy: + bond: 1 + enabled: true + image: wollomatic/socket-proxy + version: 1.13.1 + name: agent-socket-proxy + cpus: "0.25" + mem_reservation: 16m + mem_limit: 64m + pids_limit: 128 + healthcheck: + test: ["CMD", "./healthcheck"] +# nocheck: playwright-service-flag, dynamic-flag every agent call is relayed through the gateway, so the broker never runs without it +litellm: + bond: 1 + enabled: true + shared: true +# nocheck: playwright-service-flag, dynamic-flag the broker authorizes every request against Keycloak group paths and has no LDAP or no-auth path, so a group_names gate would only defer the failure to runtime +sso: + bond: 1 + enabled: true + shared: true + flavor: oidc +# nocheck: playwright-service-flag, dynamic-flag +hermes: + bond: 1 + enabled: true + shared: true +# nocheck: playwright-service-flag, dynamic-flag +openclaw: + bond: 1 + enabled: true + shared: true diff --git a/roles/svc-ai-agent-broker/meta/tests.yml b/roles/svc-ai-agent-broker/meta/tests.yml new file mode 100644 index 0000000000..db2e49b27c --- /dev/null +++ b/roles/svc-ai-agent-broker/meta/tests.yml @@ -0,0 +1,4 @@ +--- +cli: + timeout: 5400 + container_service_key: agent-broker diff --git a/roles/svc-ai-agent-broker/meta/variants.yml b/roles/svc-ai-agent-broker/meta/variants.yml new file mode 100644 index 0000000000..b55d587a02 --- /dev/null +++ b/roles/svc-ai-agent-broker/meta/variants.yml @@ -0,0 +1,5 @@ +--- +# Matrix-deploy variant overrides for this role. +# See docs/contributing/artefact/files/role/variants.md for the file format. + +- services: {} diff --git a/roles/svc-ai-agent-broker/meta/volumes.yml b/roles/svc-ai-agent-broker/meta/volumes.yml new file mode 100644 index 0000000000..4a90144551 --- /dev/null +++ b/roles/svc-ai-agent-broker/meta/volumes.yml @@ -0,0 +1,17 @@ +# nocheck: backup-service-flag the only named volume carries the socket proxy's transient unix socket, nothing to restore +--- +docker_sock: + type: bind + source: /var/run/docker.sock + read_only: true + mounts: + - service: socket-proxy + target: /var/run/docker.sock + +engine_socket: + type: volume + mounts: + - service: socket-proxy + target: &engine_socket_dir /run/agent-engine + - service: agent-broker + target: *engine_socket_dir diff --git a/roles/svc-ai-agent-broker/tasks/00_core.yml b/roles/svc-ai-agent-broker/tasks/00_core.yml new file mode 100644 index 0000000000..a6eb3e2e94 --- /dev/null +++ b/roles/svc-ai-agent-broker/tasks/00_core.yml @@ -0,0 +1,64 @@ +--- +- include_tasks: utils/once/flag.yml + +- include_tasks: utils/load/handlers.yml + vars: + handler_role_name: "sys-svc-compose" + +- name: "🔐 Verify the gateway contract for {{ application_id }}" + ansible.builtin.include_tasks: "{{ lookup('path_absolute', 'roles/svc-ai-litellm/tasks/utils/consumer_contract.yml') }}" + +- name: "🛑 Refuse an agent model the gateway never resolved" + ansible.builtin.assert: + that: + - AGENT_BROKER_MODEL | length > 0 + - AGENT_BROKER_MODEL is not search('[{][{]') + fail_msg: "services.agent-broker.agents.model resolved to '{{ AGENT_BROKER_MODEL }}'; every agent would ask the gateway for it. Deploy a backend role (svc-ai-ollama, + svc-ai-lmstudio) alongside the broker, or configure a provider key on svc-ai-litellm." + quiet: true + +- name: "⚠️ Report a platform this model cannot serve" + ansible.builtin.debug: + msg: "{{ item }} is not offered: it refuses any model below {{ AGENT_BROKER_PLATFORMS_DECLARED[item].min_context }} tokens and '{{ AGENT_BROKER_MODEL }}' declares + {{ AGENT_BROKER_CONTEXT }}, so every prompt would come back HTTP 500. Point services.agent-broker.agents.model at a larger model to offer it again." + loop: "{{ AGENT_BROKER_PLATFORMS_REFUSED }}" + +- name: "⚠️ Report a context minimum this deploy cannot prove" + ansible.builtin.debug: + msg: "{{ item.key }} requires {{ item.value.min_context }} tokens and '{{ AGENT_BROKER_MODEL }}' declares no context, so nothing here can tell whether the model + is large enough. Declare the model's context to make this checkable." + when: (AGENT_BROKER_CONTEXT | string | length) == 0 + loop: "{{ AGENT_BROKER_PLATFORMS_DECLARED | dict2items | selectattr('value.min_context', 'defined') | list }}" + loop_control: + label: "{{ item.key }}" + +- name: "🛡️ Install the isolating runtime every agent runs under" + ansible.builtin.include_role: + name: svc-virt-kata + +- name: "📦 Stage the agent broker build context" + ansible.builtin.include_tasks: + file: "{{ lookup('path_absolute', 'roles/sys-svc-compose/tasks/utils/render_replicated_templates.yml') }}" + vars: + role_templates: + - src: "{{ lookup('path_absolute', 'roles/svc-ai-agent-broker/files/python/access.py') }}" + dest: "{{ [AGENT_BROKER_BUILD_DIR, 'access.py'] | path_join }}" + mode: "0644" + notify: [compose-build, compose-up] + - src: "{{ lookup('path_absolute', 'roles/svc-ai-agent-broker/files/python/agents.py') }}" + dest: "{{ [AGENT_BROKER_BUILD_DIR, 'agents.py'] | path_join }}" + mode: "0644" + notify: [compose-build, compose-up] + - src: "{{ lookup('path_absolute', 'roles/svc-ai-agent-broker/files/python/engine.py') }}" + dest: "{{ [AGENT_BROKER_BUILD_DIR, 'engine.py'] | path_join }}" + mode: "0644" + notify: [compose-build, compose-up] + - src: "{{ lookup('path_absolute', 'roles/svc-ai-agent-broker/files/python/server.py') }}" + dest: "{{ [AGENT_BROKER_BUILD_DIR, 'server.py'] | path_join }}" + mode: "0644" + notify: [compose-build, compose-up] + +- name: "🔧 Setup container network for {{ application_id }}" + include_tasks: "{{ lookup('path_absolute', 'roles/sys-svc-compose/tasks/utils/network/routine.yml') }}" + vars: + compose_handlers_flush: true diff --git a/roles/svc-ai-agent-broker/tasks/main.yml b/roles/svc-ai-agent-broker/tasks/main.yml new file mode 100644 index 0000000000..92df6f8ca9 --- /dev/null +++ b/roles/svc-ai-agent-broker/tasks/main.yml @@ -0,0 +1,5 @@ +--- +- include_tasks: 00_core.yml + when: run_once_svc_ai_agent_broker is not defined + vars: + application_id: svc-ai-agent-broker diff --git a/roles/svc-ai-agent-broker/templates/compose.yml.j2 b/roles/svc-ai-agent-broker/templates/compose.yml.j2 new file mode 100644 index 0000000000..e4bf86c2a6 --- /dev/null +++ b/roles/svc-ai-agent-broker/templates/compose.yml.j2 @@ -0,0 +1,54 @@ +{% include 'roles/sys-svc-compose/templates/base.yml.j2' %} + + {% set service_name = 'agent-broker' %} + {{ service_name }}: + {% include 'roles/sys-svc-container/templates/base.yml.j2' %} + {{ lookup('container_image', application_id, 'agent-broker') }} + build: + context: {{ AGENT_BROKER_BUILD_DIR }} + dockerfile: Dockerfile + args: + AGENT_BROKER_BASE_IMAGE: {{ AGENT_BROKER_BASE_IMAGE }} + AGENT_BROKER_BASE_VERSION: "{{ AGENT_BROKER_BASE_VERSION }}" + {{ lookup('compose_only', 'container_name', AGENT_BROKER_CONTAINER) }} + read_only: true + cap_drop: + - ALL + security_opt: + - no-new-privileges:true + expose: + - "{{ AGENT_BROKER_PORT }}" + {{ lookup('depends_on', {'socket-proxy': 'service_healthy'}) }} + {{ lookup('container_volumes', service_name) | indent(4) }} + {{ lookup('container_healthcheck', service_name) | indent(4) }} + {{ lookup('container_networks') | indent(4) }} + + {% set service_name = 'socket-proxy' %} + {{ service_name }}: + {% include 'roles/sys-svc-container/templates/base.yml.j2' %} + {{ lookup('container_image', application_id, 'socket-proxy') }} + {{ lookup('compose_only', 'container_name', AGENT_SOCKET_PROXY_NAME) }} + {{ lookup('compose_only', 'network_mode', 'none') }} + user: "0:0" + read_only: true + cap_drop: + - ALL + security_opt: + - no-new-privileges:true + command: + - "-loglevel=info" + - "-proxysocketendpoint={{ AGENT_ENGINE_SOCKET }}" + - "-proxysocketendpointfilemode=0666" + - "-allowhealthcheck" + - "-allowbindmountfrom=/nonexistent" + - "-allowGET=^(/v[0-9.]+)?/(_ping|version|containers/json|containers/[^/]+/json|images/.+/json|networks|networks/[^/]+|services|services/[^/]+|tasks)$$" + - "-allowPOST=^(/v[0-9.]+)?/(images/create|volumes/create|networks/create|networks/[^/]+/connect|containers/create|containers/[^/]+/(start|stop)|services/create|services/[^/]+/update)$$" + - "-watchdoginterval=3600" + - "-stoponwatchdog" + - "-shutdowngracetime=5" + {{ lookup('container_volumes', service_name) | indent(4) }} + {{ lookup('container_healthcheck', service_name) | indent(4) }} + +{{ lookup('compose_networks') }} + +{{ lookup('compose_volumes') }} diff --git a/roles/svc-ai-agent-broker/templates/env.j2 b/roles/svc-ai-agent-broker/templates/env.j2 new file mode 100644 index 0000000000..d92f181d77 --- /dev/null +++ b/roles/svc-ai-agent-broker/templates/env.j2 @@ -0,0 +1,24 @@ +{% include 'roles/sys-svc-container/templates/env.j2' %} + +BROKER_PORT={{ AGENT_BROKER_PORT }} +BROKER_KEY={{ AGENT_BROKER_KEY | dotenv_quote }} +BROKER_ALIAS={{ entity_name }} +ENGINE_SOCKET={{ AGENT_ENGINE_SOCKET }} +ENGINE_MODE={{ DEPLOYMENT_MODE }} +AGENT_RUNTIME={{ AGENT_BROKER_RUNTIME }} +AGENT_CONSTRAINT={{ AGENT_BROKER_CONSTRAINT | dotenv_quote }} +LITELLM_URL={{ LITELLM_BASE_LOCAL_URL }} +LITELLM_KEY={{ AGENT_BROKER_LITELLM_KEY | dotenv_quote }} +AGENT_CONTEXT={{ AGENT_BROKER_CONTEXT }} +AGENT_MODEL={{ AGENT_BROKER_MODEL | dotenv_quote }} +AGENT_PLATFORMS={{ AGENT_BROKER_PLATFORMS | to_json | dotenv_quote }} +KEYCLOAK_URL={{ lookup('tls', 'web-app-keycloak', 'url.base') }} +KEYCLOAK_REALM={{ OIDC.CLIENT.REALM | dotenv_quote }} +KEYCLOAK_ADMIN_USERNAME={{ lookup('users', 'administrator').username | dotenv_quote }} +KEYCLOAK_ADMIN_PASSWORD={{ lookup('users', 'administrator').password | dotenv_quote }} +ACCESS_CACHE_SECONDS={{ AGENT_BROKER_AGENTS.access_cache_seconds }} +IDLE_STOP={{ AGENT_BROKER_AGENTS.idle_stop | bool | string | lower }} +IDLE_MINUTES={{ AGENT_BROKER_AGENTS.idle_minutes }} +MAX_RUNNING={{ AGENT_BROKER_AGENTS.max_running }} +START_TIMEOUT={{ AGENT_BROKER_AGENTS.start_timeout }} +REQUEST_TIMEOUT={{ AGENT_BROKER_REQUEST_TIMEOUT }} diff --git a/roles/svc-ai-agent-broker/templates/test.env.j2 b/roles/svc-ai-agent-broker/templates/test.env.j2 new file mode 100644 index 0000000000..04d9edc979 --- /dev/null +++ b/roles/svc-ai-agent-broker/templates/test.env.j2 @@ -0,0 +1,20 @@ +AGENT_BROKER_CONTAINER={{ CLI_LOCAL_CID }} +AGENT_BROKER_KEY_B64={{ AGENT_BROKER_KEY | b64encode }} +AGENT_BROKER_PORT={{ AGENT_BROKER_PORT }} +AGENT_DATA_DIR={{ AGENT_BROKER_PLATFORMS_DECLARED.hermes.data }} +AGENT_GROUP_PATH={{ AGENT_BROKER_PLATFORMS_DECLARED.hermes.group }} +AGENT_KEY_ENV={{ AGENT_BROKER_PLATFORMS_DECLARED.hermes.key_env }} +AGENT_MODEL={{ AGENT_BROKER_MODEL }} +AGENT_OPENCLAW_GROUP_PATH={{ AGENT_BROKER_PLATFORMS_DECLARED.openclaw.group }} +AGENT_PLATFORMS_OFFERED={{ AGENT_BROKER_PLATFORMS.keys() | sort | join(',') | dotenv_quote }} +AGENT_PORT={{ AGENT_BROKER_PLATFORMS_DECLARED.hermes.port }} +AGENT_RUNTIME={{ AGENT_BROKER_RUNTIME }} +DEPLOYMENT_MODE={{ DEPLOYMENT_MODE }} +IDLE_MINUTES={{ AGENT_BROKER_AGENTS.idle_minutes }} +IDLE_STOP={{ AGENT_BROKER_AGENTS.idle_stop | bool | string | lower }} +MAX_RUNNING={{ AGENT_BROKER_AGENTS.max_running }} +REQUEST_TIMEOUT={{ AGENT_BROKER_REQUEST_TIMEOUT + 300 }} +USER_A_EMAIL={{ lookup('users', 'administrator').email }} +USER_A_NAME={{ lookup('users', 'administrator').username }} +USER_B_EMAIL={{ lookup('users', 'biber').email }} +USER_B_NAME={{ lookup('users', 'biber').username }} diff --git a/roles/svc-ai-agent-broker/vars/main.yml b/roles/svc-ai-agent-broker/vars/main.yml new file mode 100644 index 0000000000..eab5f3f588 --- /dev/null +++ b/roles/svc-ai-agent-broker/vars/main.yml @@ -0,0 +1,79 @@ +--- +application_id: "svc-ai-agent-broker" +entity_name: "{{ application_id | get_entity_name }}" + +AGENT_BROKER_CONTAINER: "{{ lookup('config', application_id, 'services.agent-broker.name') }}" +AGENT_BROKER_PORT: "{{ lookup('config', application_id, 'services.agent-broker.ports.internal.http') }}" +AGENT_BROKER_KEY: "{{ lookup('config', application_id, 'secrets.credentials.api_key') }}" +AGENT_BROKER_LITELLM_KEY: "{{ lookup('config', application_id, 'secrets.credentials.litellm_api_key') }}" +AGENT_BROKER_BASE_IMAGE: "{{ lookup('config', application_id, 'services.agent-broker.image') }}" +AGENT_BROKER_BASE_VERSION: "{{ lookup('config', application_id, 'services.agent-broker.version') }}" +AGENT_BROKER_BUILD_DIR: "{{ lookup('container', application_id, 'directories.instance') }}" +AGENT_BROKER_REQUEST_TIMEOUT: "{{ lookup('config', application_id, 'services.agent-broker.request_timeout') }}" +AGENT_BROKER_AGENTS: "{{ lookup('config', application_id, 'services.agent-broker.agents') }}" +AGENT_BROKER_MODEL: "{{ AGENT_BROKER_AGENTS.model }}" +AGENT_BROKER_CONTEXT: "{{ AI_MODEL_CONTEXTS[AGENT_BROKER_MODEL] | default('', true) }}" +AGENT_BROKER_RUNTIME: "{{ SANDBOX_RUNTIME }}" +AGENT_BROKER_CONSTRAINT: "node.labels.{{ SANDBOX_NODE_LABEL }} == true" +AGENT_SOCKET_PROXY_NAME: "{{ lookup('config', application_id, 'services.socket-proxy.name') }}" +AGENT_ENGINE_SOCKET: "{{ [lookup('volume', application_id, 'engine_socket').target, 'engine.sock'] | path_join }}" + +AGENT_BROKER_PLATFORMS_DECLARED: + hermes: + group: "/{{ lookup('rbac_group_path', application_id='web-app-hermes', role='agent-user') }}" + image: "{{ lookup('image', 'web-app-hermes', 'hermes') }}" + port: "{{ lookup('config', 'web-app-hermes', 'services.hermes.ports.internal.http') }}" + health: /health + data: "{{ lookup('volume', 'web-app-hermes', 'hermes_data').target }}" + config_path: "{{ lookup('volume', 'web-app-hermes', 'hermes_config').target }}" + command: ["gateway"] + user: "" + key_env: API_SERVER_KEY + request_model: hermes-agent + min_context: "{{ lookup('config', 'web-app-hermes', 'services.hermes.min_context') }}" + env: + HERMES_SKIP_CONFIG_MIGRATION: "true" + API_SERVER_ENABLED: "true" + API_SERVER_HOST: "0.0.0.0" + API_SERVER_PORT: "{{ lookup('config', 'web-app-hermes', 'services.hermes.ports.internal.http') }}" + TZ: "{{ HOST_TIMEZONE }}" + cpus: "{{ lookup('config', 'web-app-hermes', 'services.hermes.cpus') }}" + mem_limit: "{{ lookup('config', 'web-app-hermes', 'services.hermes.mem_limit') | upper | human_to_bytes }}" + mem_reservation: "{{ lookup('config', 'web-app-hermes', 'services.hermes.mem_reservation') | upper | human_to_bytes }}" + pids_limit: "{{ lookup('config', 'web-app-hermes', 'services.hermes.pids_limit') }}" + openclaw: + group: "/{{ lookup('rbac_group_path', application_id='web-app-openclaw', role='agent-user') }}" + image: "{{ lookup('image', 'web-app-openclaw', 'openclaw') }}" + port: "{{ lookup('config', 'web-app-openclaw', 'services.openclaw.ports.internal.http') }}" + health: /healthz + data: "{{ lookup('volume', 'web-app-openclaw', 'openclaw_data').target }}" + config_path: "{{ lookup('volume', 'web-app-openclaw', 'openclaw_config').target }}" + command: + - node + - dist/index.js + - gateway + - run + - --bind + - lan + - --port + - "{{ lookup('config', 'web-app-openclaw', 'services.openclaw.ports.internal.http') }}" + user: "0:0" + key_env: OPENCLAW_GATEWAY_TOKEN + request_model: openclaw + env: + HOME: /home/node + OPENCLAW_HOME: /home/node + OPENCLAW_CONFIG_DIR: "{{ lookup('volume', 'web-app-openclaw', 'openclaw_data').target }}" + OPENCLAW_STATE_DIR: "{{ lookup('volume', 'web-app-openclaw', 'openclaw_data').target }}" + OPENCLAW_CONFIG_PATH: "{{ lookup('volume', 'web-app-openclaw', 'openclaw_config').target }}" + OPENCLAW_WORKSPACE_DIR: "{{ [lookup('volume', 'web-app-openclaw', 'openclaw_data').target, 'workspace'] | path_join }}" + TZ: "{{ HOST_TIMEZONE }}" + cpus: "{{ lookup('config', 'web-app-openclaw', 'services.openclaw.cpus') }}" + mem_limit: "{{ lookup('config', 'web-app-openclaw', 'services.openclaw.mem_limit') | upper | human_to_bytes }}" + mem_reservation: "{{ lookup('config', 'web-app-openclaw', 'services.openclaw.mem_reservation') | upper | human_to_bytes }}" + pids_limit: "{{ lookup('config', 'web-app-openclaw', 'services.openclaw.pids_limit') }}" + +AGENT_BROKER_PLATFORMS_REFUSED: "{{ (AGENT_BROKER_PLATFORMS_DECLARED | dict2items | selectattr('value.min_context', 'defined') | selectattr('value.min_context', 'gt', + AGENT_BROKER_CONTEXT | int) | map(attribute='key') | list) if (AGENT_BROKER_CONTEXT | string | length) > 0 else [] }}" + +AGENT_BROKER_PLATFORMS: "{{ AGENT_BROKER_PLATFORMS_DECLARED | dict2items | rejectattr('key', 'in', AGENT_BROKER_PLATFORMS_REFUSED) | items2dict }}" diff --git a/roles/svc-ai-jupyter/README.md b/roles/svc-ai-jupyter/README.md new file mode 100644 index 0000000000..f3e9098988 --- /dev/null +++ b/roles/svc-ai-jupyter/README.md @@ -0,0 +1,75 @@ +# Jupyter + +## Description + +[Jupyter](https://jupyter.org/) runs code in a managed kernel and returns its output. This role provides that kernel as a backend for other services rather than as a notebook UI for people. + +## Overview + +The server starts with a token from the deployment's credential store, binds to the container network and publishes no port. Cross-site request checking is off and any origin is accepted, because the caller is a service on the same network that presents the token rather than a browser carrying a cookie. + +The role declares no volume: a kernel that executes code a chat wrote gets a fresh filesystem on every restart. A consumer reaches it as `http://jupyter:8888` once it declares `jupyter` in its own `meta/services.yml` with `enabled` and `shared` set. `web-app-openwebui` ships that flag and sends `CODE_EXECUTION_ENGINE=jupyter` with it. + +## Cosmos + +The diagram places Jupyter in the Infinito.Nexus cosmos: the components it deploys (capabilities), the central services it consumes (dependencies), and its outward reach (federation and bridged external networks). + +```mermaid +flowchart LR + subgraph role [svc-ai-jupyter 🐳🐝] + svc_jupyter["jupyter"] + end +``` + +Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. + +## Features + +- **Token protected:** every request carries a token from the credential store; the token is never handed to a browser user. +- **Ephemeral:** no volume is mounted, so each restart discards whatever the executed code wrote. +- **Internal only:** no published port and no proxy entry, so the kernel is reachable from the container network alone. + +## Quick Setup + +### Development + +Clone, set up the workstation, and deploy Jupyter onto the local stack: + +```bash +git clone https://github.com/infinito-nexus/core.git +cd core +make onboard +make compose-deploy mode=reinstall apps=svc-ai-jupyter full_cycle=false +``` + +### Production + +Run the published image to provision the inventory and deploy Jupyter to a managed server (the mounted volume persists the inventory): + +```bash +APP=svc-ai-jupyter +HOST= +DOMAIN= +TLS_MODE=self_signed +SSH_PUBLIC_KEY="" + +docker run --rm -it \ + -v "$PWD/inventories:/etc/infinito.nexus/inventories" \ + -e APP="$APP" -e HOST="$HOST" -e DOMAIN="$DOMAIN" -e TLS_MODE="$TLS_MODE" -e SSH_PUBLIC_KEY="$SSH_PUBLIC_KEY" \ + ghcr.io/infinito-nexus/core/debian bash -c ' + INVENTORY=/etc/infinito.nexus/inventories/production + infinito administration inventory provision "$INVENTORY" \ + --inventory-file "$INVENTORY/devices.yml" \ + --host "$HOST" \ + --include "$APP" \ + --vars "{\"TLS_MODE\": \"$TLS_MODE\", \"DOMAIN_PRIMARY\": \"$DOMAIN\", \"users\": {\"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}" && + infinito administration deploy dedicated "$INVENTORY/devices.yml" \ + --password-file "$INVENTORY/.password" \ + --diff -vv' +``` + +## Credits + +Implemented by **[Kevin Veen-Birkenbach](https://www.veen.world)**. +Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world). +Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license). diff --git a/roles/svc-ai-jupyter/meta/info.yml b/roles/svc-ai-jupyter/meta/info.yml new file mode 100644 index 0000000000..2260b2503f --- /dev/null +++ b/roles/svc-ai-jupyter/meta/info.yml @@ -0,0 +1,5 @@ +--- +# nocheck: info-media no official demo video +logo: + class: fas fa-square-root-variable +homepage: https://jupyter.org/ diff --git a/roles/svc-ai-jupyter/meta/main.yml b/roles/svc-ai-jupyter/meta/main.yml new file mode 100644 index 0000000000..fa7a2b210c --- /dev/null +++ b/roles/svc-ai-jupyter/meta/main.yml @@ -0,0 +1,32 @@ +--- +galaxy_info: + author: Kevin Veen-Birkenbach + description: Kernel backend that runs the code a chat writes, in a container instead of in the browser sandbox + license: Infinito.Nexus Community License (Non-Commercial) + company: | + Kevin Veen-Birkenbach + https://www.veen.world + galaxy_tags: + - ai + - jupyter + - python + - sandbox + issue_tracker_url: https://s.infinito.nexus/issues + min_ansible_version: "2.9" + platforms: + - name: ArchLinux + versions: + - all + - name: Debian + versions: + - all + - name: EL + versions: + - all + - name: Fedora + versions: + - all + - name: Ubuntu + versions: + - all +dependencies: [] diff --git a/roles/svc-ai-jupyter/meta/networks.yml b/roles/svc-ai-jupyter/meta/networks.yml new file mode 100644 index 0000000000..6845cd6181 --- /dev/null +++ b/roles/svc-ai-jupyter/meta/networks.yml @@ -0,0 +1,6 @@ +--- +local: + subnet: 192.168.226.0/24 +overlay: + modes: [compose, swarm] + topology: shared_net diff --git a/roles/svc-ai-jupyter/meta/secrets.yml b/roles/svc-ai-jupyter/meta/secrets.yml new file mode 100644 index 0000000000..189a47798e --- /dev/null +++ b/roles/svc-ai-jupyter/meta/secrets.yml @@ -0,0 +1,6 @@ +--- +credentials: + auth_token: + description: Token a client presents to the kernel gateway. Anything holding it can execute code inside the sandbox, so it is not shared with browser users. + algorithm: random_hex_32 + validation: non_empty_string diff --git a/roles/svc-ai-jupyter/meta/services.yml b/roles/svc-ai-jupyter/meta/services.yml new file mode 100644 index 0000000000..6af5c925fe --- /dev/null +++ b/roles/svc-ai-jupyter/meta/services.yml @@ -0,0 +1,26 @@ +--- +# nocheck: playwright-service-flag - sandboxed kernel backend reached over the container network +jupyter: + modes: + compose: + enabled: true + swarm: + enabled: true + bond: 1 + enabled: true + shared: true + image: quay.io/jupyter/minimal-notebook + version: "2026-09-21" + name: jupyter + healthcheck: + flavor: http_status + path: api + min_storage: 1GB + cpus: "1.0" + mem_reservation: 256m + mem_limit: 1g + pids_limit: 512 + ports: + internal: + http: 8888 + lifecycle: alpha diff --git a/roles/svc-ai-jupyter/meta/variants.yml b/roles/svc-ai-jupyter/meta/variants.yml new file mode 100644 index 0000000000..25cf154307 --- /dev/null +++ b/roles/svc-ai-jupyter/meta/variants.yml @@ -0,0 +1,2 @@ +--- +- services: {} diff --git a/roles/svc-ai-jupyter/tasks/00_core.yml b/roles/svc-ai-jupyter/tasks/00_core.yml new file mode 100644 index 0000000000..bd183d0e32 --- /dev/null +++ b/roles/svc-ai-jupyter/tasks/00_core.yml @@ -0,0 +1,7 @@ +--- +- include_tasks: utils/once/flag.yml + +- name: "🔧 Setup container network for {{ application_id }}" + include_tasks: "{{ lookup('path_absolute', 'roles/sys-svc-compose/tasks/utils/network/routine.yml') }}" + vars: + compose_handlers_flush: true diff --git a/roles/svc-ai-jupyter/tasks/main.yml b/roles/svc-ai-jupyter/tasks/main.yml new file mode 100644 index 0000000000..ae16240953 --- /dev/null +++ b/roles/svc-ai-jupyter/tasks/main.yml @@ -0,0 +1,5 @@ +--- +- include_tasks: 00_core.yml + when: run_once_svc_ai_jupyter is not defined + vars: + application_id: svc-ai-jupyter diff --git a/roles/svc-ai-jupyter/templates/compose.yml.j2 b/roles/svc-ai-jupyter/templates/compose.yml.j2 new file mode 100644 index 0000000000..873567b16b --- /dev/null +++ b/roles/svc-ai-jupyter/templates/compose.yml.j2 @@ -0,0 +1,22 @@ +{% include 'roles/sys-svc-compose/templates/base.yml.j2' %} + + {% set service_name = 'jupyter' %} + {{ service_name }}: + {{ lookup('container_healthcheck', service_name) | indent(4) }} + {% include 'roles/sys-svc-container/templates/base.yml.j2' %} + {{ lookup('container_image', application_id, 'jupyter') }} + {{ lookup('compose_only', 'container_name', JUPYTER_CONTAINER) }} + expose: + - "{{ JUPYTER_PORT }}" + command: > + start-notebook.py + --IdentityProvider.token={{ JUPYTER_AUTH_TOKEN }} + --ServerApp.ip=0.0.0.0 + --ServerApp.port={{ JUPYTER_PORT }} + --ServerApp.allow_origin=* + --ServerApp.disable_check_xsrf=True + {{ lookup('container_networks') | indent(4) }} + +{{ lookup('compose_networks') }} + +{{ lookup('compose_volumes') }} diff --git a/roles/svc-ai-jupyter/vars/main.yml b/roles/svc-ai-jupyter/vars/main.yml new file mode 100644 index 0000000000..ff917a5876 --- /dev/null +++ b/roles/svc-ai-jupyter/vars/main.yml @@ -0,0 +1,7 @@ +--- +application_id: "svc-ai-jupyter" +entity_name: "{{ application_id | get_entity_name }}" + +JUPYTER_CONTAINER: "{{ lookup('config', application_id, 'services.jupyter.name') }}" +JUPYTER_PORT: "{{ lookup('config', application_id, 'services.jupyter.ports.internal.http') }}" +JUPYTER_AUTH_TOKEN: "{{ lookup('config', application_id, 'secrets.credentials.auth_token') }}" diff --git a/roles/svc-ai-litellm/README.md b/roles/svc-ai-litellm/README.md index a975a2d0b9..fc40d67d9b 100644 --- a/roles/svc-ai-litellm/README.md +++ b/roles/svc-ai-litellm/README.md @@ -17,15 +17,21 @@ flowchart LR subgraph deps [Dependencies] dep_svc_ai_lmstudio["svc-ai-lmstudio 🐳🐝"] dep_svc_ai_ollama["svc-ai-ollama 🐳🐝"] + dep_svc_ai_s1["svc-ai-s1 🐳🐝"] dep_svc_db_postgres["svc-db-postgres 🐳🐝"] end subgraph role [svc-ai-litellm 🐳🐝] svc_litellm["litellm"] svc_postgres["postgres"] + svc_s1["s1"] svc_ollama["ollama"] svc_lmstudio["lmstudio"] end subgraph dependents [Dependents] + dpt_dsk_gnt_claude["dsk-gnt-claude 💻"] + dpt_dsk_gnt_codex["dsk-gnt-codex 💻"] + dpt_dsk_gnt_pi["dsk-gnt-pi 💻"] + dpt_svc_ai_agent_broker["svc-ai-agent-broker 🐳🐝"] dpt_web_app_discourse["web-app-discourse 🐳🐝"] dpt_web_app_flowise["web-app-flowise 🐳🐝"] dpt_web_app_hermes["web-app-hermes 🐳🐝"] @@ -34,16 +40,17 @@ flowchart LR dpt_web_app_mattermost["web-app-mattermost 🐳🐝"] dpt_web_app_mediawiki["web-app-mediawiki 🐳🐝"] dpt_web_app_moodle["web-app-moodle 🐳🐝"] - dpt_web_app_n8n["web-app-n8n 🐳🐝"] - dpt_web_app_nextcloud["web-app-nextcloud 🐳🐝"] - dpt_web_app_openclaw["web-app-openclaw 🐳🐝"] - dpt_web_app_openwebui["web-app-openwebui 🐳🐝"] dpt_more["..."] end dep_svc_ai_lmstudio -. "0..1" .-> svc_lmstudio dep_svc_ai_ollama -. "0..1" .-> svc_ollama + dep_svc_ai_s1 -. "0..1" .-> svc_s1 dep_svc_db_postgres -. "0..1" .-> svc_postgres + svc_litellm -. "0..1" .-> dpt_dsk_gnt_claude + svc_litellm -. "0..1" .-> dpt_dsk_gnt_codex + svc_litellm -. "0..1" .-> dpt_dsk_gnt_pi svc_litellm -- "1:1" --> dpt_more + svc_litellm -- "1:1" --> dpt_svc_ai_agent_broker svc_litellm -. "0..1" .-> dpt_web_app_discourse svc_litellm -. "0..1" .-> dpt_web_app_flowise svc_litellm -. "0..1" .-> dpt_web_app_hermes @@ -52,10 +59,6 @@ flowchart LR svc_litellm -. "0..1" .-> dpt_web_app_mattermost svc_litellm -. "0..1" .-> dpt_web_app_mediawiki svc_litellm -. "0..1" .-> dpt_web_app_moodle - svc_litellm -. "0..1" .-> dpt_web_app_n8n - svc_litellm -. "0..1" .-> dpt_web_app_nextcloud - svc_litellm -. "0..1" .-> dpt_web_app_openclaw - svc_litellm -. "0..1" .-> dpt_web_app_openwebui ``` Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. @@ -65,6 +68,8 @@ Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional ( - **OpenAI-compatible API:** One HTTP endpoint serves every model listed in the gateway configuration. - **Backend routing:** Model entries are generated for Ollama and LM Studio when those services run on the host, under the shared alias each model carries so a consumer names one model whichever local backend answers it. - **Remote providers:** OpenAI, Anthropic and OpenRouter each add their models when a key is configured. The keys are declared in `meta/secrets.yml` with the `type` and `regex` their value must satisfy, and default to the central `API..api_key` entry; an unset key leaves that provider's routes unpublished. +- **Configurable remote models:** The published remote models are the `litellm.remote_models` list in `meta/services.yml`, each entry an `alias`, the LiteLLM `model` string and its `provider`, plus an optional `context` in tokens. An inventory replaces the list through `applications.svc-ai-litellm.services.litellm.remote_models`, and a provider without a key slot aborts the deploy. +- **Declared context windows:** A model entry carrying `context` publishes it as `model_info.max_input_tokens`, and an Ollama entry also runs with that window as `num_ctx` instead of the server's default, so a caller's prompt is not silently truncated. - **Per-consumer virtual keys:** Each consuming application receives its own virtual key, created through the gateway admin API under an alias naming that application. - **File-based model list:** The model list is mounted as a read-only config file, with database-stored model entries turned off. - **Managed credentials:** The gateway master key and the admin UI password are generated and kept as role credentials, and the admin UI username is the platform administrator name. diff --git a/roles/svc-ai-litellm/files/python/ensure_keys.py b/roles/svc-ai-litellm/files/python/ensure_keys.py index e39e335680..5f7976f086 100644 --- a/roles/svc-ai-litellm/files/python/ensure_keys.py +++ b/roles/svc-ai-litellm/files/python/ensure_keys.py @@ -1,6 +1,7 @@ import contextlib import json import os +import sys import urllib.error import urllib.request @@ -54,11 +55,11 @@ def main(): call = http_call( "http://localhost:" + os.environ["LITELLM_PORT"], { - "Authorization": "Bearer " + os.environ["LITELLM_MK"], + "Authorization": "Bearer " + os.environ["LITELLM_MASTER_KEY"], "Content-Type": "application/json", }, ) - for entry in json.loads(os.environ["LITELLM_KEYS_PAYLOAD"]): + for entry in json.load(sys.stdin): if ensure_key(entry, call): print("CHANGED " + entry["alias"]) diff --git a/roles/svc-ai-litellm/files/python/measure_speed.py b/roles/svc-ai-litellm/files/python/measure_speed.py new file mode 100644 index 0000000000..9d9952e2a9 --- /dev/null +++ b/roles/svc-ai-litellm/files/python/measure_speed.py @@ -0,0 +1,164 @@ +"""Measure how fast each served model generates, so the router can rank them. + +The router scores a route on locality, price and speed. The first two are +declared or catalogued; speed has no source at all, because it is a property of +this machine under this load and not of the model. So it is measured here, +against the gateway that will serve the traffic. + +Only a model without a stored rate is measured. A deploy that re-measured +everything would send requests whose results the tolerance below discards +anyway, and that traffic is the whole live surface of this script. + +The result is a per-model median of output tokens per second, printed as JSON +for the deploy to store and render back into ``model_info.traits.speed``. + +Environment: + LITELLM_PORT: port the gateway listens on inside its own container. + LITELLM_SAMPLES: requests per model; above one, the first is discarded. + LITELLM_TIMEOUT: seconds one measured request may take. + LITELLM_WARMUP: seconds the discarded first request may take. + LITELLM_EXCLUDE: comma-separated aliases to leave unranked, namely the + router's own and every mock. A mock generates nothing, + so its rate is the proxy's echo latency rather than a + model's, and ranking on it sends every request to a + fixture that answers the same string to anything. + LITELLM_REMEASURE: non-empty to re-measure models that already have a rate. + LITELLM_PREVIOUS: the JSON of the last measurement. +""" + +from __future__ import annotations + +import json +import os +import statistics +import sys +import time +import urllib.error +import urllib.request + +MASTER_KEY = os.environ["LITELLM_MASTER_KEY"] +PORT = os.environ["LITELLM_PORT"] +SAMPLES = int(os.environ["LITELLM_SAMPLES"]) +TIMEOUT = float(os.environ["LITELLM_TIMEOUT"]) +WARMUP_TIMEOUT = float(os.environ["LITELLM_WARMUP"]) +EXCLUDE = { + alias.strip() + for alias in (os.environ.get("LITELLM_EXCLUDE") or "").split(",") + if alias.strip() +} +REMEASURE = bool(os.environ.get("LITELLM_REMEASURE")) +PREVIOUS = json.loads(os.environ.get("LITELLM_PREVIOUS") or "{}") + +LISTING_TIMEOUT = 30 +MAX_TOKENS = 64 +PROMPT = "Count from one to twenty in words." +TOLERANCE = 0.10 + + +def call(path, payload=None, timeout=LISTING_TIMEOUT): + """One gateway call, decoded. + + Args: + path: the API path below the gateway root. + payload: a mapping to POST, or None to GET. + timeout: seconds the call may take. + + Returns: + The decoded response body. + """ + request = urllib.request.Request( + f"http://127.0.0.1:{PORT}{path}", + data=json.dumps(payload).encode() if payload else None, + headers={ + "Authorization": f"Bearer {MASTER_KEY}", + "Content-Type": "application/json", + }, + method="POST" if payload else "GET", + ) + with urllib.request.urlopen(request, timeout=timeout) as response: # noqa: S310 fixed internal http origin + return json.loads(response.read()) + + +def sample(model, timeout): + """Output tokens per second for one request, or None if it yielded none.""" + started = time.monotonic() + body = call( + "/v1/chat/completions", + { + "model": model, + "messages": [{"role": "user", "content": PROMPT}], + "max_tokens": MAX_TOKENS, + }, + timeout, + ) + elapsed = time.monotonic() - started + tokens = (body.get("usage") or {}).get("completion_tokens") or 0 + if not tokens or elapsed <= 0: + return None + return tokens / elapsed + + +def measure(model): + """The median rate over ``LITELLM_SAMPLES`` calls, after one warm-up call. + + A backend that has never answered this model pays its load cost on the + first call, which is not the rate a later caller will see, and that call + gets the longer budget for the same reason. The median then absorbs + whatever else the machine was doing during one sample. + + The warm-up is setup, not a sample, so it is discarded at every setting. + Counting it as one would make a one-sample run report load time, which + ranks a small model below a larger one and routes on the inversion. + """ + sample(model, WARMUP_TIMEOUT) + rates = [ + rate + for rate in (sample(model, TIMEOUT) for _ in range(SAMPLES)) + if rate is not None + ] + return statistics.median(rates) if rates else None + + +def settled(model, measured): + """The measurement, or the stored one when the move is within tolerance.""" + stored = PREVIOUS.get(model) + if stored and abs(measured - stored) <= TOLERANCE * stored: + return stored + return round(measured, 2) + + +def rank(model): + """This model's rate, measuring only when it has none to fall back on.""" + if model in PREVIOUS and not REMEASURE: + print(f"KEPT {model}: {PREVIOUS[model]} tok/s", file=sys.stderr) + return PREVIOUS[model] + try: + measured = measure(model) + except (urllib.error.URLError, OSError, ValueError) as error: + measured, why = None, type(error).__name__ + else: + why = "no request produced output tokens" + if measured is None: + print(f"SKIPPED {model}: {why}", file=sys.stderr) + return PREVIOUS.get(model) + rate = settled(model, measured) + print(f"MEASURED {model}: {rate} tok/s", file=sys.stderr) + return rate + + +def main(): + served = [ + entry["id"] + for entry in call("/v1/models").get("data") or [] + if entry.get("id") and entry["id"] not in EXCLUDE + ] + speeds = {} + for model in served: + rate = rank(model) + if rate is not None: + speeds[model] = rate + print(json.dumps(speeds, indent=2, sort_keys=True)) + + +if __name__ == "__main__": + main() diff --git a/roles/svc-ai-litellm/files/python/probe_auth.py b/roles/svc-ai-litellm/files/python/probe_auth.py index 2ac9cdf471..1451711ee1 100644 --- a/roles/svc-ai-litellm/files/python/probe_auth.py +++ b/roles/svc-ai-litellm/files/python/probe_auth.py @@ -11,9 +11,12 @@ consumer key still working afterwards, so it proves isolation without touching any credential a consumer depends on. Environment: - LITELLM_MK: master key the gateway accepts. + LITELLM_MASTER_KEY: master key the gateway accepts, read from its own + environment rather than passed in. LITELLM_PORT: port the gateway listens on inside its own container. - LITELLM_KEY: a live consumer virtual key that must survive the revocation. + stdin: JSON ``{"key": ""}``, a live key that + must survive the revocation. It arrives on stdin because an -e argument + would put it in the container exec command line. """ from __future__ import annotations @@ -21,6 +24,7 @@ from __future__ import annotations import contextlib import json import os +import sys import time import urllib.error import urllib.request @@ -152,8 +156,8 @@ def main(): base = "http://127.0.0.1:" + os.environ["LITELLM_PORT"] probe( lambda key: status_of(base, key), - admin_call(base, os.environ["LITELLM_MK"]), - os.environ["LITELLM_KEY"], + admin_call(base, os.environ["LITELLM_MASTER_KEY"]), + json.load(sys.stdin)["key"], ) print("AUTHENTICATED") diff --git a/roles/svc-ai-litellm/files/python/warm_model.py b/roles/svc-ai-litellm/files/python/warm_model.py index ca1925916a..f093b7e727 100644 --- a/roles/svc-ai-litellm/files/python/warm_model.py +++ b/roles/svc-ai-litellm/files/python/warm_model.py @@ -6,7 +6,8 @@ timeout, so the first consumer request comes back as a 504 instead of an answer while every later one succeeds. Warming here moves the cost into the deploy. Environment: - LITELLM_MK: master key the gateway accepts. + LITELLM_MASTER_KEY: master key the gateway accepts, read from its own + environment rather than passed in. LITELLM_PORT: port the gateway listens on inside its own container. LITELLM_MODEL: model name to warm. """ @@ -17,7 +18,7 @@ import json import os import urllib.request -MASTER_KEY = os.environ["LITELLM_MK"] +MASTER_KEY = os.environ["LITELLM_MASTER_KEY"] PORT = os.environ["LITELLM_PORT"] MODEL = os.environ["LITELLM_MODEL"] TIMEOUT = 900 diff --git a/roles/svc-ai-litellm/files/test/probe.py b/roles/svc-ai-litellm/files/test/probe.py index f6ad07ce32..a9ab4b26df 100644 --- a/roles/svc-ai-litellm/files/test/probe.py +++ b/roles/svc-ai-litellm/files/test/probe.py @@ -6,17 +6,22 @@ the litellm app, not a test toolchain. Env (rendered into test.env from templates/test.env.j2): PORT gateway http port inside the container - MASTER_KEY gateway master key + LITELLM_MASTER_KEY gateway master key, read from the gateway's own + environment rather than passed in CHAT_MODEL the model every consumer asks for (LITELLM_CHAT_MODEL) CHAT_MODEL_SERVED true|false, whether a backend can answer it EXPECTED_MODELS JSON list the config template published LMSTUDIO_ALIASES JSON list of aliases only LM Studio provides, rendered whether or not the backend is deployed so the absent case stays assertable + MOCK_ALIASES JSON list of aliases answered from a canned string, whose + reported prompt token count is litellm's own constant REMOTE_ALIASES JSON list of aliases a configured provider key publishes (OpenAI, Anthropic, OpenRouter) OLLAMA_ENABLED true|false LMSTUDIO_ENABLED true|false + MIN_CHARS_PER_TOKEN the hook's own characters-per-token floor, so the probe + sizes its prompt against the number the hook will use RETRIES completion attempts (default 10) SLEEP_SECONDS wait between attempts (default 6) """ @@ -41,6 +46,7 @@ def evaluate( lmstudio_enabled: bool, lmstudio_aliases: list[str], remote_aliases: list[str], + router_alias: str = "", ) -> list[str]: """Every routing violation the published model list shows, as messages. @@ -53,12 +59,24 @@ def evaluate( lmstudio_enabled: svc-ai-lmstudio is deployed on the gateway's host. lmstudio_aliases: aliases only LM Studio provides, whatever is deployed. remote_aliases: aliases a configured remote provider key publishes. + router_alias: the alias the pre-call hook rewrites. It is backed by no + backend of its own, so it is accounted for separately and then held + out of the backend arithmetic. Returns: One message per violation; empty when the routing is consistent. """ failures: list[str] = [] + if router_alias: + if served and router_alias not in served: + failures.append( + f"the config declares the router alias '{router_alias}' but the " + f"gateway serves {sorted(served)}; consumers asking for it get a 400" + ) + served = served - {router_alias} + expected = [alias for alias in expected if alias != router_alias] + undelivered = sorted(set(expected) - served) if undelivered: failures.append( @@ -118,7 +136,21 @@ def _call(base: str, key: str, path: str, payload: dict | None = None): return json.loads(response.read().decode() or "{}") -def _completion(base: str, key: str, model: str, retries: int, pause: float) -> str: +def _completion( + base: str, + key: str, + model: str, + retries: int, + pause: float, + prompt: str = "Reply with the word pong.", +) -> tuple[str, str, int]: + """The answer, the model that produced it, and the tokens it counted. + + The second element is what makes a router observable: the request names an + alias, the response names whatever the pre-call hook rewrote it to. The + third is the backend's own prompt token count, the only number here that + does not come from this deployment's own declarations. + """ last_error = "no attempt ran" for attempt in range(1, retries + 1): try: @@ -128,17 +160,22 @@ def _completion(base: str, key: str, model: str, retries: int, pause: float) -> "/v1/chat/completions", { "model": model, - "messages": [ - {"role": "user", "content": "Reply with the word pong."} - ], + "messages": [{"role": "user", "content": prompt}], "max_tokens": 16, }, ) choices = completion.get("choices") or [{}] answer = (choices[0].get("message", {}).get("content") or "").strip() if answer: - return answer + usage = completion.get("usage") or {} + return ( + answer, + str(completion.get("model") or ""), + int(usage.get("prompt_tokens") or 0), + ) last_error = "empty content" + except urllib.error.HTTPError as exc: + last_error = f"HTTP {exc.code}: {exc.read().decode(errors='replace')[:400]}" except (urllib.error.URLError, TimeoutError, ValueError) as exc: last_error = f"{type(exc).__name__}: {exc}" if attempt < retries: @@ -146,8 +183,142 @@ def _completion(base: str, key: str, model: str, retries: int, pause: float) -> raise RuntimeError(last_error) +TRUNCATION_RATIO = 0.9 + + +def _declared_window(windows: dict, served: str): + """The window declared for *served*, whichever name the response carries. + + litellm sets ``model_response.model`` to the resolved model rather than the + alias, so an answer can come back as ``ollama/qwen2.5:0.5b`` while the + windows are keyed on ``qwen2.5:0.5b``. Looking up only the literal name + would find nothing and pass every answer, which is worse than no check. + """ + for key in (served, served.split("/", 1)[-1]): + if key in windows: + return windows[key] + return None + + +def route_verdict( + router_alias: str, + windows: dict, + served: str, + needed_tokens: int, + counted_tokens: int = 0, + sent_tokens: int = 0, + mocks: tuple = (), +) -> str: + """Why this answer does not prove the router worked, or the empty string. + + Args: + router_alias: the alias the request named. + windows: alias -> declared context window, for the routes that declare one. + served: the model the response named. + needed_tokens: what the prompt demands, as the hook would estimate it. + counted_tokens: what the backend evaluated, from ``usage.prompt_tokens``. + Every other number here is this deployment's own declaration, so + this is the only one that can contradict them. + sent_tokens: what the prompt really costs. Exact rather than estimated, + because ``"x "`` is one token on every tokenizer served here, and a + backend that reports fewer truncated the prompt instead of refusing + it. A declared window above the model's trained one is answered + that way, so comparing against the window would miss it. + mocks: the aliases that answer from a canned string. They report a + handful of prompt tokens whatever they are sent, because they never + read the prompt at all, so the truncation test cannot tell them + from a backend that dropped it. + + Returns: + One message, empty when the answer is a legitimate routing outcome. + """ + if not served: + return f"'{router_alias}' answered without naming a model, so nothing shows which route ran" + if served == router_alias: + return ( + f"'{router_alias}' answered as itself; the pre-call hook did not rewrite " + f"the model, so the request took the alias's own fallback route" + ) + window = _declared_window(windows, served) + if window is not None and int(window) < needed_tokens: + return ( + f"'{router_alias}' routed a prompt of about {needed_tokens} tokens to " + f"'{served}', whose declared window is {window}; the eligibility filter " + f"did not exclude it" + ) + if served in mocks: + return "" + if counted_tokens and counted_tokens < sent_tokens * TRUNCATION_RATIO: + return ( + f"'{served}' evaluated {counted_tokens} prompt tokens of the " + f"{sent_tokens} sent; the backend silently dropped the rest rather " + f"than refusing, so the answer is not the answer to this prompt" + ) + return "" + + +def oversized_prompt(smallest_window: int, chars_per_token: int) -> tuple[str, int]: + """A prompt no route with *smallest_window* can hold, and its token demand. + + Args: + smallest_window: the narrowest window any published route declares. + chars_per_token: the hook's own floor, passed in rather than repeated, + so the demand this returns is the one the hook will compute. + + Returns: + The prompt and the tokens it demands. ``"x "`` costs one token per two + characters on every tokenizer here, so the floor is exact for it. + """ + characters = smallest_window * 4 + return "x " * (characters // 2), characters // chars_per_token + + +def _probe_router( + base: str, + key: str, + router_alias: str, + windows: dict, + retries: int, + pause: float, + mocks: tuple = (), +) -> list[str]: + """Whether the alias routes, or only answers. + + The alias falls back to the first published route, which is the first + preloaded model and therefore the one with the smallest window. A prompt + that route cannot hold separates a routed request from a fallback: without + the hook the answer comes back from a model whose window is too small. + """ + failures: list[str] = [] + smallest = min(windows.values()) + prompt, needed = oversized_prompt(smallest, int(os.environ["MIN_CHARS_PER_TOKEN"])) + try: + answer, served, counted = _completion( + base, key, router_alias, retries, pause, prompt=prompt + ) + except RuntimeError as exc: + return [ + ( + f"'{router_alias}' answered nothing for a prompt of about {needed} " + f"tokens after {retries} attempts; last failure: {exc}" + ) + ] + verdict = route_verdict( + router_alias, windows, served, needed, counted, len(prompt) // 2, mocks + ) + if verdict: + failures.append(verdict) + else: + print( + f"[OK] '{router_alias}' routed about {needed} tokens to '{served}' " + f"(window {_declared_window(windows, served) or 'undeclared'}, " + f"{counted} counted), answered: {answer[:40]}" + ) + return failures + + def main() -> int: - key = os.environ["MASTER_KEY"] + key = os.environ["LITELLM_MASTER_KEY"] base = f"http://127.0.0.1:{os.environ['PORT']}" chat_model = os.environ["CHAT_MODEL"].strip() chat_model_served = os.environ["CHAT_MODEL_SERVED"] == "true" @@ -167,11 +338,12 @@ def main() -> int: lmstudio_enabled=os.environ["LMSTUDIO_ENABLED"] == "true", lmstudio_aliases=json.loads(os.environ["LMSTUDIO_ALIASES"]), remote_aliases=json.loads(os.environ["REMOTE_ALIASES"]), + router_alias=os.environ.get("ROUTER_ALIAS", "").strip(), ) if chat_model_served and not failures: try: - answer = _completion(base, key, chat_model, retries, pause) + answer, _, _ = _completion(base, key, chat_model, retries, pause) print(f"[OK] '{chat_model}' answered: {answer[:60]}") except RuntimeError as exc: failures.append( @@ -179,6 +351,20 @@ def main() -> int: f"{retries} attempts; last failure: {exc}" ) + router_alias = os.environ.get("ROUTER_ALIAS", "").strip() + windows = { + alias: int(window) + for alias, window in json.loads( + os.environ.get("ROUTER_WINDOWS", "{}") or "{}" + ).items() + if alias in served + } + mocks = tuple(json.loads(os.environ["MOCK_ALIASES"])) + if router_alias and len(windows) > 1 and not failures: + failures.extend( + _probe_router(base, key, router_alias, windows, retries, pause, mocks) + ) + for failure in failures: print(f"[FAIL] {failure}", file=sys.stderr) if not failures: diff --git a/roles/svc-ai-litellm/files/test/test.sh b/roles/svc-ai-litellm/files/test/test.sh index f8e8a895f1..613dfa4b44 100644 --- a/roles/svc-ai-litellm/files/test/test.sh +++ b/roles/svc-ai-litellm/files/test/test.sh @@ -8,11 +8,12 @@ # Env (rendered into test.env from templates/test.env.j2): # LITELLM_CONTAINER resolved gateway container (CLI_LOCAL_CID) # LITELLM_PORT gateway http port inside the container -# LITELLM_MASTER_KEY_B64 base64 master key (avoids shell-quoting issues) +# LITELLM_ROUTER_MIN_CHARS_PER_TOKEN the hook's characters-per-token floor # LITELLM_CHAT_MODEL the model every consumer asks for # LITELLM_CHAT_MODEL_SERVED true|false # LITELLM_EXPECTED_MODELS JSON list the config template published # LITELLM_LMSTUDIO_ALIASES JSON list of aliases only LM Studio provides +# LITELLM_MOCK_ALIASES JSON list of aliases answered from a canned string # LITELLM_REMOTE_ALIASES JSON list of aliases a configured provider key adds # LITELLM_OLLAMA_ENABLED true|false # LITELLM_LMSTUDIO_ENABLED true|false @@ -25,11 +26,6 @@ here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" READY_RETRIES="${READY_RETRIES:-30}" READY_SLEEP_SECONDS="${READY_SLEEP_SECONDS:-5}" -MASTER_KEY="$(printf '%s' "${LITELLM_MASTER_KEY_B64}" | base64 -d 2>/dev/null)" -[ -n "${MASTER_KEY}" ] || { - echo "[FATAL] LITELLM_MASTER_KEY_B64 missing or undecodable" >&2 - exit 2 -} [ -n "${LITELLM_CONTAINER}" ] || { echo "[FATAL] LITELLM_CONTAINER unset; the gateway container was not resolved" >&2 exit 2 @@ -50,14 +46,17 @@ probe() { # nocheck: container-exec-resolver address resolved by the caller and passed in container exec -i \ -e "PORT=${LITELLM_PORT}" \ - -e "MASTER_KEY=${MASTER_KEY}" \ + -e "MIN_CHARS_PER_TOKEN=${LITELLM_ROUTER_MIN_CHARS_PER_TOKEN}" \ -e "CHAT_MODEL=${LITELLM_CHAT_MODEL}" \ -e "CHAT_MODEL_SERVED=${LITELLM_CHAT_MODEL_SERVED}" \ -e "EXPECTED_MODELS=${LITELLM_EXPECTED_MODELS}" \ -e "LMSTUDIO_ALIASES=${LITELLM_LMSTUDIO_ALIASES}" \ + -e "MOCK_ALIASES=${LITELLM_MOCK_ALIASES}" \ -e "REMOTE_ALIASES=${LITELLM_REMOTE_ALIASES}" \ -e "OLLAMA_ENABLED=${LITELLM_OLLAMA_ENABLED}" \ -e "LMSTUDIO_ENABLED=${LITELLM_LMSTUDIO_ENABLED}" \ + -e "ROUTER_ALIAS=${LITELLM_ROUTER_ALIAS}" \ + -e "ROUTER_WINDOWS=${LITELLM_ROUTER_WINDOWS}" \ "${LITELLM_CONTAINER}" "$(interpreter)" - <"${here}/probe.py" } diff --git a/roles/svc-ai-litellm/meta/secrets.yml b/roles/svc-ai-litellm/meta/secrets.yml index a2fb9ce606..504b91dd68 100644 --- a/roles/svc-ai-litellm/meta/secrets.yml +++ b/roles/svc-ai-litellm/meta/secrets.yml @@ -22,7 +22,7 @@ credentials: regex: "\\S{20,}" default: "{{ lookup('api', 'anthropic.api_key') }}" openrouter_api_key: - description: "OpenRouter API key; empty leaves openrouter/auto unpublished" + description: "OpenRouter API key; empty leaves the openrouter provider's remote_models unpublished" type: "string" regex: "\\S{20,}" default: "{{ lookup('api', 'openrouter.api_key') }}" diff --git a/roles/svc-ai-litellm/meta/services.yml b/roles/svc-ai-litellm/meta/services.yml index 663f5e67c0..3d8b744190 100644 --- a/roles/svc-ai-litellm/meta/services.yml +++ b/roles/svc-ai-litellm/meta/services.yml @@ -15,6 +15,39 @@ litellm: name: litellm max_output_tokens: 512 upstream_timeout: 240 + router: + alias: auto + remote_fallback: false + # nocheck: dynamic-flag + strategy: "{{ 'system_one' if (lookup('config', 'svc-ai-litellm', 'services.s1.enabled') | bool) else 'weighted' }}" + min_chars_per_token: 2 + state_chars: 512 + weights: + locality: 1.0 + cost: 0.4 + speed: 0.25 + caching: 0.15 + throughput: 0.1 + speed: + samples: 12 + timeout: 60 + remeasure: false + sample: + enabled: false + mode: shadow + trigger: seconds + interval: 900 + width: 3 + remote_models: + - alias: openai/gpt-4o-mini + model: openai/gpt-4o-mini + provider: openai + - alias: anthropic/claude-sonnet-4-5 + model: anthropic/claude-sonnet-4-5 + provider: anthropic + - alias: openrouter/auto + model: openrouter/openrouter/auto + provider: openrouter min_storage: 1022MB cpus: "2.0" mem_reservation: 1g @@ -34,6 +67,13 @@ postgres: enabled: true # nocheck: dynamic-flag shared: "{{ 'svc-db-postgres' in group_names }}" +# nocheck: playwright-service-flag, dynamic-flag +# nocheck: gateway-backend-registered Reason: the decider picks which model answers, it is not one; the router hook calls /v1/systemone, no model alias resolves to it +s1: + bond: 0.5 + enabled: "{{ 'svc-ai-s1' in group_names }}" + shared: "{{ 'svc-ai-s1' in group_names }}" + # nocheck: playwright-service-flag, dynamic-flag ollama: bond: 1 diff --git a/roles/svc-ai-litellm/meta/variants.yml b/roles/svc-ai-litellm/meta/variants.yml index 8ae1ddd8aa..8b01676c90 100644 --- a/roles/svc-ai-litellm/meta/variants.yml +++ b/roles/svc-ai-litellm/meta/variants.yml @@ -1,6 +1,13 @@ --- # Matrix-deploy variant overrides for this role. # See docs/contributing/artefact/files/role/variants.md for the file format. +# +# The deterministic mock is declared in every variant: it needs no key, costs no +# wall clock, and the agent broker runs its agents on it in every round so the +# broker never waits on inference to reach a verdict. Whether a real model is +# LOADED is decided in the backend roles' own variants, because preload_models +# belongs to svc-ai-ollama and svc-ai-lmstudio, not to this role's consumer +# flags. With nothing preloaded the chat model falls through to the mock here. - services: ollama: @@ -11,16 +18,37 @@ shared: false postgres: shared: true + litellm: + router: + speed: + samples: 1 + remote_models: + - alias: mock/deterministic + provider: mock + response: "pong" + context: 131072 - services: postgres: shared: false ollama: - enabled: false - shared: false # nocheck: variants-static-override backup shared stays literally true + enabled: true + shared: true lmstudio: enabled: false shared: false # nocheck: variants-static-override backup shared stays literally true + s1: + enabled: true + shared: true + litellm: + router: + speed: + samples: 1 + remote_models: + - alias: mock/deterministic + provider: mock + response: "pong" + context: 131072 - services: ollama: @@ -32,3 +60,23 @@ shared: true postgres: shared: true + s1: + enabled: true + shared: true + litellm: + router: + speed: + samples: 1 + remote_models: + - alias: mock/deterministic + provider: mock + response: "pong" + context: 131072 + - alias: mock/medium + provider: mock + response: "pong" + context: 16384 + - alias: mock/small + provider: mock + response: "pong" + context: 4096 diff --git a/roles/svc-ai-litellm/meta/volumes.yml b/roles/svc-ai-litellm/meta/volumes.yml index 7ceb601ef6..a2c57107dc 100644 --- a/roles/svc-ai-litellm/meta/volumes.yml +++ b/roles/svc-ai-litellm/meta/volumes.yml @@ -6,3 +6,11 @@ litellm_config: mounts: - service: litellm target: /etc/litellm/config.yaml + +litellm_router_hook: + type: config + source: "{{ [ lookup('container', 'svc-ai-litellm', 'directories.config'), 'router_hook.py' ] | path_join }}" + mode: "0444" + mounts: + - service: litellm + target: /etc/litellm/router_hook.py diff --git a/roles/svc-ai-litellm/tasks/00_core.yml b/roles/svc-ai-litellm/tasks/00_core.yml index 9076c123b0..44a57461cc 100644 --- a/roles/svc-ai-litellm/tasks/00_core.yml +++ b/roles/svc-ai-litellm/tasks/00_core.yml @@ -13,11 +13,43 @@ database_consumer_id: "{{ application_id }}" when: lookup('database', application_id, 'enabled') | bool +- name: "🛑 Refuse a remote model whose provider has no key slot" + ansible.builtin.assert: + that: + - LITELLM_REMOTE_MODELS | map(attribute='provider') | difference((LITELLM_PROVIDER_KEYS | list) + [AI_MOCK_PROVIDER]) | length == 0 + fail_msg: >- + services.litellm.remote_models names provider(s) + {{ LITELLM_REMOTE_MODELS | map(attribute='provider') | difference((LITELLM_PROVIDER_KEYS | list) + [AI_MOCK_PROVIDER]) | unique | join(', ') }}; + known providers are {{ ((LITELLM_PROVIDER_KEYS | list) + [AI_MOCK_PROVIDER]) | join(', ') }}. A model served by + '{{ AI_MOCK_PROVIDER }}' answers from its own declared response and needs no key. + quiet: true + +- name: "🔍 Look for the rates the last deploy measured" + ansible.builtin.stat: + path: "{{ LITELLM_SPEED_PATH_HOST }}" + delegate_to: "{{ STACK_HOST }}" + run_once: true + register: litellm_speed_file + +- name: "📥 Read the rates the last deploy measured" + ansible.builtin.slurp: + src: "{{ LITELLM_SPEED_PATH_HOST }}" + delegate_to: "{{ STACK_HOST }}" + run_once: true + register: litellm_speed_stored + when: litellm_speed_file.stat.exists + +- name: "🧮 Rank this render on those rates" + ansible.builtin.set_fact: + LITELLM_MEASURED_SPEED: "{{ (litellm_speed_stored.content | b64decode | from_json) if litellm_speed_file.stat.exists else {} }}" + - name: "📄 Render LiteLLM gateway config (local + swarm peers)" ansible.builtin.include_tasks: file: "{{ lookup('path_absolute', 'roles/sys-svc-compose/tasks/utils/render_replicated_templates.yml') }}" vars: role_templates: + - src: "router_hook.py.j2" + dest: "{{ LITELLM_ROUTER_HOOK_PATH_HOST }}" - src: "config.yaml.j2" dest: "{{ LITELLM_CONFIG_PATH_HOST }}" @@ -40,7 +72,7 @@ ansible.builtin.set_fact: LITELLM_CONSUMER_KEYS: >- {%- set out = [] -%} - {%- for consumer in lookup('roles_with_service', 'litellm', scope='all') -%} + {%- for consumer in lookup('roles_with_service', 'litellm', scope='all', include_headless=true) -%} {%- set key = lookup('config', consumer.id, 'secrets.credentials.litellm_api_key', '') -%} {%- if key -%} {%- set _ = out.append({'alias': consumer.id, 'key': key}) -%} @@ -53,3 +85,13 @@ when: - IS_STACK_HOST | bool - LITELLM_CONSUMER_KEYS | length > 0 + +- name: "🔁 Publish the current gateway models in Open WebUI" + ansible.builtin.include_role: + name: web-app-openwebui + tasks_from: utils/refresh_model_grants.yml + vars: + application_id: web-app-openwebui + when: + - "'web-app-openwebui' in group_names" + - lookup('config', 'web-app-openwebui', 'services.agent-broker.enabled') | bool diff --git a/roles/svc-ai-litellm/tasks/01_keys.yml b/roles/svc-ai-litellm/tasks/01_keys.yml index bd43ae8eb5..2e3ad7da49 100644 --- a/roles/svc-ai-litellm/tasks/01_keys.yml +++ b/roles/svc-ai-litellm/tasks/01_keys.yml @@ -18,7 +18,9 @@ - name: "🔑 Ensure a virtual key exists per consumer" ansible.builtin.shell: | # nocheck: container-exec-resolver runtime-resolved CID, delegated to the hosting node - container exec -i -e LITELLM_KEYS_PAYLOAD={{ LITELLM_CONSUMER_KEYS | to_json | quote }} -e LITELLM_MK={{ LITELLM_MASTER_KEY | quote }} -e LITELLM_PORT={{ LITELLM_PORT | quote }} {{ LITELLM_LOCAL_CID | quote }} python3 -c {{ lookup('file', lookup('path_absolute', 'roles/svc-ai-litellm/files/python/ensure_keys.py')) | quote }} + container exec -i -e LITELLM_PORT={{ LITELLM_PORT | quote }} {{ LITELLM_LOCAL_CID | quote }} python3 -c {{ lookup('file', lookup('path_absolute', 'roles/svc-ai-litellm/files/python/ensure_keys.py')) | quote }} + args: + stdin: "{{ LITELLM_CONSUMER_KEYS | to_json }}" register: litellm_keys_result retries: 10 delay: 6 @@ -29,7 +31,9 @@ - name: "🛡️ Prove the gateway is closed and revocation stays per consumer" ansible.builtin.shell: | # nocheck: container-exec-resolver runtime-resolved CID, delegated to the hosting node - container exec -i -e LITELLM_MK={{ LITELLM_MASTER_KEY | quote }} -e LITELLM_PORT={{ LITELLM_PORT | quote }} -e LITELLM_KEY={{ (LITELLM_CONSUMER_KEYS | first).key | quote }} {{ LITELLM_LOCAL_CID | quote }} python3 -c {{ lookup('file', lookup('path_absolute', 'roles/svc-ai-litellm/files/python/probe_auth.py')) | quote }} + container exec -i -e LITELLM_PORT={{ LITELLM_PORT | quote }} {{ LITELLM_LOCAL_CID | quote }} python3 -c {{ lookup('file', lookup('path_absolute', 'roles/svc-ai-litellm/files/python/probe_auth.py')) | quote }} + args: + stdin: "{{ {'key': (LITELLM_CONSUMER_KEYS | first).key} | to_json }}" register: litellm_auth_probe retries: 5 delay: 10 @@ -40,12 +44,17 @@ - name: "🔥 Warm the chat model so the first prompt is not a cold load" ansible.builtin.shell: | # nocheck: container-exec-resolver runtime-resolved CID, delegated to the hosting node - container exec -i -e LITELLM_MK={{ LITELLM_MASTER_KEY | quote }} -e LITELLM_PORT={{ LITELLM_PORT | quote }} -e LITELLM_MODEL={{ LITELLM_CHAT_MODEL | quote }} {{ LITELLM_LOCAL_CID | quote }} python3 -c {{ lookup('file', lookup('path_absolute', 'roles/svc-ai-litellm/files/python/warm_model.py')) | quote }} + container exec -i -e LITELLM_PORT={{ LITELLM_PORT | quote }} -e LITELLM_MODEL={{ LITELLM_CHAT_MODEL | quote }} {{ LITELLM_LOCAL_CID | quote }} python3 -c {{ lookup('file', lookup('path_absolute', 'roles/svc-ai-litellm/files/python/warm_model.py')) | quote }} register: litellm_warm_result retries: 5 delay: 10 until: litellm_warm_result.rc == 0 changed_when: false - no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" delegate_to: "{{ LITELLM_HOST_NODE }}" when: LITELLM_CHAT_MODEL_SERVED | bool + +- name: "⏱️ Rank the served models on measured generation speed" + ansible.builtin.include_tasks: 02_speed.yml + when: + - LITELLM_ROUTER_ALIAS | length > 0 + - LITELLM_CHAT_MODEL_SERVED | bool diff --git a/roles/svc-ai-litellm/tasks/02_speed.yml b/roles/svc-ai-litellm/tasks/02_speed.yml new file mode 100644 index 0000000000..b2db7de51a --- /dev/null +++ b/roles/svc-ai-litellm/tasks/02_speed.yml @@ -0,0 +1,30 @@ +--- +- name: "⏱️ Measure how fast each served model generates" + ansible.builtin.shell: | # nocheck: container-exec-resolver runtime-resolved CID, delegated to the hosting node + container exec -i -e LITELLM_PORT={{ LITELLM_PORT | quote }} -e LITELLM_SAMPLES={{ LITELLM_ROUTER_SPEED_SAMPLES | quote }} -e LITELLM_TIMEOUT={{ LITELLM_ROUTER_SPEED_TIMEOUT | quote }} -e LITELLM_WARMUP={{ LITELLM_UPSTREAM_TIMEOUT | quote }} -e LITELLM_EXCLUDE={{ LITELLM_UNRANKED_ALIASES | quote }} -e LITELLM_REMEASURE={{ LITELLM_ROUTER_SPEED_REMEASURE | bool | ternary('1', '') | quote }} -e LITELLM_PREVIOUS={{ LITELLM_MEASURED_SPEED | to_json | quote }} {{ LITELLM_LOCAL_CID | quote }} python3 -c {{ lookup('file', lookup('path_absolute', 'roles/svc-ai-litellm/files/python/measure_speed.py')) | quote }} + register: litellm_speed_result + retries: 3 + delay: 5 + until: litellm_speed_result.rc == 0 + changed_when: false + delegate_to: "{{ LITELLM_HOST_NODE }}" + +- name: "📈 Store the measured rates for the next deploy" + ansible.builtin.copy: + content: "{{ litellm_speed_result.stdout }}\n" + dest: "{{ LITELLM_SPEED_PATH_HOST }}" + mode: "0644" + delegate_to: "{{ STACK_HOST }}" + run_once: true + +- name: "🧮 Rank the gateway on the rates just measured" + ansible.builtin.set_fact: + LITELLM_MEASURED_SPEED: "{{ litellm_speed_result.stdout | from_json }}" + +- name: "📄 Re-render the gateway config with the measured rates" + ansible.builtin.include_tasks: + file: "{{ lookup('path_absolute', 'roles/sys-svc-compose/tasks/utils/render_replicated_templates.yml') }}" + vars: + role_templates: + - src: "config.yaml.j2" + dest: "{{ LITELLM_CONFIG_PATH_HOST }}" diff --git a/roles/svc-ai-litellm/tasks/utils/consumer_contract.yml b/roles/svc-ai-litellm/tasks/utils/consumer_contract.yml index deeffb6c59..0e163a9462 100644 --- a/roles/svc-ai-litellm/tasks/utils/consumer_contract.yml +++ b/roles/svc-ai-litellm/tasks/utils/consumer_contract.yml @@ -1,4 +1,8 @@ --- +- name: "🔧 Read how '{{ application_id }}' reaches the gateway" + ansible.builtin.set_fact: + _litellm_consumer_on_host: "{{ lookup('config', application_id, 'services.' ~ (application_id | get_entity_name) ~ '.modes.host.enabled', false) | bool }}" + - name: "✅ Assert '{{ application_id }}' carries a gateway virtual key" ansible.builtin.assert: that: @@ -13,7 +17,7 @@ that: - (LITELLM_BASE_LOCAL_URL | urlsplit('scheme')) == 'http' - "'.' not in (LITELLM_BASE_LOCAL_URL | urlsplit('hostname'))" - - (LITELLM_BASE_LOCAL_URL | urlsplit('hostname')) != lookup('domain', application_id) + - (application_id not in lookup('domains')) or ((LITELLM_BASE_LOCAL_URL | urlsplit('hostname')) != lookup('domain', application_id)) - (LITELLM_BASE_LOCAL_URL | urlsplit('port') | int) == (lookup('config', 'svc-ai-litellm', 'services.litellm.ports.internal.http') | int) fail_msg: >- {{ application_id }} would send inference traffic to @@ -23,3 +27,20 @@ ingress), a host that is not this role's own domain, and the gateway's internal port {{ lookup('config', 'svc-ai-litellm', 'services.litellm.ports.internal.http') }} rather than its published one. + when: not (_litellm_consumer_on_host | bool) + +- name: "✅ Assert the AI base URL of '{{ application_id }}' stays on this machine" + ansible.builtin.assert: + that: + - (LITELLM_BASE_LOOPBACK_URL | urlsplit('scheme')) == 'http' + - (LITELLM_BASE_LOOPBACK_URL | urlsplit('hostname')) in LITELLM_LOOPBACK_HOSTS + - (LITELLM_BASE_LOOPBACK_URL | urlsplit('port') | int) == (lookup('config', 'svc-ai-litellm', 'services.litellm.ports.local.http') | int) + fail_msg: >- + {{ application_id }} runs on the host rather than in the deployment's + container network, so it would send inference traffic to + '{{ LITELLM_BASE_LOOPBACK_URL }}'. A workstation reaches the gateway only + over the loopback address the engine publishes it on: plain http, one of + {{ LITELLM_LOOPBACK_HOSTS | join(', ') }}, and the published port + {{ lookup('config', 'svc-ai-litellm', 'services.litellm.ports.local.http') }}. + Anything routable would carry the workstation's key off this machine. + when: _litellm_consumer_on_host | bool diff --git a/roles/svc-ai-litellm/templates/config.yaml.j2 b/roles/svc-ai-litellm/templates/config.yaml.j2 index c4bed5dcce..2a985c26e8 100644 --- a/roles/svc-ai-litellm/templates/config.yaml.j2 +++ b/roles/svc-ai-litellm/templates/config.yaml.j2 @@ -1,38 +1,44 @@ -{% set _ollama_models = lookup('config', 'svc-ai-ollama', 'services.ollama.preload_models') if LITELLM_OLLAMA_BACKEND | bool else [] %} -{% set _lmstudio_models = lookup('config', 'svc-ai-lmstudio', 'services.lmstudio.preload_models') if LITELLM_LMSTUDIO_BACKEND | bool else [] %} -{% set _remote_models = LITELLM_REMOTE_BACKENDS | selectattr('key') | list %} -{% set has_backend = _ollama_models or _lmstudio_models or _remote_models %} -model_list:{{ ' []' if not has_backend else '' }} -{% for _model in _ollama_models %} - - model_name: "{{ _model.alias }}" +{% set _routes = (lookup('config', 'svc-ai-ollama', 'services.ollama.preload_models') if LITELLM_OLLAMA_BACKEND | bool else []) + | litellm_model_routes( + (lookup('config', 'svc-ai-lmstudio', 'services.lmstudio.preload_models') if LITELLM_LMSTUDIO_BACKEND | bool else []), + (LITELLM_REMOTE_MODELS | selectattr('provider', 'in', LITELLM_SERVED_PROVIDERS) | list), + ollama_url=OLLAMA_BASE_LOCAL_URL, + lmstudio_url=LMSTUDIO_BASE_LOCAL_URL, + max_tokens=LITELLM_MAX_OUTPUT_TOKENS, + timeout=LITELLM_UPSTREAM_TIMEOUT, + mock_provider=AI_MOCK_PROVIDER, + router_alias=LITELLM_ROUTER_ALIAS) %} +model_list:{{ ' []' if not _routes else '' }} +{% for _route in _routes %} +{% set _traits = _route.traits or {} %} +{% set _speed = _route.alias in LITELLM_MEASURED_SPEED and 'speed' not in _traits %} + - model_name: {{ _route.alias | to_json }} litellm_params: - model: "ollama/{{ _model.alias }}" - api_base: {{ OLLAMA_BASE_LOCAL_URL }} - rpm: 1000 - max_tokens: {{ LITELLM_MAX_OUTPUT_TOKENS }} - timeout: {{ LITELLM_UPSTREAM_TIMEOUT }} +{% for _key, _value in _route.params.items() %} + {{ _key }}: {{ _value | to_json }} {% endfor %} -{% for _model in _lmstudio_models if _model.alias not in (_ollama_models | map(attribute='alias') | list) %} - - model_name: "{{ _model.alias }}" - litellm_params: - model: "openai/{{ _model.name }}" - api_base: {{ LMSTUDIO_BASE_LOCAL_URL }}/v1 - api_key: lm-studio - rpm: 1000 - max_tokens: {{ LITELLM_MAX_OUTPUT_TOKENS }} - timeout: {{ LITELLM_UPSTREAM_TIMEOUT }} +{% if _route.context or _traits or _speed %} + model_info: +{% if _route.context %} + max_input_tokens: {{ _route.context }} +{% endif %} +{% if _traits or _speed %} + traits: +{% for _trait, _value in _traits.items() %} + {{ _trait }}: {{ _value | to_json }} {% endfor %} -{% for _backend in _remote_models %} - - model_name: "{{ _backend.alias }}" - litellm_params: - model: "{{ _backend.model }}" - api_key: os.environ/{{ _backend.env }} - max_tokens: {{ LITELLM_MAX_OUTPUT_TOKENS }} - timeout: {{ LITELLM_UPSTREAM_TIMEOUT }} +{% if _speed %} + speed: {{ LITELLM_MEASURED_SPEED[_route.alias] | to_json }} +{% endif %} +{% endif %} +{% endif %} {% endfor %} litellm_settings: drop_params: true +{% if LITELLM_ROUTER_ALIAS %} + callbacks: ["router_hook.instance"] +{% endif %} general_settings: master_key: os.environ/LITELLM_MASTER_KEY diff --git a/roles/svc-ai-litellm/templates/env.j2 b/roles/svc-ai-litellm/templates/env.j2 index e77c43da8e..693cdd3707 100644 --- a/roles/svc-ai-litellm/templates/env.j2 +++ b/roles/svc-ai-litellm/templates/env.j2 @@ -5,8 +5,8 @@ DATABASE_URL={{ lookup('database', application_id, 'url_full') | dotenv_quote }} STORE_MODEL_IN_DB=False UI_USERNAME={{ lookup('users', 'administrator').username }} UI_PASSWORD={{ lookup('config', application_id, 'secrets.credentials.ui_password') | dotenv_quote }} -{% for _backend in LITELLM_REMOTE_BACKENDS if _backend.key %} -{{ _backend.env }}={{ _backend.key | dotenv_quote }} +{% for _provider in LITELLM_KEYED_PROVIDERS %} +{{ _provider | upper }}_API_KEY={{ LITELLM_PROVIDER_KEYS[_provider] | dotenv_quote }} {% endfor %} {% if LITELLM_UI_SSO | bool %} PROXY_BASE_URL={{ LITELLM_UI_BASE_URL }} diff --git a/roles/svc-ai-litellm/templates/router_hook.py.j2 b/roles/svc-ai-litellm/templates/router_hook.py.j2 new file mode 100644 index 0000000000..28fa01f06d --- /dev/null +++ b/roles/svc-ai-litellm/templates/router_hook.py.j2 @@ -0,0 +1,818 @@ +"""Pick the model that can serve a request, before the proxy dispatches it. + +LiteLLM loads this through ``get_instance_fn``, which resolves +``router_hook.instance`` against the directory of ``config.yaml``, so the file +lives beside it in the container and the image stays the upstream one. + +The demand side is estimated from the request body. The supply side is read off +LiteLLM's own model catalog, which ships 1595 entries in the image carrying the +context window, the modality and tool support, the per-token prices and the +deprecation date of every model it knows. A route may override any of that +through ``model_info.traits``, and nothing here probes a backend. + +Two polarities govern the filters, and mixing them up routes silently wrong: + +* A capability the request *demands* must be proven. Vision, tools, a response + schema, PDF or audio input are catalogued wherever they exist, so an absent + flag means absent support and the route is dropped only when the request asks + for it. +* A *disqualifier* acts on explicit evidence only. Most entries carry no + ``supports_system_messages`` and local tags are catalogued not at all, so + treating silence as denial would drop nearly every route. A model is + therefore excluded only when the catalog states a non-chat mode, denies + system messages outright, or gives a deprecation date already past. +""" + +from __future__ import annotations + +import asyncio +import json +import random +import re +import time +import urllib.request +from datetime import date + +import litellm +from litellm.integrations.custom_logger import CustomLogger + +ROUTER_ALIAS = {{ LITELLM_ROUTER_ALIAS | to_json }} +REMOTE_FALLBACK = {{ 'True' if LITELLM_ROUTER_REMOTE_FALLBACK | bool else 'False' }} +ROUTER_WEIGHTS = {{ LITELLM_ROUTER_WEIGHTS | to_json }} +ROUTER_STRATEGY = {{ LITELLM_ROUTER_STRATEGY | to_json }} +S1_URL = {{ LITELLM_S1_URL | to_json }} +S1_KEY = {{ LITELLM_S1_KEY | to_json }} +S1_MODEL = {{ LITELLM_S1_MODEL | to_json }} +S1_TIMEOUT = {{ LITELLM_S1_TIMEOUT | int }} +S1_QUESTION = "route" +S1_VERDICT_QUESTION = "verdict" +S1_MAX_STATE_CHARS = {{ LITELLM_ROUTER_STATE_CHARS | int }} + +SAMPLE_ENABLED = {{ 'True' if LITELLM_ROUTER_SAMPLE_ENABLED | bool else 'False' }} +SAMPLE_MODE = {{ LITELLM_ROUTER_SAMPLE_MODE | to_json }} +SAMPLE_TRIGGER = {{ LITELLM_ROUTER_SAMPLE_TRIGGER | to_json }} +SAMPLE_INTERVAL = {{ LITELLM_ROUTER_SAMPLE_INTERVAL | int }} +SAMPLE_WIDTH = {{ LITELLM_ROUTER_SAMPLE_WIDTH | int }} +HISTORY_TABLE = "infinito_router_history" +TOKEN_BUCKETS = (256, 1024, 4096, 16384, 65536) + +SECRET_SHAPES = re.compile( + r"-----BEGIN [A-Z ]*PRIVATE KEY-----" + r"|\bsk-[A-Za-z0-9_-]{20,}" + r"|\bgh[pousr]_[A-Za-z0-9]{20,}" + r"|\bgithub_pat_[A-Za-z0-9_]{20,}" + r"|\bAKIA[A-Z0-9]{16}\b" + r"|\bxox[baprs]-[A-Za-z0-9-]{10,}" + r"|\bAIza[A-Za-z0-9_-]{35}\b" +) +ISO_DATE = re.compile(r"^\d{4}-\d{2}-\d{2}$") + +MIN_CHARS_PER_TOKEN = {{ LITELLM_ROUTER_MIN_CHARS_PER_TOKEN | int }} +IMAGE_PARTS = ("image_url", "input_image") +AUDIO_PARTS = ("input_audio", "audio_url") +DOCUMENT_PARTS = ("file", "input_file", "document") +SYSTEM_ROLES = ("system", "developer") +UNPRICED_REMOTE_COST = 1.0 +NOMINAL_OUTPUT_TOKENS = 512 +CHAT_MODES = ("chat", "responses", None) + + +def summarize(data): + """What the request demands, in the terms a route declares. + + Args: + data: the OpenAI-shaped request body the proxy received. + + Returns: + A mapping with the estimated ``input_tokens`` and ``max_output`` plus + one flag per capability the body asks for. + """ + characters = 0 + need = { + "vision": False, + "audio": False, + "documents": False, + "images": 0, + "system": False, + } + for message in data.get("messages") or []: + if message.get("role") in SYSTEM_ROLES: + need["system"] = True + content = message.get("content") + if isinstance(content, str): + characters += len(content) + continue + for part in content or []: + if not isinstance(part, dict): + continue + kind = part.get("type") + if kind in IMAGE_PARTS: + need["vision"] = True + need["images"] += 1 + elif kind in AUDIO_PARTS: + need["audio"] = True + elif kind in DOCUMENT_PARTS: + need["documents"] = True + else: + characters += len(part.get("text") or "") + response_format = data.get("response_format") or {} + need.update( + { + "input_tokens": -(-characters // MIN_CHARS_PER_TOKEN), + "max_output": int(data.get("max_tokens") or 0), + "tools": bool(data.get("tools") or data.get("functions")), + "tool_choice": data.get("tool_choice") not in (None, "none"), + "parallel_tools": bool(data.get("parallel_tool_calls")), + "schema": response_format.get("type") + in ("json_schema", "json_object"), + } + ) + return need + + +def _catalogued(model, catalog): + """The catalog entry for *model*, trying the prefixed and the bare key. + + ``ollama/llama3`` is a key as it stands, ``openai/gpt-4o-mini`` is filed + under ``gpt-4o-mini``, and a local tag or a mock is in neither. + """ + for key in (model, model.split("/", 1)[-1]): + found = catalog.get(key) + if isinstance(found, dict): + return found + return {} + + +def is_local(params): + """Whether this deployment serves the route itself. + + An ``api_base`` points at a backend of this deployment and a + ``mock_response`` never leaves the proxy. Anything else reaches a third + party, which is why this is read twice: once as the factor the operator + weights, and once as the filter that no weight can reach. + """ + return params.get("mock_response") is not None or bool(params.get("api_base")) + + +def _deprecated(entry, today=None): + """Whether the catalog gives a deprecation date already past. + + The catalog ships a ``sample_spec`` documentation stub whose date field is + prose, so only an ISO value counts and anything else reads as undated. + """ + value = str(entry.get("deprecation_date") or "") + if not ISO_DATE.match(value): + return False + return value < (today or date.today().isoformat()) + + +def capabilities(entry, catalog, today=None): + """What a route can do: the catalog's facts, overridden by declarations. + + Args: + entry: one ``model_list`` entry. + catalog: ``litellm.model_cost`` or a stand-in keyed the same way. + today: the ISO date deprecation is measured against. + + Returns: + A mapping of every capability, limit and price the filters and the + strategies read. A demanded capability counts as absent unless proven; + a disqualifier counts as harmless unless the catalog states it. + """ + params = entry.get("litellm_params") or {} + info = entry.get("model_info") or {} + known = _catalogued(params.get("model") or "", catalog) + local = is_local(params) + caps = { + "context": info.get("max_input_tokens") or known.get("max_input_tokens"), + "max_output": known.get("max_output_tokens"), + "vision": bool(known.get("supports_vision")), + "tools": bool(known.get("supports_function_calling")), + "tool_choice": bool(known.get("supports_tool_choice")), + "parallel_tools": bool(known.get("supports_parallel_function_calling")), + "schema": bool(known.get("supports_response_schema")), + "documents": bool(known.get("supports_pdf_input")), + "audio": bool(known.get("supports_audio_input")), + "images": known.get("max_images_per_prompt"), + "system": known.get("supports_system_messages") is not False, + "servable": known.get("mode") in CHAT_MODES or not known, + "deprecated": _deprecated(known, today), + "input_cost": known.get( + "input_cost_per_token", 0.0 if local else UNPRICED_REMOTE_COST + ), + "output_cost": known.get( + "output_cost_per_token", 0.0 if local else UNPRICED_REMOTE_COST + ), + "caching": bool(known.get("supports_prompt_caching")), + "throughput": known.get("tpm") or 0, + "local": local, + "speed": 0.0, + } + caps.update( + {key: value for key, value in (info.get("traits") or {}).items() if key in caps} + ) + return caps + + +def expected_cost(caps, need): + """What this route would bill for this request, input plus output. + + Ranking on the input price alone reads the cheaper half of the bill: an + output token commonly costs several times an input one, and a short prompt + with a long answer inverts the order the input price suggests. + """ + output_tokens = need["max_output"] or NOMINAL_OUTPUT_TOKENS + return ( + float(caps["input_cost"]) * need["input_tokens"] + + float(caps["output_cost"]) * output_tokens + ) + + +def request_text(data): + """Everything the caller wrote, as one string.""" + parts = [] + for message in data.get("messages") or []: + content = message.get("content") + if isinstance(content, str): + parts.append(content) + continue + for part in content or []: + if isinstance(part, dict) and part.get("text"): + parts.append(str(part["text"])) + return "\n".join(parts) + + +def carries_secret(data): + """Whether the request shows a structured secret.""" + return bool(SECRET_SHAPES.search(request_text(data))) + + +def local_reason(data, deployments, catalog): + """Why this request must stay local, or the empty string. + + Returns empty when no local route is deployed at all, because then there is + nothing to keep it on and refusing would only turn a working request into + an error without protecting anything. + """ + if not any( + capabilities(entry, catalog)["local"] + for entry in deployments or [] + if entry.get("model_name") not in (None, ROUTER_ALIAS) + ): + return "" + if carries_secret(data): + return "the request shows a structured secret" + if not REMOTE_FALLBACK: + return "services.litellm.router.remote_fallback is off" + return "" + + +def serves(caps, need): + """Whether one route can take this request, or the reason it cannot.""" + context = caps["context"] + if context and need["input_tokens"] + need["max_output"] > int(context): + return "its context window is too small" + output_cap = caps["max_output"] + if need["max_output"] and output_cap and need["max_output"] > int(output_cap): + return "it caps the output below the request" + for demand in ("vision", "audio", "documents", "tools", "tool_choice", + "parallel_tools", "schema"): + if need[demand] and not caps[demand]: + return f"it does not support {demand.replace('_', ' ')}" + if need["images"] and caps["images"] and need["images"] > int(caps["images"]): + return "it takes fewer images than the request carries" + if need["system"] and not caps["system"]: + return "it does not take system messages" + if not caps["servable"]: + return "it is not a chat model" + if caps["deprecated"]: + return "it is past its deprecation date" + return "" + + +def eligible(deployments, need, exclude, catalog, today=None): + """The routes that can serve this request, in publication order. + + Args: + deployments: the proxy's own ``model_list`` entries. + need: the mapping ``summarize`` returned. + exclude: the alias the request named, so the router never picks itself. + catalog: the model catalog ``capabilities`` reads. + today: the ISO date deprecation is measured against. + + Returns: + The surviving entries, each carrying its resolved capabilities under + ``_caps`` and this request's expected price under ``_cost``. + """ + survivors = [] + for entry in deployments or []: + alias = entry.get("model_name") + if not alias or alias == exclude: + continue + caps = capabilities(entry, catalog, today) + if serves(caps, need): + continue + survivors.append( + {**entry, "_caps": caps, "_cost": expected_cost(caps, need)} + ) + return survivors + + +def _spread(values): + """The low and high of *values*, for scaling one factor against another.""" + numbers = [float(value) for value in values] + return min(numbers), max(numbers) + + +def _scaled(value, spread, *, higher_is_better): + """*value* as 0..1 within *spread*, where 1 is always the better end. + + A spread of zero means the factor cannot separate this candidate set, so + every route scores 1 and the factor drops out instead of deciding on a + rounding artefact. + """ + low, high = spread + if high == low: + return 1.0 + position = (float(value) - low) / (high - low) + return position if higher_is_better else 1.0 - position + + +def score(entry, spreads, weights): + """How well a route serves the operator's stated priorities, 0..1 each. + + Each factor is scaled inside the candidate set rather than against an + absolute, because the units do not compare: a price is a fraction of a + cent, a rate is tens of tokens per second and a quota is millions. + """ + caps = entry["_caps"] + return ( + float(weights.get("locality", 0.0)) * (1.0 if caps["local"] else 0.0) + + float(weights.get("cost", 0.0)) + * _scaled(entry["_cost"], spreads["cost"], higher_is_better=False) + + float(weights.get("speed", 0.0)) + * _scaled(caps["speed"], spreads["speed"], higher_is_better=True) + + float(weights.get("caching", 0.0)) * (1.0 if caps["caching"] else 0.0) + + float(weights.get("throughput", 0.0)) + * _scaled(caps["throughput"], spreads["throughput"], higher_is_better=True) + ) + + +def decide_weighted(candidates, weights): + """The highest scoring route, ties broken by alias so a rerun repeats. + + Weights come from ``services.litellm.router_weights`` and say how much each + factor matters, not whether it may be overruled. Confidentiality is not in + here: a request that must stay local was already restricted before this + ran, because a weight is a preference and a credential leaving the cluster + is not something a preference should be able to buy. + """ + spreads = { + "cost": _spread(entry["_cost"] for entry in candidates), + "speed": _spread(entry["_caps"]["speed"] for entry in candidates), + "throughput": _spread(entry["_caps"]["throughput"] for entry in candidates), + } + return min( + candidates, + key=lambda entry: (-score(entry, spreads, weights), entry["model_name"]), + )["model_name"] + + +def bucket(tokens): + """The token band *tokens* falls in, as the label history groups under.""" + for edge in TOKEN_BUCKETS: + if tokens <= edge: + return str(edge) + return f"{TOKEN_BUCKETS[-1]}+" + + +def signature(need): + """The request class routing history accumulates under. + + Two requests share a class when they ask a route for the same kind of work: + the same token band and the same capability demands. Grouping finer spreads + the evidence too thin to ever decide; grouping coarser lets a verdict about + a vision request speak for a plain chat. + """ + demands = "+".join( + flag + for flag in ("vision", "tools", "schema", "audio", "documents") + if need[flag] + ) + return f"{bucket(need['input_tokens'] + need['max_output'])}:{demands or 'plain'}" + + +class Sampler: + """Whether this request is answered by several routes instead of one. + + Args: + trigger: ``seconds`` spaces comparisons in time, ``requests`` counts + them out, ``probability`` draws one in *interval*. + interval: seconds, requests or odds, read according to *trigger*. + clock: monotonic source, injected so a test does not wait. + draw: uniform source in [0, 1), injected for the same reason. + """ + + def __init__(self, trigger, interval, clock=time.monotonic, draw=random.random): + self._trigger = trigger + self._interval = max(int(interval), 1) + self._clock = clock + self._draw = draw + self._last = None + self._seen = 0 + + def due(self): + """True when this request should be compared across routes.""" + if self._trigger == "requests": + self._seen += 1 + if self._seen < self._interval: + return False + self._seen = 0 + return True + if self._trigger == "probability": + return self._draw() < 1.0 / self._interval + now = self._clock() + if self._last is not None and now - self._last < self._interval: + return False + self._last = now + return True + + +class History: + """What earlier comparisons said about each route, per request class. + + The store is the proxy's own database, so a verdict outlives a restart. A + proxy without one keeps nothing at all: a process-local tally read as + history would let a fresh container claim experience it never gathered. + """ + + DDL = ( + f"CREATE TABLE IF NOT EXISTS {HISTORY_TABLE} (" + "id BIGSERIAL PRIMARY KEY," + "decided_at TIMESTAMPTZ NOT NULL DEFAULT now()," + "signature TEXT NOT NULL," + "alias TEXT NOT NULL," + "won BOOLEAN NOT NULL)" + ) + INDEX = ( + f"CREATE INDEX IF NOT EXISTS {HISTORY_TABLE}_signature_idx " + f"ON {HISTORY_TABLE} (signature)" + ) + + def __init__(self, client): + self._client = client + self._ready = False + + async def _ensure(self): + if not self._ready: + await self._client.execute_raw(self.DDL) + await self._client.execute_raw(self.INDEX) + self._ready = True + + async def record(self, request_class, winner, offered): + """Write one row per offered route, marking which one won.""" + await self._ensure() + for alias in offered: + await self._client.execute_raw( + f"INSERT INTO {HISTORY_TABLE} (signature, alias, won) " + f"VALUES ($1, $2, $3)", + request_class, + alias, + alias == winner, + ) + + async def wins(self, request_class): + """``{alias: (won, seen)}`` for *request_class*, empty when untried.""" + await self._ensure() + rows = await self._client.query_raw( + f"SELECT alias, SUM(CASE WHEN won THEN 1 ELSE 0 END) AS won, " + f"COUNT(*) AS seen FROM {HISTORY_TABLE} " + f"WHERE signature = $1 GROUP BY alias", + request_class, + ) + return {row["alias"]: (int(row["won"]), int(row["seen"])) for row in rows or []} + + +def record_of(alias, tally): + """How *alias* fared before, as a clause, or empty when it has no record.""" + won, seen = tally.get(alias, (0, 0)) + if not seen: + return "" + return f"won {won} of {seen} comparisons like this" + + +def headroom(entry, need): + """How much context this route would leave unused on this request. + + A route whose window is unknown sorts last rather than first: no window + means no evidence of fit, and reading it as unlimited would hand every + request to the least described route. + """ + context = entry["_caps"]["context"] + if not context: + return float("inf") + return int(context) - (need["input_tokens"] + need["max_output"]) + + +def describe(entry, need, tally=None): + """One line telling the decider what this route is, in its own terms. + + Only measured or catalogued facts go in. A figure the catalog does not + state is left out rather than defaulted, because a decider cannot tell an + absent number from a zero one. + """ + caps = entry["_caps"] + window = caps["context"] or "unstated" + room = headroom(entry, need) + facts = [ + "runs in this cluster" if caps["local"] else "runs at a provider", + f"context window {window}", + "unknown headroom" if room == float("inf") else f"{int(room)} tokens spare", + f"costs about {entry['_cost']:.4f} for this request", + ] + if caps["speed"]: + facts.append(f"{caps['speed']:.1f} tokens per second measured") + if caps["throughput"]: + facts.append(f"{int(caps['throughput'])} tokens per minute quota") + if caps["caching"]: + facts.append("caches prompts") + earlier = record_of(entry["model_name"], tally or {}) + if earlier: + facts.append(earlier) + return ", ".join(facts) + + +def ask_s1(text, candidates, need, tally=None): + """Put the routing choice to the decider as one typed choice question. + + The decider answers typed questions rather than generating, so the + candidates become the options of a single ``choice`` and the prompt becomes + the state it classifies. Returns ``(alias, confidence)``. + + Raises: + OSError: when the decider cannot be reached. + ValueError: when it answers a shape or an option this router did not + offer, which must not be read as a route. + """ + criteria = { + entry["model_name"]: describe(entry, need, tally) for entry in candidates + } + answer = _systemone( + text, + S1_QUESTION, + "Which of these models should answer the request? Prefer the smallest " + "one that can serve it well.", + criteria, + ) + alias = answer.get("choice") + if alias not in criteria: + raise ValueError( + f"the decider chose {alias!r}, which was not among {sorted(criteria)}" + ) + return alias, float(answer.get("confidence") or 0.0) + + +def _systemone(state, question, instructions, criteria): + """Put one typed choice to the decider and return the answer block it sent. + + Raises: + OSError: when the decider cannot be reached. + ValueError: when it answers a different question than the one asked. + """ + payload = { + "state": state[:S1_MAX_STATE_CHARS], + "model": S1_MODEL, + "questions": { + question: { + "type": "choice", + "instructions": instructions, + "criteria": criteria, + } + }, + } + request = urllib.request.Request( + f"{S1_URL}/v1/systemone", + data=json.dumps(payload).encode(), + headers={ + "Authorization": f"Bearer {S1_KEY}", + "Content-Type": "application/json", + }, + method="POST", + ) + with urllib.request.urlopen(request, timeout=S1_TIMEOUT) as response: # noqa: S310 configured internal origin + body = json.loads(response.read()) + answer = (body.get("answers") or {}).get(question) + if not answer: + raise ValueError(f"the decider returned no answer to {question!r}") + return answer + + +def judge_answers(text, replies): + """The alias whose reply the decider calls the best answer to *text*. + + The options are the replies themselves, not the routes that produced them, + so the verdict rests on what came back rather than on what the catalog + promised. Aliases are never shown to the decider for the same reason. + """ + labels = {f"option-{index}": alias for index, alias in enumerate(sorted(replies))} + answer = _systemone( + text, + S1_VERDICT_QUESTION, + "Which of these answers serves the request best?", + {label: replies[alias][:S1_MAX_STATE_CHARS] for label, alias in labels.items()}, + ) + label = answer.get("choice") + if label not in labels: + raise ValueError( + f"the decider chose {label!r}, which was not among {sorted(labels)}" + ) + return labels[label] + + +def decide_system_one(candidates, need, text, tally=None): + """The route the decider picks for this prompt. + + The weighted strategy ranks on standing preferences and stops reading the + prompt once the filter has run. This one hands the prompt and the surviving + candidates to a System One model, which answers a typed choice without + generating. An unreachable decider is not routed around: the caller named + an alias precisely so it would not have to know which model answered, and + quietly substituting a different decider would make that answer mean + something else. + """ + alias, _confidence = ask_s1(text, candidates, need, tally) + return alias + + +def decide(candidates, need, text="", weights=None, strategy=None, tally=None): + """Dispatch to the configured strategy. + + Args: + candidates: the surviving entries from ``eligible``. + need: the mapping ``summarize`` returned. + text: what the caller wrote, which only ``system_one`` reads. + weights: the weighted strategy's factor weights. + strategy: ``system_one`` or ``weighted``. + tally: what earlier comparisons said, which only ``system_one`` reads. + + Returns: + The winning alias. + """ + strategy = ROUTER_STRATEGY if strategy is None else strategy + if strategy == "system_one": + return decide_system_one(candidates, need, text, tally) + return decide_weighted(candidates, ROUTER_WEIGHTS if weights is None else weights) + + +async def collect(router, data, candidates, width): + """Ask the first *width* candidates the same question. + + Returns ``{alias: reply}`` for those that answered. A route that raises is + left out rather than recorded as a loss: failing to answer is a fact about + the moment, not evidence that its answers are worse. + """ + chosen = candidates[:width] + + async def one(entry): + payload = {key: value for key, value in data.items() if key != "mock_response"} + payload["model"] = entry["model_name"] + reply = await router.acompletion(**payload) + return entry["model_name"], reply.choices[0].message.content or "" + + replies = {} + for outcome in await asyncio.gather( + *(one(entry) for entry in chosen), return_exceptions=True + ): + if isinstance(outcome, BaseException): + continue + alias, content = outcome + if content: + replies[alias] = content + return replies + + +class SystemOneRouter(CustomLogger): + """Rewrites the router alias to a model that can actually serve it. + + Every path that cannot route returns a string, which the proxy turns into a + RejectedRequestError carrying that text. Answering from some default + instead would hide a broken router behind a plausible reply, and the alias + exists precisely so callers do not have to know which model answered. + """ + + def __init__(self): + super().__init__() + self._sampler = Sampler(SAMPLE_TRIGGER, SAMPLE_INTERVAL) + self._store = None + + async def async_pre_call_hook(self, user_api_key_dict, cache, data, call_type): # noqa: ARG002 - LiteLLM hook signature + if data.get("model") != ROUTER_ALIAS: + return None + from litellm.proxy.proxy_server import llm_router + + if llm_router is None: + return ( + f"'{ROUTER_ALIAS}' cannot route: the proxy has no router, so no " + f"model list is available to choose from" + ) + need = summarize(data) + candidates = eligible( + llm_router.model_list, need, ROUTER_ALIAS, litellm.model_cost + ) + demand = ( + f"about {need['input_tokens']} input plus {need['max_output']} " + f"output tokens with vision={need['vision']} tools={need['tools']} " + f"schema={need['schema']} audio={need['audio']} " + f"documents={need['documents']}" + ) + if not candidates: + return f"'{ROUTER_ALIAS}' cannot route: no served model can take {demand}" + + reason = local_reason(data, llm_router.model_list, litellm.model_cost) + if reason: + candidates = [entry for entry in candidates if entry["_caps"]["local"]] + if not candidates: + return ( + f"'{ROUTER_ALIAS}' cannot route: {reason}, and no local model " + f"can take {demand}. A remote model could, so either shrink " + f"the request or set services.litellm.router.remote_fallback" + ) + text = request_text(data) + request_class = signature(need) + tally = await self._tally(request_class) + try: + data["model"] = decide(candidates, need, text, tally=tally) + except (OSError, ValueError) as error: + return ( + f"'{ROUTER_ALIAS}' cannot route: the {ROUTER_STRATEGY} strategy " + f"could not reach a decision ({type(error).__name__}: {error}). " + f"Answering from a substitute decider would make the alias mean " + f"something the caller did not ask for" + ) + if SAMPLE_ENABLED and len(candidates) > 1 and self._sampler.due(): + await self._compare(llm_router, data, candidates, text, request_class) + return data + + async def _tally(self, request_class): + """What earlier comparisons said, or nothing when they cannot be read.""" + store = self._history() + if store is None: + return {} + try: + return await store.wins(request_class) + except Exception as error: # noqa: BLE001 - history must never fail a route + print( + f"[router] routing history unreadable, deciding without it: " + f"{type(error).__name__}: {error}", + flush=True, + ) + return {} + + def _history(self): + """The store, or None when this proxy exposes no database to write to.""" + try: + from litellm.proxy.proxy_server import prisma_client + except ImportError: + return None + if prisma_client is None: + return None + if self._store is None: + self._store = History(prisma_client.db) + return self._store + + async def _compare(self, router, data, candidates, text, request_class): + """Answer this request from several routes and record which won. + + In ``serve`` mode the winning reply is handed back to the caller + through ``mock_response``, so the fan-out is the answer path and no + further call is made. In ``shadow`` mode the caller keeps the route + already decided and only the verdict is kept. + """ + try: + replies = await collect(router, data, candidates, SAMPLE_WIDTH) + if len(replies) < 2: + return + winner = judge_answers(text, replies) + except Exception as error: # noqa: BLE001 - a comparison must never fail a route + print( + f"[router] comparison abandoned, serving the decided route: " + f"{type(error).__name__}: {error}", + flush=True, + ) + return + store = self._history() + if store is not None: + try: + await store.record(request_class, winner, sorted(replies)) + except Exception as error: # noqa: BLE001 - see above + print( + f"[router] comparison not recorded: " + f"{type(error).__name__}: {error}", + flush=True, + ) + if SAMPLE_MODE == "serve": + data["model"] = winner + data["mock_response"] = replies[winner] + + +instance = SystemOneRouter() diff --git a/roles/svc-ai-litellm/templates/test.env.j2 b/roles/svc-ai-litellm/templates/test.env.j2 index a3a33b2cc6..d05518897a 100644 --- a/roles/svc-ai-litellm/templates/test.env.j2 +++ b/roles/svc-ai-litellm/templates/test.env.j2 @@ -1,15 +1,19 @@ {%- set _ollama_models = (lookup('config', 'svc-ai-ollama', 'services.ollama.preload_models') | map(attribute='alias') | list) if LITELLM_TEST_CONTRACT.ollama | bool else [] -%} {%- set _lmstudio_declared = lookup('config', 'svc-ai-lmstudio', 'services.lmstudio.preload_models') | map(attribute='alias') | list -%} {%- set _lmstudio_only = _lmstudio_declared | reject('in', _ollama_models) | list -%} -{%- set _remote = LITELLM_REMOTE_BACKENDS | selectattr('key') | map(attribute='alias') | list -%} +{%- set _remote = LITELLM_REMOTE_MODELS | selectattr('provider', 'in', LITELLM_SERVED_PROVIDERS) | map(attribute='alias') | list -%} {%- set _expected = _ollama_models + (_lmstudio_only if LITELLM_TEST_CONTRACT.lmstudio | bool else []) + _remote -%} +{%- set _windowed = ((lookup('config', 'svc-ai-ollama', 'services.ollama.preload_models') if LITELLM_TEST_CONTRACT.ollama | bool else []) + LITELLM_REMOTE_MODELS) | selectattr('context', 'defined') | items2dict(key_name='alias', value_name='context') -%} LITELLM_CONTAINER={{ CLI_LOCAL_CID }} LITELLM_PORT={{ LITELLM_PORT }} -LITELLM_MASTER_KEY_B64={{ LITELLM_MASTER_KEY | b64encode }} LITELLM_CHAT_MODEL={{ LITELLM_CHAT_MODEL }} LITELLM_CHAT_MODEL_SERVED={{ LITELLM_TEST_CONTRACT.served | bool | string | lower }} LITELLM_OLLAMA_ENABLED={{ LITELLM_TEST_CONTRACT.ollama | bool | string | lower }} LITELLM_LMSTUDIO_ENABLED={{ LITELLM_TEST_CONTRACT.lmstudio | bool | string | lower }} LITELLM_LMSTUDIO_ALIASES='{{ _lmstudio_only | to_json }}' +LITELLM_MOCK_ALIASES='{{ AI_MOCK_ALIASES | to_json }}' LITELLM_REMOTE_ALIASES='{{ _remote | to_json }}' LITELLM_EXPECTED_MODELS='{{ _expected | to_json }}' +LITELLM_ROUTER_ALIAS={{ LITELLM_ROUTER_ALIAS if _expected else '' }} +LITELLM_ROUTER_WINDOWS='{{ _windowed | to_json }}' +LITELLM_ROUTER_MIN_CHARS_PER_TOKEN={{ LITELLM_ROUTER_MIN_CHARS_PER_TOKEN | int }} diff --git a/roles/svc-ai-litellm/vars/main.yml b/roles/svc-ai-litellm/vars/main.yml index a2cbbf9cef..bab7ce1b8f 100644 --- a/roles/svc-ai-litellm/vars/main.yml +++ b/roles/svc-ai-litellm/vars/main.yml @@ -7,24 +7,38 @@ LITELLM_PORT: "{{ lookup('config', application_id, 'services.litellm.ports.inter LITELLM_MASTER_KEY: "{{ lookup('config', application_id, 'secrets.credentials.master_key') }}" LITELLM_MAX_OUTPUT_TOKENS: "{{ lookup('config', application_id, 'services.litellm.max_output_tokens') }}" LITELLM_UPSTREAM_TIMEOUT: "{{ lookup('config', application_id, 'services.litellm.upstream_timeout') }}" +LITELLM_ROUTER_ALIAS: "{{ lookup('config', application_id, 'services.litellm.router.alias') }}" +LITELLM_ROUTER_REMOTE_FALLBACK: "{{ lookup('config', application_id, 'services.litellm.router.remote_fallback') }}" +LITELLM_ROUTER_WEIGHTS: "{{ lookup('config', application_id, 'services.litellm.router.weights') }}" +LITELLM_ROUTER_STRATEGY: "{{ lookup('config', application_id, 'services.litellm.router.strategy') }}" +LITELLM_S1_URL: "{{ S1_BASE_LOCAL_URL }}" +LITELLM_S1_KEY: "{{ lookup('config', 'svc-ai-s1', 'secrets.credentials.api_key') }}" +LITELLM_S1_MODEL: "{{ lookup('config', 'svc-ai-s1', 'services.s1.model_alias') }}" +LITELLM_S1_TIMEOUT: "{{ lookup('config', 'svc-ai-s1', 'services.s1.request_timeout') }}" +LITELLM_ROUTER_MIN_CHARS_PER_TOKEN: "{{ lookup('config', application_id, 'services.litellm.router.min_chars_per_token') }}" +LITELLM_ROUTER_STATE_CHARS: "{{ lookup('config', application_id, 'services.litellm.router.state_chars') }}" +LITELLM_ROUTER_SPEED_SAMPLES: "{{ lookup('config', application_id, 'services.litellm.router.speed.samples') }}" +LITELLM_ROUTER_SPEED_TIMEOUT: "{{ lookup('config', application_id, 'services.litellm.router.speed.timeout') }}" +LITELLM_ROUTER_SPEED_REMEASURE: "{{ lookup('config', application_id, 'services.litellm.router.speed.remeasure') }}" +LITELLM_ROUTER_SAMPLE_ENABLED: "{{ lookup('config', application_id, 'services.litellm.router.sample.enabled') }}" +LITELLM_ROUTER_SAMPLE_MODE: "{{ lookup('config', application_id, 'services.litellm.router.sample.mode') }}" +LITELLM_ROUTER_SAMPLE_TRIGGER: "{{ lookup('config', application_id, 'services.litellm.router.sample.trigger') }}" +LITELLM_ROUTER_SAMPLE_INTERVAL: "{{ lookup('config', application_id, 'services.litellm.router.sample.interval') }}" +LITELLM_ROUTER_SAMPLE_WIDTH: "{{ lookup('config', application_id, 'services.litellm.router.sample.width') }}" +LITELLM_ROUTER_HOOK_PATH_HOST: "{{ [ lookup('container', application_id, 'directories.config'), 'router_hook.py' ] | path_join }}" +LITELLM_SPEED_PATH_HOST: "{{ [ lookup('container', application_id, 'directories.config'), 'measured_speed.json' ] | path_join }}" LITELLM_CONFIG_PATH_HOST: "{{ [ lookup('container', application_id, 'directories.config'), 'config.yaml' ] | path_join }}" LITELLM_CONFIG_PATH_DOCKER: "{{ lookup('volume', application_id, 'litellm_config').target }}" -LITELLM_OPENAI_API_KEY: "{{ lookup('config', application_id, 'secrets.credentials.openai_api_key') }}" -LITELLM_ANTHROPIC_API_KEY: "{{ lookup('config', application_id, 'secrets.credentials.anthropic_api_key') }}" -LITELLM_OPENROUTER_API_KEY: "{{ lookup('config', application_id, 'secrets.credentials.openrouter_api_key') }}" -LITELLM_REMOTE_BACKENDS: - - alias: openai/gpt-4o-mini - model: openai/gpt-4o-mini - key: "{{ LITELLM_OPENAI_API_KEY }}" - env: OPENAI_API_KEY - - alias: anthropic/claude-sonnet-4-5 - model: anthropic/claude-sonnet-4-5 - key: "{{ LITELLM_ANTHROPIC_API_KEY }}" - env: ANTHROPIC_API_KEY - - alias: openrouter/auto - model: openrouter/openrouter/auto - key: "{{ LITELLM_OPENROUTER_API_KEY }}" - env: OPENROUTER_API_KEY +LITELLM_PROVIDER_KEYS: + openai: "{{ lookup('config', application_id, 'secrets.credentials.openai_api_key') }}" + anthropic: "{{ lookup('config', application_id, 'secrets.credentials.anthropic_api_key') }}" + openrouter: "{{ lookup('config', application_id, 'secrets.credentials.openrouter_api_key') }}" +LITELLM_KEYED_PROVIDERS: "{{ LITELLM_PROVIDER_KEYS | dict2items | selectattr('value') | map(attribute='key') | list }}" + +LITELLM_SERVED_PROVIDERS: "{{ LITELLM_KEYED_PROVIDERS + [AI_MOCK_PROVIDER] }}" +LITELLM_UNRANKED_ALIASES: "{{ ([LITELLM_ROUTER_ALIAS] + (LITELLM_REMOTE_MODELS | selectattr('provider', 'equalto', AI_MOCK_PROVIDER) | map(attribute='alias') | list)) + | join(',') }}" +LITELLM_REMOTE_MODELS: "{{ lookup('config', application_id, 'services.litellm.remote_models') }}" LITELLM_UI_ROLE: "web-app-litellm" LITELLM_UI_SSO: "{{ (LITELLM_UI_ROLE in group_names) and (lookup('sso', LITELLM_UI_ROLE, 'is_oidc_native') | bool) }}" LITELLM_UI_BASE_URL: "{{ lookup('tls', LITELLM_UI_ROLE, 'url.base') }}" diff --git a/roles/svc-ai-lmstudio/README.md b/roles/svc-ai-lmstudio/README.md index aec4fade00..eda7aa9442 100644 --- a/roles/svc-ai-lmstudio/README.md +++ b/roles/svc-ai-lmstudio/README.md @@ -35,7 +35,7 @@ Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional ( - **OpenAI-compatible endpoint:** The headless server answers `/v1` requests on port 1234 inside the container network. - **CPU inference:** The role builds its own image from the pinned llmster release for the host's architecture, amd64 or arm64, keeps only the CPU llama.cpp engine and passes no GPU device into the container. - **Persistent model store:** A named volume mounted at `/root/.lmstudio` keeps downloaded models and server settings across redeploys. -- **Preloaded models:** Every entry of `services.lmstudio.preload_models` is downloaded on the hosting node with `lms get --gguf`, overlapped and reaped like the Ollama pre-pull. Each entry carries the `source` repository to download, the `name` LM Studio indexes that repository under and the gateway addresses it by, and the `alias` the gateway publishes it as. `source` is a full Hugging Face URL, never a search term: a search term resolves to whichever model the catalogue lists first. +- **Preloaded models:** Every entry of `services.lmstudio.preload_models` is downloaded on the hosting node with `lms get --gguf`, overlapped and reaped like the Ollama pre-pull. Each entry carries the `source` repository to download, the `quant` variant to take from it, the `name` LM Studio indexes that repository under and the gateway addresses it by, and the `alias` the gateway publishes it as. `source` is a full Hugging Face URL, never a search term: a search term resolves to whichever model the catalogue lists first. `quant` is appended as `@` and is required: without it LM Studio picks the variant itself, and the `file`, `sha256` and `bytes` the entry declares then describe a guess that the store check and the download budget are both measured against. - **Loading:** The pre-pull downloads without loading. LM Studio serves with just-in-time loading, so the first `/v1/chat/completions` naming a `name` loads that model; the gateway's warm task issues that first request at deploy time. - **Gateway backend:** The LiteLLM Gateway publishes those aliases, which are the names Ollama serves too, so the same model name routes to whichever local backend a deployment runs. - **Bounded resources:** The container is capped at 4 CPUs, 8 GB of memory and 2048 processes, and the role declares a minimum of 4 GB free storage for building the image; downloaded models grow the volume beyond that. diff --git a/roles/svc-ai-lmstudio/tasks/utils/preload/pull.yml b/roles/svc-ai-lmstudio/tasks/utils/preload/pull.yml index 162812fb1c..39ca8ad9d2 100644 --- a/roles/svc-ai-lmstudio/tasks/utils/preload/pull.yml +++ b/roles/svc-ai-lmstudio/tasks/utils/preload/pull.yml @@ -1,7 +1,7 @@ --- - name: ⬇️ Pre-pull LM Studio models ansible.builtin.shell: > # nocheck: container-exec-resolver runtime-resolved CID, delegated to the hosting node - container exec -i {{ LMSTUDIO_LOCAL_CID | quote }} lms get --yes --gguf {{ model.source | quote }} + container exec -i {{ LMSTUDIO_LOCAL_CID | quote }} lms get --yes --gguf {{ (model.source ~ '@' ~ model.quant) | quote }} environment: CONTAINER_EXEC_TIMEOUT: >- {{ [ LMSTUDIO_PRELOAD_FLOOR_SECONDS | int, diff --git a/roles/svc-ai-ollama/meta/variants.yml b/roles/svc-ai-ollama/meta/variants.yml index 6b9ad2d919..5cf1d87f14 100644 --- a/roles/svc-ai-ollama/meta/variants.yml +++ b/roles/svc-ai-ollama/meta/variants.yml @@ -6,6 +6,21 @@ # variant changed. Apps with fewer variants stay on variant 0 in the # extra rounds. # See docs/contributing/artefact/files/role/variants.md for the file format. +# +# Variant 0 runs the round that every other role's variant 0 runs with, so it +# preloads nothing: the gateway then serves its deterministic mock and no row +# pays for pulling or running a model. Variant 1 preloads for real and is the +# round that proves this role actually serves. +# +# Variant 1 serves three models whose declared windows differ, because that is +# what makes the gateway's router observable: a prompt larger than the smallest +# window can only be answered by a larger one, so the model that answers says +# which route was chosen. A declared window below what the tag supports is +# deliberate; it bounds the KV cache and it is the number the router filters on. +# +# Order is load-bearing. The CLI probe sizes its prompt off the smallest +# declared window, so whichever model holds that window is the one the request +# must exclude, and the model answering instead is what shows the filter ran. - services: container_backup: @@ -13,3 +28,18 @@ ollama: mem_reservation: 1g mem_limit: 2g + preload_models: [] + +- services: + container_backup: + enabled: true + ollama: + mem_reservation: 3g + mem_limit: 6g + preload_models: + - alias: qwen2.5:0.5b + context: 4096 + - alias: smollm2:135m + context: 8192 + - alias: llama3.2:1b + context: 65536 diff --git a/roles/svc-ai-s1/README.md b/roles/svc-ai-s1/README.md new file mode 100644 index 0000000000..d00f45548f --- /dev/null +++ b/roles/svc-ai-s1/README.md @@ -0,0 +1,96 @@ +# System One + +## Description + +Self-hosted System One decision service. It answers typed questions (`choice`, `score`, `noul`) over a non-autoregressive encoder instead of generating text, on the `POST /v1/systemone` contract. + +## Overview + +[`svc-ai-litellm`](../svc-ai-litellm/) publishes one routing alias. When this service is deployed, the gateway's pre-call hook asks it which of the eligible models should answer, passing the prompt as the state and the surviving aliases as the options of a single `choice` question. `services.litellm.router.strategy` follows `services.s1.enabled`, so deploying this role is what switches the gateway from a weighted score to a System One decision. + +`services.s1.flavor` selects which implementation answers that contract. `laya` runs [Laya](https://huggingface.co/convaiinnovations/laya), the default; `torch` and `onnx` run [jeff](https://github.com/logan-markewich/jeff) over a GLiFormer encoder, which is what the role carried before. Neither upstream ships a container image, so the role builds one and bakes the checkpoints into it. Baking keeps the deploy free of a download at container start, which is the failure mode a sibling role's model pull already hit. + +## Cosmos + +The diagram places System One in the Infinito.Nexus cosmos: the components it deploys (capabilities), the central services it consumes (dependencies), and its outward reach (federation and bridged external networks). + +```mermaid +flowchart LR + subgraph role [svc-ai-s1 🐳🐝] + svc_s1["s1"] + end + subgraph dependents [Dependents] + dpt_svc_ai_litellm["svc-ai-litellm 🐳🐝"] + end + svc_s1 -. "0..1" .-> dpt_svc_ai_litellm +``` + +Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. + +## Features + +- **One contract, three flavors.** `laya` serves multilingual checkpoints and preloads them at start; `torch` runs the GLiFormer checkpoint as published; `onnx` exports an int8 graph at build time and needs roughly a quarter of the memory. Every flavor answers the same `POST /v1/systemone`, so the gateway does not know which one it asked. +- **Typed answers, no generation.** A routing decision costs one forward pass over an encoder, not a completion. +- **Build-time model.** The checkpoint is a cached image layer rather than a download the first container start waits for. +- **Bearer authentication.** An empty key list would disable auth, so the role always renders one. + +## Settings + +| Setting | Meaning | +| --- | --- | +| `services.s1.flavor` | `laya`, `torch` or `onnx`; the role refuses anything else | +| `services.s1.model_alias` | alias the API accepts in a request's `model` field | +| `services.s1.request_timeout` | seconds the gateway waits for a routing answer | +| `services.s1.laya.checkpoints` | checkpoints the `laya` flavor bakes, each a `name` the server preloads and the `repo` it comes from | +| `services.s1.jeff.ref` | upstream commit the `torch` and `onnx` flavors build from; upstream publishes no tags | +| `services.s1.jeff.model_repo` | Hugging Face repository those flavors download at build time | +| `secrets.credentials.api_key` | bearer key the server accepts | + +## Quick Setup + +### Development + +Clone, set up the workstation, and deploy System One onto the local stack: + +```bash +git clone https://github.com/infinito-nexus/core.git +cd core +make onboard +make compose-deploy mode=reinstall apps=svc-ai-s1 full_cycle=false +``` + +### Production + +Run the published image to provision the inventory and deploy System One to a managed server (the mounted volume persists the inventory): + +```bash +APP=svc-ai-s1 +HOST= +DOMAIN= +TLS_MODE=self_signed +SSH_PUBLIC_KEY="" + +docker run --rm -it \ + -v "$PWD/inventories:/etc/infinito.nexus/inventories" \ + -e APP="$APP" -e HOST="$HOST" -e DOMAIN="$DOMAIN" -e TLS_MODE="$TLS_MODE" -e SSH_PUBLIC_KEY="$SSH_PUBLIC_KEY" \ + ghcr.io/infinito-nexus/core/debian bash -c ' + INVENTORY=/etc/infinito.nexus/inventories/production + infinito administration inventory provision "$INVENTORY" \ + --inventory-file "$INVENTORY/devices.yml" \ + --host "$HOST" \ + --include "$APP" \ + --vars "{\"TLS_MODE\": \"$TLS_MODE\", \"DOMAIN_PRIMARY\": \"$DOMAIN\", \"users\": {\"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}" && + infinito administration deploy dedicated "$INVENTORY/devices.yml" \ + --password-file "$INVENTORY/.password" \ + --diff -vv' +``` + +## Limits that shape the caller + +Both flavors cap the state and the option count and answer HTTP 422 above them, so the gateway truncates the prompt to `services.litellm.router.state_chars` before asking. The `laya` flavor shares a fixed token budget across the options of one question, so accuracy falls off well before its own cap; the deployments here put a handful of model aliases to it, far below where that matters. + +## Credits + +Implemented by **[Kevin Veen-Birkenbach](https://www.veen.world)**. +Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world). +Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license). diff --git a/roles/svc-ai-s1/TODO.md b/roles/svc-ai-s1/TODO.md new file mode 100644 index 0000000000..0de0db6af6 --- /dev/null +++ b/roles/svc-ai-s1/TODO.md @@ -0,0 +1,5 @@ +# TODO + +- Add a post-deploy assert that the GPU actually reached the container, once `feature/store-i18n` is merged. The role already selects the path by itself: `services.s1.gpu` states the intent and `tasks/00_core.yml` ANDs it with `/dev/nvidia0` and `/usr/bin/nvidia-container-runtime`, so a host without the toolkit builds the CPU image whatever the flag says. What is missing is the check that the device arrived, in the shape of `web-svc-libretranslate/tasks/utils/assert_gpu.yml`: ask the served container how many CUDA devices it counts rather than trusting the two stats. The `runtime: nvidia` line itself comes from `sys-svc-container/templates/resource.yml.j2` on that branch. Upstream measures `Router(preload=True)` at 32.8 ms on GPU against 193 to 464 ms on CPU. + +- Cap torch's intra-op threads on the CPU path. `meta/services.yml` limits the container to `cpus: "2.0"`, but torch sizes its thread pool from the host's core count, so a 16-core runner spawns 16 threads for 2 cores. laya reads `LAYA_THREADS` (`serve.py`, `_apply_thread_limit`) and upstream says to keep it at or below the physical core count; `templates/env.j2` does not set it. Costs latency, breaks nothing, so no test catches it. diff --git a/roles/svc-ai-s1/files/Dockerfile b/roles/svc-ai-s1/files/Dockerfile new file mode 100644 index 0000000000..8f38fa7060 --- /dev/null +++ b/roles/svc-ai-s1/files/Dockerfile @@ -0,0 +1,19 @@ +ARG S1_BASE_IMAGE +ARG S1_BASE_VERSION +FROM ${S1_BASE_IMAGE}:${S1_BASE_VERSION} + +ARG S1_FLAVOR +ARG S1_GPU +ARG S1_LAYA_CHECKPOINTS +ARG S1_JEFF_REPOSITORY +ARG S1_JEFF_REF +ARG S1_JEFF_MODEL_REPO +ARG S1_JEFF_MODEL_PATH + +COPY build.sh entrypoint.sh /usr/local/bin/ + +RUN bash /usr/local/bin/build.sh + +ENV S1_FLAVOR=${S1_FLAVOR} + +ENTRYPOINT ["bash", "/usr/local/bin/entrypoint.sh"] diff --git a/roles/svc-ai-s1/files/python/warm_model.py b/roles/svc-ai-s1/files/python/warm_model.py new file mode 100644 index 0000000000..071017ee84 --- /dev/null +++ b/roles/svc-ai-s1/files/python/warm_model.py @@ -0,0 +1,64 @@ +"""Put one question to the decider so the deploy pays its first inference. + +svc-ai-litellm's router hook allows one attempt of services.s1.request_timeout +and refuses to route on any error rather than substituting a decider, so a +consumer that arrives before the server answers gets a 400 while every later +one is served. Asking here moves that first answer into the deploy. + +The request is the contract both flavors serve, POST /v1/systemone, so the +same script warms whichever one the flavor selected. + +Environment (flavor-neutral, rendered by templates/env.j2): + S1_PORT: port the decider listens on inside its own container. + S1_MODEL_ALIAS: alias the request names; a flavor that resolves its own + checkpoint ignores the field rather than failing on it. + S1_API_KEY: the accepted key, read from the container's own environment + rather than passed in. +""" + +from __future__ import annotations + +import json +import os +import sys +import urllib.request + +PORT = os.environ["S1_PORT"] +MODEL = os.environ["S1_MODEL_ALIAS"] +KEY = os.environ["S1_API_KEY"] +QUESTION = "warmup" +TIMEOUT = 900 + +payload = json.dumps( + { + "state": "It rained all morning and the forecast promises more storms tonight.", + "model": MODEL, + "questions": { + QUESTION: { + "type": "choice", + "instructions": "Pick the topic this text belongs to.", + "criteria": { + "weather": "the text is about weather, rain, storms or a forecast", + "cooking": "the text is about food, recipes, ingredients or a kitchen", + }, + } + }, + } +).encode() + +request = urllib.request.Request( + f"http://127.0.0.1:{PORT}/v1/systemone", + data=payload, + headers={ + "Authorization": f"Bearer {KEY}", + "Content-Type": "application/json", + }, + method="POST", +) + +with urllib.request.urlopen(request, timeout=TIMEOUT) as response: # noqa: S310 loopback inside the container + body = json.loads(response.read()) + +if not (body.get("answers") or {}).get(QUESTION): + print(f"the decider answered without an answer to {QUESTION!r}", file=sys.stderr) + sys.exit(1) diff --git a/roles/svc-ai-s1/files/shell/build.sh b/roles/svc-ai-s1/files/shell/build.sh new file mode 100644 index 0000000000..9e81a283cf --- /dev/null +++ b/roles/svc-ai-s1/files/shell/build.sh @@ -0,0 +1,46 @@ +#!/usr/bin/env bash +# Build the System One server for one flavor. Every input is a Dockerfile ARG, +# which docker exposes to this RUN as an environment variable. +# +# Param: S1_FLAVOR laya | torch | onnx +# Param: S1_GPU 'true' when the deploy resolved a usable GPU +# Param: S1_LAYA_CHECKPOINTS comma list of HuggingFace repo ids (laya only) +# Param: S1_JEFF_REPOSITORY git url of the jeff server (torch, onnx) +# Param: S1_JEFF_REF commit the jeff install is pinned to +# Param: S1_JEFF_MODEL_REPO HuggingFace repo id of the jeff checkpoint +# Param: S1_JEFF_MODEL_PATH where that checkpoint lands in the image +set -euo pipefail + +rm -rf /var/lib/apt/lists/* +apt-get -o Acquire::Retries=3 update +apt-get -o Acquire::Retries=3 install -y --no-install-recommends git +rm -rf /var/lib/apt/lists/* + +if [ "${S1_GPU}" = "true" ]; then + pip install --no-cache-dir torch + laya_extras="serve,fast" +else + pip install --no-cache-dir --index-url https://download.pytorch.org/whl/cpu torch + laya_extras="serve" +fi + +if [ "${S1_FLAVOR}" = "laya" ]; then + pip install --no-cache-dir "laya[${laya_extras}]" huggingface_hub + IFS=',' read -ra checkpoints <<<"${S1_LAYA_CHECKPOINTS}" + for checkpoint in "${checkpoints[@]}"; do + hf download "${checkpoint}" + done + exit 0 +fi + +extra="" +if [ "${S1_FLAVOR}" = "onnx" ]; then + extra="[onnx]" +fi + +pip install --no-cache-dir "jeff${extra} @ git+${S1_JEFF_REPOSITORY}@${S1_JEFF_REF}" huggingface_hub +hf download "${S1_JEFF_MODEL_REPO}" --local-dir "${S1_JEFF_MODEL_PATH}" + +if [ "${S1_FLAVOR}" = "onnx" ]; then + python -c "from jeff.backends.onnx_export import export_encoder; export_encoder('${S1_JEFF_MODEL_PATH}', int8=True)" +fi diff --git a/roles/svc-ai-s1/files/shell/entrypoint.sh b/roles/svc-ai-s1/files/shell/entrypoint.sh new file mode 100644 index 0000000000..2adfa0290f --- /dev/null +++ b/roles/svc-ai-s1/files/shell/entrypoint.sh @@ -0,0 +1,10 @@ +#!/usr/bin/env bash +# Start whichever server the flavor built. S1_FLAVOR is baked as ENV because a +# Dockerfile ARG does not survive into the running container. +set -euo pipefail + +if [ "${S1_FLAVOR}" = "laya" ]; then + exec laya-serve +fi + +exec jeff diff --git a/roles/svc-ai-s1/files/test/.nocheck b/roles/svc-ai-s1/files/test/.nocheck new file mode 100644 index 0000000000..2b9d82d840 --- /dev/null +++ b/roles/svc-ai-s1/files/test/.nocheck @@ -0,0 +1 @@ +# nocheck: role-files-layout - Role test harness with test.sh as its entry point; test-e2e-cli stages files/test/ by that fixed path. diff --git a/roles/svc-ai-s1/files/test/probe.py b/roles/svc-ai-s1/files/test/probe.py new file mode 100644 index 0000000000..50f06af893 --- /dev/null +++ b/roles/svc-ai-s1/files/test/probe.py @@ -0,0 +1,269 @@ +"""Assert the System One server answers every question type on its input. + +Runs inside the server's container, where the API answers on loopback. Standard +library only: the image ships the server, not a test toolchain. + +Each type is asked twice with texts whose correct answer is unambiguous and +opposite, so a server that loads the model but does not read the state fails: +it either breaks the wire contract or returns the same answer to both. + +Env: + PORT http port the server listens on inside the container + MODEL alias the API accepts in a request's ``model`` field + S1_API_KEY the server's own key, read from its environment rather + than passed in, so no key reaches a command line +""" + +from __future__ import annotations + +import json +import os +import sys +import urllib.error +import urllib.request + +TIMEOUT = 120 + +CHOICE_OPTIONS = { + "weather": "the text is about weather, rain, storms or a forecast", + "cooking": "the text is about food, recipes, ingredients or a kitchen", +} +NOUL_CRITERIA = { + "true": "the text is about weather, rain, storms or a forecast", + "false": "the text is about food, recipes, ingredients or a kitchen", +} +SCORE_LEVELS = [ + "the text is negative and unhappy", + "the text is neutral", + "the text is positive and happy", +] +SCORE_MIDPOINT = (len(SCORE_LEVELS) - 1) / 2 +NOUL_MIN_SEPARATION = 0.2 + +RAIN = "It rained all morning and the forecast promises more storms tonight." +ONIONS = "Dice the onions, brown them in butter, then fold in the flour." +PRAISE = "Wonderful work, I am delighted with how well this turned out." +COMPLAINT = "Terrible work, I am furious about how badly this turned out." + + +def choice_question(name: str) -> dict: + return { + name: { + "type": "choice", + "instructions": "Pick the topic this text belongs to.", + "criteria": CHOICE_OPTIONS, + } + } + + +def noul_question(name: str) -> dict: + return { + name: { + "type": "noul", + "instructions": "Decide whether this text is about weather.", + "criteria": NOUL_CRITERIA, + } + } + + +def score_question(name: str) -> dict: + return { + name: { + "type": "score", + "instructions": "Rate how positive this text is.", + "criteria": SCORE_LEVELS, + } + } + + +def verify_choice(answer: dict, expected: str) -> tuple[str, str]: + """The observed value and a violation message, empty when the answer holds.""" + choice = answer.get("choice") + if choice not in CHOICE_OPTIONS: + return str(choice), ( + f"chose {choice!r}, which is not one of the options it was given " + f"({sorted(CHOICE_OPTIONS)})" + ) + if choice != expected: + return choice, f"chose {choice!r} where the text is plainly {expected!r}" + return choice, "" + + +def verify_noul(answer: dict, expected: str) -> tuple[str, str]: + """noul is P(true) for the criterion indexed first upstream. + + Only the shape is judged here. The server does not centre this probability + on 0.5: measured against a topic the encoder separates cleanly, a plainly + false text still answered 0.5908 where a true one answered 0.9507. The + ordering is what it carries, so the pair is judged in ``run_type``. + """ + raw = answer.get("noul") + if not isinstance(raw, (int, float)): + return str(raw), f"returned noul={raw!r}, which is not a number" + return f"{raw}", "" + + +def separated(observed: list[str], cases: tuple) -> str: + """Why the true text did not outscore the false one, or the empty string.""" + scores = dict(zip((expected for _state, expected in cases), observed, strict=True)) + spread = float(scores["true"]) - float(scores["false"]) + if spread < NOUL_MIN_SEPARATION: + return ( + f"answered {scores['true']} for the true text and {scores['false']} " + f"for the false one, a spread of {spread:.4f}; below " + f"{NOUL_MIN_SEPARATION} the two are not told apart" + ) + return "" + + +def verify_score(answer: dict, expected: str) -> tuple[str, str]: + """score is the expected level index over SCORE_LEVELS.""" + raw = answer.get("score") + if not isinstance(raw, (int, float)): + return str(raw), f"returned score={raw!r}, which is not a number" + wants_high = expected == "high" + if (raw > SCORE_MIDPOINT) != wants_high: + side = "above" if raw > SCORE_MIDPOINT else "below" + return f"{raw}", ( + f"returned score={raw}, {side} the {SCORE_MIDPOINT} midpoint, where " + f"the text is plainly {expected}" + ) + return f"{raw}", "" + + +PROBES = ( + ( + "choice", + choice_question, + verify_choice, + ((RAIN, "weather"), (ONIONS, "cooking")), + None, + ), + ( + "noul", + noul_question, + verify_noul, + ((RAIN, "true"), (ONIONS, "false")), + separated, + ), + ( + "score", + score_question, + verify_score, + ((PRAISE, "high"), (COMPLAINT, "low")), + None, + ), +) + + +def ask(base: str, key: str, state: str, questions: dict, model: str) -> dict: + """The answers block the server replied with. + + Args: + base: ``http://host:port`` of the server. + key: bearer key to authenticate with. + state: the text the server classifies. + questions: the ``questions`` mapping to send. + model: the model alias to address. + + Returns: + The ``answers`` mapping. + """ + payload = json.dumps({"state": state, "model": model, "questions": questions}) + request = urllib.request.Request( # noqa: S310 - literal http:// to loopback + base + "/v1/systemone", + data=payload.encode(), + headers={"Authorization": f"Bearer {key}", "Content-Type": "application/json"}, + ) + with urllib.request.urlopen( # noqa: S310 - literal http:// to loopback + request, timeout=TIMEOUT + ) as response: + return json.loads(response.read().decode() or "{}").get("answers") or {} + + +def run_type( + base: str, key: str, model: str, name, build, verify, cases, pair=None +) -> list[str]: + """Every violation this question type showed, as messages.""" + failures: list[str] = [] + observed: list[str] = [] + for state, expected in cases: + answers = ask(base, key, state, build(name), model) + if name not in answers: + failures.append( + f"{name}: the answer carries no {name!r}: {sorted(answers)}" + ) + continue + if answers[name].get("type") != name: + failures.append( + f"{name}: the answer declares type " + f"{answers[name].get('type')!r}, not {name!r}" + ) + continue + value, violation = verify(answers[name], expected) + observed.append(value) + if violation: + failures.append(f"{name}: {violation}") + else: + print(f"[OK] {name}: {expected} text -> {value}") + if len(observed) == len(cases) and len(set(observed)) == 1: + failures.append( + f"{name}: both texts were answered {observed[0]}; the server returns a " + f"constant rather than reading its input" + ) + if pair is not None and len(observed) == len(cases): + violation = pair(observed, cases) + if violation: + failures.append(f"{name}: {violation}") + return failures + + +def rejects_a_wrong_key(base: str, model: str) -> str: + """Why the server accepted a bad key, or the empty string. + + An empty ``S1_API_KEY`` disables authentication upstream, so a server + that answers an unknown key is open to anything that reaches its port. + """ + try: + ask(base, "not-the-configured-key", "anything", choice_question("probe"), model) + except urllib.error.HTTPError as error: + if error.code in (401, 403): + return "" + return f"a wrong bearer key was refused with HTTP {error.code}, not 401/403" + except (urllib.error.URLError, TimeoutError) as exc: + return f"the wrong-key probe could not reach the server: {exc}" + return "the server answered a request carrying a bearer key it never issued" + + +def main() -> int: + base = f"http://127.0.0.1:{os.environ['PORT']}" + keys = [k for k in os.environ["S1_API_KEY"].replace(",", " ").split() if k] + if not keys: + print( + "[FAIL] S1_API_KEY is empty, which disables authentication and " + "leaves the server open to anything that reaches its port", + file=sys.stderr, + ) + return 1 + key = keys[0] + model = os.environ["MODEL"] + + failures: list[str] = [] + for name, build, verify, cases, pair in PROBES: + failures.extend(run_type(base, key, model, name, build, verify, cases, pair)) + + violation = rejects_a_wrong_key(base, model) + if violation: + failures.append(violation) + else: + print("[OK] a bearer key the server never issued is refused") + + for failure in failures: + print(f"[FAIL] {failure}", file=sys.stderr) + if not failures: + print("[OK] every question type decides on its input") + return 1 if failures else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/roles/svc-ai-s1/files/test/test.sh b/roles/svc-ai-s1/files/test/test.sh new file mode 100644 index 0000000000..2318663382 --- /dev/null +++ b/roles/svc-ai-s1/files/test/test.sh @@ -0,0 +1,48 @@ +#!/usr/bin/env bash +# shellcheck shell=bash +# +# svc-ai-s1 System One test: runs probe.py inside the decider's container. +# +# Env (rendered into test.env from templates/test.env.j2): +# S1_CONTAINER resolved server container (CLI_LOCAL_CID) +# S1_PORT http port the server listens on inside the container +# S1_MODEL_ALIAS alias the API accepts in a request's model field +# READY_RETRIES readiness attempts (default 30) +# READY_SLEEP_SECONDS wait between attempts (default 10) + +set -uo pipefail + +here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +READY_RETRIES="${READY_RETRIES:-30}" +READY_SLEEP_SECONDS="${READY_SLEEP_SECONDS:-10}" + +[ -n "${S1_CONTAINER}" ] || { + echo "[FATAL] S1_CONTAINER unset; the server container was not resolved" >&2 + exit 2 +} + +probe() { + # nocheck: container-exec-resolver address resolved by the caller and passed in + container exec -i \ + -e "PORT=${S1_PORT}" \ + -e "MODEL=${S1_MODEL_ALIAS}" \ + "${S1_CONTAINER}" python3 - <"${here}/probe.py" +} + +attempt=1 +while [ "${attempt}" -le "${READY_RETRIES}" ]; do + output="$(probe 2>&1)" + rc=$? + case "${output}" in + *"[OK]"* | *"[FAIL]"*) + echo "${output}" + exit "${rc}" + ;; + esac + attempt=$((attempt + 1)) + [ "${attempt}" -le "${READY_RETRIES}" ] && sleep "${READY_SLEEP_SECONDS}" +done + +echo "[FAIL] the System One server never answered after ${READY_RETRIES} attempts" >&2 +echo "${output}" >&2 +exit 1 diff --git a/roles/svc-ai-s1/meta/info.yml b/roles/svc-ai-s1/meta/info.yml new file mode 100644 index 0000000000..4aac579f28 --- /dev/null +++ b/roles/svc-ai-s1/meta/info.yml @@ -0,0 +1,5 @@ +--- +logo: + class: fa-solid fa-signs-post +homepage: https://huggingface.co/convaiinnovations/laya +video: https://www.youtube.com/watch?v=UoRNo5LoDRE diff --git a/roles/svc-ai-s1/meta/main.yml b/roles/svc-ai-s1/meta/main.yml new file mode 100644 index 0000000000..9a8d04cfdd --- /dev/null +++ b/roles/svc-ai-s1/meta/main.yml @@ -0,0 +1,32 @@ +--- +galaxy_info: + author: Kevin Veen-Birkenbach + description: Self-hosted System One classifier that answers typed questions so the gateway can route without generating + license: Infinito.Nexus Community License (Non-Commercial) + company: | + Kevin Veen-Birkenbach + https://www.veen.world + galaxy_tags: + - ai + - routing + - classification + - jeff + issue_tracker_url: https://s.infinito.nexus/issues + min_ansible_version: "2.9" + platforms: + - name: ArchLinux + versions: + - all + - name: Debian + versions: + - all + - name: EL + versions: + - all + - name: Fedora + versions: + - all + - name: Ubuntu + versions: + - all +dependencies: [] diff --git a/roles/svc-ai-s1/meta/networks.yml b/roles/svc-ai-s1/meta/networks.yml new file mode 100644 index 0000000000..5a98220164 --- /dev/null +++ b/roles/svc-ai-s1/meta/networks.yml @@ -0,0 +1,6 @@ +--- +local: + subnet: 192.168.228.0/24 +overlay: + modes: [compose, swarm] + topology: shared_net diff --git a/roles/svc-ai-s1/meta/secrets.yml b/roles/svc-ai-s1/meta/secrets.yml new file mode 100644 index 0000000000..26ed49be30 --- /dev/null +++ b/roles/svc-ai-s1/meta/secrets.yml @@ -0,0 +1,7 @@ +--- +credentials: + api_key: + description: "Bearer key the jeff classifier accepts; the gateway's router hook is its only consumer" + algorithm: "alphanumeric" + validation: + min_length: 32 diff --git a/roles/svc-ai-s1/meta/services.yml b/roles/svc-ai-s1/meta/services.yml new file mode 100644 index 0000000000..b84a76c8f0 --- /dev/null +++ b/roles/svc-ai-s1/meta/services.yml @@ -0,0 +1,45 @@ +--- +# nocheck: playwright-service-flag - typed classification API without a browser surface +s1: + modes: + compose: + enabled: true + swarm: + enabled: true + bond: 1 + enabled: true + shared: true + # nocheck: dockerfile-custom upstream ships a pyproject and no image, so the role builds one + # nocheck: docker-version base of the self-built server, not the served application + image: python + version: "3.12.14-slim" + custom: true + name: s1 + model_alias: s1-latest + request_timeout: 20 + gpu: true + flavor: laya + laya: + checkpoints: + - name: english + repo: convaiinnovations/laya + - name: multilingual + repo: convaiinnovations/laya-multilingual + jeff: + repository: https://github.com/logan-markewich/jeff.git + # nocheck: semver-ref upstream publishes no tags, so the pin is a commit + ref: 34b32f99a727c47b679adde33f4702a001e02979 + model_repo: knowledgator/gliformer-large-v1 + healthcheck: + flavor: http_status + path: health + # nocheck: min-storage-headroom Reason: the declared image is the Dockerfile base; the built image adds the runtime and the checkpoints the flavor bakes + min_storage: 4GB + cpus: "2.0" + mem_reservation: 3g + mem_limit: 6g + pids_limit: 512 + ports: + internal: + http: 8000 + lifecycle: beta diff --git a/roles/svc-ai-s1/meta/tests.yml b/roles/svc-ai-s1/meta/tests.yml new file mode 100644 index 0000000000..4eff0defaa --- /dev/null +++ b/roles/svc-ai-s1/meta/tests.yml @@ -0,0 +1,4 @@ +--- +cli: + timeout: 900 + container_service_key: s1 diff --git a/roles/svc-ai-s1/meta/variants.yml b/roles/svc-ai-s1/meta/variants.yml new file mode 100644 index 0000000000..8c298b9535 --- /dev/null +++ b/roles/svc-ai-s1/meta/variants.yml @@ -0,0 +1,13 @@ +--- +# Matrix-deploy variant overrides for this role. +# See docs/contributing/artefact/files/role/variants.md for the file format. + +- services: + s1: + flavor: laya + +- services: + s1: + flavor: torch + healthcheck: + path: healthz diff --git a/roles/svc-ai-s1/tasks/00_core.yml b/roles/svc-ai-s1/tasks/00_core.yml new file mode 100644 index 0000000000..8644c6af76 --- /dev/null +++ b/roles/svc-ai-s1/tasks/00_core.yml @@ -0,0 +1,44 @@ +--- +- include_tasks: utils/once/flag.yml + +- name: "🛑 Refuse a backend flavor the role does not carry" + ansible.builtin.assert: + that: + - S1_FLAVOR in ['laya', 'torch', 'onnx'] + fail_msg: "services.s1.flavor must be 'laya', 'torch' or 'onnx' (got '{{ S1_FLAVOR }}')" + quiet: true + +- name: "🎮 Look for the GPU and the runtime that hands it to a container" + ansible.builtin.stat: + path: "{{ item }}" + register: s1_gpu_stat + loop: + - "{{ S1_NVIDIA_DEVICE }}" + - "{{ S1_NVIDIA_RUNTIME }}" + +- name: "🎮 Decide whether this deploy builds the CUDA image" + ansible.builtin.set_fact: + S1_GPU: "{{ (S1_GPU_REQUESTED | bool) and (s1_gpu_stat.results | map(attribute='stat.exists') | select | list | length == 2) }}" + +- name: "📦 Stage the System One build context" + ansible.builtin.include_tasks: + file: "{{ lookup('path_absolute', 'roles/sys-svc-compose/tasks/utils/render_replicated_templates.yml') }}" + vars: + role_templates: + - src: "{{ lookup('path_absolute', 'roles/svc-ai-s1/files/shell/build.sh') }}" + dest: "{{ [S1_BUILD_DIR, 'build.sh'] | path_join }}" + mode: "0644" + notify: [compose-build, compose-up] + - src: "{{ lookup('path_absolute', 'roles/svc-ai-s1/files/shell/entrypoint.sh') }}" + dest: "{{ [S1_BUILD_DIR, 'entrypoint.sh'] | path_join }}" + mode: "0644" + notify: [compose-build, compose-up] + +- name: "🔧 Setup container network for {{ application_id }}" + include_tasks: "{{ lookup('path_absolute', 'roles/sys-svc-compose/tasks/utils/network/routine.yml') }}" + vars: + compose_handlers_flush: true + +- name: "🔥 Manager-side model warm-up on the hosting node" + ansible.builtin.include_tasks: 01_manager_ops.yml + when: IS_STACK_HOST | bool diff --git a/roles/svc-ai-s1/tasks/01_manager_ops.yml b/roles/svc-ai-s1/tasks/01_manager_ops.yml new file mode 100644 index 0000000000..fd0fa9aded --- /dev/null +++ b/roles/svc-ai-s1/tasks/01_manager_ops.yml @@ -0,0 +1,17 @@ +--- +- name: "🔍 Resolve {{ application_id }} hosting node + container id" + ansible.builtin.include_tasks: "{{ lookup('path_absolute', 'roles/sys-svc-compose/tasks/utils/swarm/resolve_host_cid.yml') }}" + vars: + resolve_service_key: "{{ entity_name }}" + resolve_output_prefix: S1 + resolve_compose_cid: "{{ S1_CONTAINER }}" + +- name: "🔥 Warm the decider so the first route is not a cold load" + ansible.builtin.shell: | # nocheck: container-exec-resolver runtime-resolved CID, delegated to the hosting node + container exec -i {{ S1_LOCAL_CID | quote }} python3 -c {{ lookup('file', lookup('path_absolute', 'roles/svc-ai-s1/files/python/warm_model.py')) | quote }} + register: s1_warm_result + retries: 5 + delay: 10 + until: s1_warm_result.rc == 0 + changed_when: false + delegate_to: "{{ S1_HOST_NODE }}" diff --git a/roles/svc-ai-s1/tasks/main.yml b/roles/svc-ai-s1/tasks/main.yml new file mode 100644 index 0000000000..1869d11dd3 --- /dev/null +++ b/roles/svc-ai-s1/tasks/main.yml @@ -0,0 +1,5 @@ +--- +- include_tasks: 00_core.yml + when: run_once_svc_ai_s1 is not defined + vars: + application_id: svc-ai-s1 diff --git a/roles/svc-ai-s1/templates/compose.yml.j2 b/roles/svc-ai-s1/templates/compose.yml.j2 new file mode 100644 index 0000000000..aeea9ce585 --- /dev/null +++ b/roles/svc-ai-s1/templates/compose.yml.j2 @@ -0,0 +1,29 @@ +{% include 'roles/sys-svc-compose/templates/base.yml.j2' %} + + {% set service_name = 's1' %} + {{ service_name }}: + {{ lookup('container_healthcheck', service_name) | indent(4) }} + {% include 'roles/sys-svc-container/templates/base.yml.j2' %} + {{ lookup('container_image', application_id, 's1') }} + {{ lookup('compose_only', 'pull_policy', 'never') }} + build: + context: {{ S1_BUILD_DIR }} + dockerfile: Dockerfile + args: + S1_BASE_IMAGE: {{ S1_IMAGE }} + S1_BASE_VERSION: "{{ S1_VERSION }}" + S1_FLAVOR: "{{ S1_FLAVOR }}" + S1_GPU: "{{ S1_GPU | bool | lower }}" + S1_LAYA_CHECKPOINTS: "{{ S1_LAYA_REPOS }}" + S1_JEFF_REPOSITORY: "{{ S1_JEFF_REPOSITORY }}" + S1_JEFF_REF: "{{ S1_JEFF_REF }}" + S1_JEFF_MODEL_REPO: "{{ S1_JEFF_MODEL_REPO }}" + S1_JEFF_MODEL_PATH: "{{ S1_JEFF_MODEL_PATH }}" + {{ lookup('compose_only', 'container_name', S1_CONTAINER) }} + expose: + - "{{ S1_PORT }}" + {{ lookup('container_networks') | indent(4) }} + +{{ lookup('compose_networks') }} + +{{ lookup('compose_volumes') }} diff --git a/roles/svc-ai-s1/templates/env.j2 b/roles/svc-ai-s1/templates/env.j2 new file mode 100644 index 0000000000..fd1dd9a97d --- /dev/null +++ b/roles/svc-ai-s1/templates/env.j2 @@ -0,0 +1,24 @@ +{% include 'roles/sys-svc-container/templates/env.j2' %} + +S1_PORT={{ S1_PORT }} +S1_MODEL_ALIAS={{ S1_MODEL_ALIAS }} +S1_API_KEY={{ S1_API_KEY | dotenv_quote }} + +{% if S1_FLAVOR == 'laya' %} +LAYA_HOST=0.0.0.0 +LAYA_PORT={{ S1_PORT }} +LAYA_DEVICE={{ S1_LAYA_DEVICE }} +LAYA_PRELOAD=1 +LAYA_MODELS={{ S1_LAYA_CHECKPOINTS | map(attribute='name') | join(',') }} +LAYA_API_KEY={{ S1_API_KEY | dotenv_quote }} +{% else %} +JEFF_HOST=0.0.0.0 +JEFF_PORT={{ S1_PORT }} +JEFF_MODEL={{ S1_JEFF_MODEL_PATH }} +JEFF_MODEL_ALIASES={{ S1_MODEL_ALIAS }} +JEFF_BACKEND={{ S1_FLAVOR }} +{% if S1_FLAVOR == 'onnx' %} +JEFF_QUANT=int8 +{% endif %} +JEFF_API_KEYS={{ S1_API_KEY | dotenv_quote }} +{% endif %} diff --git a/roles/svc-ai-s1/templates/test.env.j2 b/roles/svc-ai-s1/templates/test.env.j2 new file mode 100644 index 0000000000..26634ed5bf --- /dev/null +++ b/roles/svc-ai-s1/templates/test.env.j2 @@ -0,0 +1,3 @@ +S1_CONTAINER={{ CLI_LOCAL_CID }} +S1_MODEL_ALIAS={{ S1_MODEL_ALIAS }} +S1_PORT={{ S1_PORT }} diff --git a/roles/svc-ai-s1/vars/main.yml b/roles/svc-ai-s1/vars/main.yml new file mode 100644 index 0000000000..8a59d8a1e7 --- /dev/null +++ b/roles/svc-ai-s1/vars/main.yml @@ -0,0 +1,27 @@ +--- +application_id: "svc-ai-s1" +entity_name: "{{ application_id | get_entity_name }}" + +S1_CONTAINER: "{{ lookup('config', application_id, 'services.s1.name') }}" +S1_PORT: "{{ lookup('config', application_id, 'services.s1.ports.internal.http') }}" +S1_IMAGE: "{{ lookup('config', application_id, 'services.s1.image') }}" +S1_VERSION: "{{ lookup('config', application_id, 'services.s1.version') }}" +S1_FLAVOR: "{{ lookup('config', application_id, 'services.s1.flavor') }}" +S1_API_KEY: "{{ lookup('config', application_id, 'secrets.credentials.api_key') }}" +S1_MODEL_ALIAS: "{{ lookup('config', application_id, 'services.s1.model_alias') }}" + +S1_GPU_REQUESTED: "{{ lookup('applications') | resource_filter(application_id, 'gpu', 's1', false) }}" +S1_NVIDIA_DEVICE: /dev/nvidia0 +S1_NVIDIA_RUNTIME: /usr/bin/nvidia-container-runtime +S1_LAYA_DEVICE: "{{ 'cuda' if S1_GPU | bool else 'cpu' }}" + +S1_BUILD_DIR: "{{ lookup('container', application_id, 'directories.instance') }}" +S1_MODEL_ROOT: "/opt/s1/models" + +S1_LAYA_CHECKPOINTS: "{{ lookup('config', application_id, 'services.s1.laya.checkpoints') }}" +S1_LAYA_REPOS: "{{ S1_LAYA_CHECKPOINTS | map(attribute='repo') | join(',') }}" + +S1_JEFF_REPOSITORY: "{{ lookup('config', application_id, 'services.s1.jeff.repository') }}" +S1_JEFF_REF: "{{ lookup('config', application_id, 'services.s1.jeff.ref') }}" +S1_JEFF_MODEL_REPO: "{{ lookup('config', application_id, 'services.s1.jeff.model_repo') }}" +S1_JEFF_MODEL_PATH: "{{ [S1_MODEL_ROOT, S1_JEFF_MODEL_REPO | basename] | path_join }}" diff --git a/roles/svc-ai-searxng/README.md b/roles/svc-ai-searxng/README.md new file mode 100644 index 0000000000..55336f477d --- /dev/null +++ b/roles/svc-ai-searxng/README.md @@ -0,0 +1,75 @@ +# Searxng + +## Description + +[SearXNG](https://docs.searxng.org/) is a metasearch engine. It forwards a query to other search engines and returns the merged result without an account and without a tracking profile. + +## Overview + +The role renders `settings.yml` from the deployment's own secret key and mounts it read-only. `formats` carries `json` next to `html`, because a machine consumer gets HTTP 403 on the JSON endpoint otherwise, and the rate limiter is off since the only clients are platform services on the container network. + +A consumer reaches it as `http://searxng:8080/search?q=` once it declares `searxng` in its own `meta/services.yml` with `enabled` and `shared` set. `web-app-openwebui` ships that flag and sends `WEB_SEARCH_ENGINE=searxng` with it. + +## Cosmos + +The diagram places Searxng in the Infinito.Nexus cosmos: the components it deploys (capabilities), the central services it consumes (dependencies), and its outward reach (federation and bridged external networks). + +```mermaid +flowchart LR + subgraph role [svc-ai-searxng 🐳🐝] + svc_searxng["searxng"] + end +``` + +Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. + +## Features + +- **JSON enabled:** the settings add the `json` output format, which upstream ships disabled and which every API client needs. +- **Internal only:** no published port, no proxy entry; the limiter is off because the clients are platform services. +- **Named state:** the configuration and cache directories the image declares as volumes are pinned to named volumes, so no anonymous volume is left behind. + +## Quick Setup + +### Development + +Clone, set up the workstation, and deploy Searxng onto the local stack: + +```bash +git clone https://github.com/infinito-nexus/core.git +cd core +make onboard +make compose-deploy mode=reinstall apps=svc-ai-searxng full_cycle=false +``` + +### Production + +Run the published image to provision the inventory and deploy Searxng to a managed server (the mounted volume persists the inventory): + +```bash +APP=svc-ai-searxng +HOST= +DOMAIN= +TLS_MODE=self_signed +SSH_PUBLIC_KEY="" + +docker run --rm -it \ + -v "$PWD/inventories:/etc/infinito.nexus/inventories" \ + -e APP="$APP" -e HOST="$HOST" -e DOMAIN="$DOMAIN" -e TLS_MODE="$TLS_MODE" -e SSH_PUBLIC_KEY="$SSH_PUBLIC_KEY" \ + ghcr.io/infinito-nexus/core/debian bash -c ' + INVENTORY=/etc/infinito.nexus/inventories/production + infinito administration inventory provision "$INVENTORY" \ + --inventory-file "$INVENTORY/devices.yml" \ + --host "$HOST" \ + --include "$APP" \ + --vars "{\"TLS_MODE\": \"$TLS_MODE\", \"DOMAIN_PRIMARY\": \"$DOMAIN\", \"users\": {\"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}" && + infinito administration deploy dedicated "$INVENTORY/devices.yml" \ + --password-file "$INVENTORY/.password" \ + --diff -vv' +``` + +## Credits + +Implemented by **[Kevin Veen-Birkenbach](https://www.veen.world)**. +Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world). +Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license). diff --git a/roles/svc-ai-searxng/meta/info.yml b/roles/svc-ai-searxng/meta/info.yml new file mode 100644 index 0000000000..955562178c --- /dev/null +++ b/roles/svc-ai-searxng/meta/info.yml @@ -0,0 +1,5 @@ +--- +# nocheck: info-media no official demo video +logo: + class: fas fa-magnifying-glass +homepage: https://docs.searxng.org/ diff --git a/roles/svc-ai-searxng/meta/main.yml b/roles/svc-ai-searxng/meta/main.yml new file mode 100644 index 0000000000..cb22410b61 --- /dev/null +++ b/roles/svc-ai-searxng/meta/main.yml @@ -0,0 +1,32 @@ +--- +galaxy_info: + author: Kevin Veen-Birkenbach + description: Metasearch engine the platform queries for web results without handing a user's query to a tracking search provider + license: Infinito.Nexus Community License (Non-Commercial) + company: | + Kevin Veen-Birkenbach + https://www.veen.world + galaxy_tags: + - search + - privacy + - ai + - searxng + issue_tracker_url: https://s.infinito.nexus/issues + min_ansible_version: "2.9" + platforms: + - name: ArchLinux + versions: + - all + - name: Debian + versions: + - all + - name: EL + versions: + - all + - name: Fedora + versions: + - all + - name: Ubuntu + versions: + - all +dependencies: [] diff --git a/roles/svc-ai-searxng/meta/networks.yml b/roles/svc-ai-searxng/meta/networks.yml new file mode 100644 index 0000000000..f5c82bfe27 --- /dev/null +++ b/roles/svc-ai-searxng/meta/networks.yml @@ -0,0 +1,6 @@ +--- +local: + subnet: 192.168.225.0/24 +overlay: + modes: [compose, swarm] + topology: shared_net diff --git a/roles/svc-ai-searxng/meta/secrets.yml b/roles/svc-ai-searxng/meta/secrets.yml new file mode 100644 index 0000000000..9d9277a85b --- /dev/null +++ b/roles/svc-ai-searxng/meta/secrets.yml @@ -0,0 +1,6 @@ +--- +credentials: + secret_key: + description: Secret the search engine signs its own session and preference cookies with. + algorithm: random_hex_32 + validation: non_empty_string diff --git a/roles/svc-ai-searxng/meta/services.yml b/roles/svc-ai-searxng/meta/services.yml new file mode 100644 index 0000000000..8d35af81bc --- /dev/null +++ b/roles/svc-ai-searxng/meta/services.yml @@ -0,0 +1,26 @@ +--- +# nocheck: playwright-service-flag - internal search backend consumed over the container network +searxng: + modes: + compose: + enabled: true + swarm: + enabled: true + bond: 1 + enabled: true + shared: true + image: searxng/searxng + version: 2026.9.22-2ed96e6fc + name: searxng + healthcheck: + flavor: http + path: healthz + min_storage: 500MB + cpus: "0.5" + mem_reservation: 128m + mem_limit: 512m + pids_limit: 256 + ports: + internal: + http: 8080 + lifecycle: alpha diff --git a/roles/svc-ai-searxng/meta/variants.yml b/roles/svc-ai-searxng/meta/variants.yml new file mode 100644 index 0000000000..25cf154307 --- /dev/null +++ b/roles/svc-ai-searxng/meta/variants.yml @@ -0,0 +1,2 @@ +--- +- services: {} diff --git a/roles/svc-ai-searxng/meta/volumes.yml b/roles/svc-ai-searxng/meta/volumes.yml new file mode 100644 index 0000000000..f5213f93ec --- /dev/null +++ b/roles/svc-ai-searxng/meta/volumes.yml @@ -0,0 +1,23 @@ +# nocheck: backup-service-flag both volumes exist only because the image declares them as VOLUME; the settings file is rendered by this role on every deploy and the cache is derived, so a restore would return nothing the deploy does not rebuild +--- +searxng_settings: + type: config + source: "{{ [ lookup('container', 'svc-ai-searxng', 'directories.config'), 'settings.yml' ] | path_join }}" + mode: "0444" + mounts: + - service: searxng + target: /etc/searxng/settings.yml +searxng_config: + type: volume + # nocheck: nfs-false-data-loss Reason: holds the image's own /etc/searxng copy, which docker seeds per node on first start; the deployment's settings.yml is bind-mounted over it, so nothing a replica writes here has to reach another replica + nfs: false + mounts: + - service: searxng + target: /etc/searxng +searxng_cache: + type: volume + # nocheck: nfs-false-data-loss Reason: derived search cache, rebuilt from the upstream engines on demand; shared over NFS every replica would write the same keys into one tree for no gain + nfs: false + mounts: + - service: searxng + target: /var/cache/searxng diff --git a/roles/svc-ai-searxng/tasks/00_core.yml b/roles/svc-ai-searxng/tasks/00_core.yml new file mode 100644 index 0000000000..3812b3c8d9 --- /dev/null +++ b/roles/svc-ai-searxng/tasks/00_core.yml @@ -0,0 +1,25 @@ +--- +- include_tasks: utils/once/flag.yml + +- include_tasks: utils/load/handlers.yml + vars: + handler_role_name: "sys-svc-compose" + +- name: "🥾 For '{{ application_id }}': bootstrap the compose host once" + ansible.builtin.include_role: + name: sys-svc-compose + tasks_from: 00_core.yml + when: run_once_sys_svc_compose is not defined + +- name: "📄 Render the search engine settings (local + swarm peers)" + ansible.builtin.include_tasks: + file: "{{ lookup('path_absolute', 'roles/sys-svc-compose/tasks/utils/render_replicated_templates.yml') }}" + vars: + role_templates: + - src: "settings.yml.j2" + dest: "{{ SEARXNG_SETTINGS_PATH_HOST }}" + +- name: "🔧 Setup container network for {{ application_id }}" + include_tasks: "{{ lookup('path_absolute', 'roles/sys-svc-compose/tasks/utils/network/routine.yml') }}" + vars: + compose_handlers_flush: true diff --git a/roles/svc-ai-searxng/tasks/main.yml b/roles/svc-ai-searxng/tasks/main.yml new file mode 100644 index 0000000000..8fee1308d7 --- /dev/null +++ b/roles/svc-ai-searxng/tasks/main.yml @@ -0,0 +1,5 @@ +--- +- include_tasks: 00_core.yml + when: run_once_svc_ai_searxng is not defined + vars: + application_id: svc-ai-searxng diff --git a/roles/svc-ai-searxng/templates/compose.yml.j2 b/roles/svc-ai-searxng/templates/compose.yml.j2 new file mode 100644 index 0000000000..285390465b --- /dev/null +++ b/roles/svc-ai-searxng/templates/compose.yml.j2 @@ -0,0 +1,16 @@ +{% include 'roles/sys-svc-compose/templates/base.yml.j2' %} + + {% set service_name = 'searxng' %} + {{ service_name }}: + {{ lookup('container_healthcheck', service_name) | indent(4) }} + {% include 'roles/sys-svc-container/templates/base.yml.j2' %} + {{ lookup('container_image', application_id, 'searxng') }} + {{ lookup('compose_only', 'container_name', SEARXNG_CONTAINER) }} + expose: + - "{{ SEARXNG_PORT }}" + {{ lookup('container_volumes', service_name) | indent(4) }} + {{ lookup('container_networks') | indent(4) }} + +{{ lookup('compose_networks') }} + +{{ lookup('compose_volumes') }} diff --git a/roles/svc-ai-searxng/templates/settings.yml.j2 b/roles/svc-ai-searxng/templates/settings.yml.j2 new file mode 100644 index 0000000000..0d18cc1420 --- /dev/null +++ b/roles/svc-ai-searxng/templates/settings.yml.j2 @@ -0,0 +1,15 @@ +use_default_settings: true + +server: + secret_key: {{ SEARXNG_SECRET_KEY | to_json }} + bind_address: "0.0.0.0" + port: {{ SEARXNG_PORT }} + limiter: false + image_proxy: true + +search: + safe_search: 0 + autocomplete: "" + formats: + - html + - json diff --git a/roles/svc-ai-searxng/vars/main.yml b/roles/svc-ai-searxng/vars/main.yml new file mode 100644 index 0000000000..1841e082ba --- /dev/null +++ b/roles/svc-ai-searxng/vars/main.yml @@ -0,0 +1,8 @@ +--- +application_id: "svc-ai-searxng" +entity_name: "{{ application_id | get_entity_name }}" + +SEARXNG_CONTAINER: "{{ lookup('config', application_id, 'services.searxng.name') }}" +SEARXNG_PORT: "{{ lookup('config', application_id, 'services.searxng.ports.internal.http') }}" +SEARXNG_SECRET_KEY: "{{ lookup('config', application_id, 'secrets.credentials.secret_key') }}" +SEARXNG_SETTINGS_PATH_HOST: "{{ [ lookup('container', application_id, 'directories.config'), 'settings.yml' ] | path_join }}" diff --git a/roles/svc-ai-tika/README.md b/roles/svc-ai-tika/README.md new file mode 100644 index 0000000000..155309b9bd --- /dev/null +++ b/roles/svc-ai-tika/README.md @@ -0,0 +1,75 @@ +# Tika + +## Description + +[Apache Tika](https://tika.apache.org/) turns an uploaded file into plain text and metadata. It reads PDF, the Office formats, mail, archives and images. + +## Overview + +The role runs the `-full` image, which carries the OCR and language packs, and serves it on port 9998 inside the container network only. No port is published and no proxy entry is created. + +A consumer reaches it as `http://tika:9998` once it declares `tika` in its own `meta/services.yml` with `enabled` and `shared` set, which is what attaches the consumer to this role's network. `web-app-openwebui` ships that flag and sends `CONTENT_EXTRACTION_ENGINE=tika` with it. + +## Cosmos + +The diagram places Tika in the Infinito.Nexus cosmos: the components it deploys (capabilities), the central services it consumes (dependencies), and its outward reach (federation and bridged external networks). + +```mermaid +flowchart LR + subgraph role [svc-ai-tika 🐳🐝] + svc_tika["tika"] + end +``` + +Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. + +## Features + +- **Internal only:** the server listens on the container network; nothing is published to a host port or a proxy. +- **Full extraction set:** the `-full` image ships the OCR and language detection dependencies, so scanned documents are extracted too. +- **Shared instance:** one container serves every consumer that carries the service flag. + +## Quick Setup + +### Development + +Clone, set up the workstation, and deploy Tika onto the local stack: + +```bash +git clone https://github.com/infinito-nexus/core.git +cd core +make onboard +make compose-deploy mode=reinstall apps=svc-ai-tika full_cycle=false +``` + +### Production + +Run the published image to provision the inventory and deploy Tika to a managed server (the mounted volume persists the inventory): + +```bash +APP=svc-ai-tika +HOST= +DOMAIN= +TLS_MODE=self_signed +SSH_PUBLIC_KEY="" + +docker run --rm -it \ + -v "$PWD/inventories:/etc/infinito.nexus/inventories" \ + -e APP="$APP" -e HOST="$HOST" -e DOMAIN="$DOMAIN" -e TLS_MODE="$TLS_MODE" -e SSH_PUBLIC_KEY="$SSH_PUBLIC_KEY" \ + ghcr.io/infinito-nexus/core/debian bash -c ' + INVENTORY=/etc/infinito.nexus/inventories/production + infinito administration inventory provision "$INVENTORY" \ + --inventory-file "$INVENTORY/devices.yml" \ + --host "$HOST" \ + --include "$APP" \ + --vars "{\"TLS_MODE\": \"$TLS_MODE\", \"DOMAIN_PRIMARY\": \"$DOMAIN\", \"users\": {\"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}" && + infinito administration deploy dedicated "$INVENTORY/devices.yml" \ + --password-file "$INVENTORY/.password" \ + --diff -vv' +``` + +## Credits + +Implemented by **[Kevin Veen-Birkenbach](https://www.veen.world)**. +Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world). +Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license). diff --git a/roles/svc-ai-tika/meta/info.yml b/roles/svc-ai-tika/meta/info.yml new file mode 100644 index 0000000000..b9c95ab14b --- /dev/null +++ b/roles/svc-ai-tika/meta/info.yml @@ -0,0 +1,5 @@ +--- +# nocheck: info-media no official demo video +logo: + class: fas fa-file-lines +homepage: https://tika.apache.org/ diff --git a/roles/svc-ai-tika/meta/main.yml b/roles/svc-ai-tika/meta/main.yml new file mode 100644 index 0000000000..0f483682c3 --- /dev/null +++ b/roles/svc-ai-tika/meta/main.yml @@ -0,0 +1,32 @@ +--- +galaxy_info: + author: Kevin Veen-Birkenbach + description: Central document extraction engine that turns uploaded files into text for retrieval-augmented chat + license: Infinito.Nexus Community License (Non-Commercial) + company: | + Kevin Veen-Birkenbach + https://www.veen.world + galaxy_tags: + - ai + - rag + - documents + - tika + issue_tracker_url: https://s.infinito.nexus/issues + min_ansible_version: "2.9" + platforms: + - name: ArchLinux + versions: + - all + - name: Debian + versions: + - all + - name: EL + versions: + - all + - name: Fedora + versions: + - all + - name: Ubuntu + versions: + - all +dependencies: [] diff --git a/roles/svc-ai-tika/meta/networks.yml b/roles/svc-ai-tika/meta/networks.yml new file mode 100644 index 0000000000..caee4c75a6 --- /dev/null +++ b/roles/svc-ai-tika/meta/networks.yml @@ -0,0 +1,6 @@ +--- +local: + subnet: 192.168.224.0/24 +overlay: + modes: [compose, swarm] + topology: shared_net diff --git a/roles/svc-ai-tika/meta/services.yml b/roles/svc-ai-tika/meta/services.yml new file mode 100644 index 0000000000..279d3b46d1 --- /dev/null +++ b/roles/svc-ai-tika/meta/services.yml @@ -0,0 +1,26 @@ +--- +# nocheck: playwright-service-flag - internal extraction engine without a browser surface +tika: + modes: + compose: + enabled: true + swarm: + enabled: true + bond: 1 + enabled: true + shared: true + image: apache/tika + version: 4.0.0-full + name: tika + healthcheck: + flavor: http_status + path: tika + min_storage: 1GB + cpus: "1.0" + mem_reservation: 512m + mem_limit: 2g + pids_limit: 512 + ports: + internal: + http: 9998 + lifecycle: alpha diff --git a/roles/svc-ai-tika/meta/variants.yml b/roles/svc-ai-tika/meta/variants.yml new file mode 100644 index 0000000000..25cf154307 --- /dev/null +++ b/roles/svc-ai-tika/meta/variants.yml @@ -0,0 +1,2 @@ +--- +- services: {} diff --git a/roles/svc-ai-tika/tasks/00_core.yml b/roles/svc-ai-tika/tasks/00_core.yml new file mode 100644 index 0000000000..bd183d0e32 --- /dev/null +++ b/roles/svc-ai-tika/tasks/00_core.yml @@ -0,0 +1,7 @@ +--- +- include_tasks: utils/once/flag.yml + +- name: "🔧 Setup container network for {{ application_id }}" + include_tasks: "{{ lookup('path_absolute', 'roles/sys-svc-compose/tasks/utils/network/routine.yml') }}" + vars: + compose_handlers_flush: true diff --git a/roles/svc-ai-tika/tasks/main.yml b/roles/svc-ai-tika/tasks/main.yml new file mode 100644 index 0000000000..80ce7b670d --- /dev/null +++ b/roles/svc-ai-tika/tasks/main.yml @@ -0,0 +1,5 @@ +--- +- include_tasks: 00_core.yml + when: run_once_svc_ai_tika is not defined + vars: + application_id: svc-ai-tika diff --git a/roles/svc-ai-tika/templates/compose.yml.j2 b/roles/svc-ai-tika/templates/compose.yml.j2 new file mode 100644 index 0000000000..f56a92c406 --- /dev/null +++ b/roles/svc-ai-tika/templates/compose.yml.j2 @@ -0,0 +1,15 @@ +{% include 'roles/sys-svc-compose/templates/base.yml.j2' %} + + {% set service_name = 'tika' %} + {{ service_name }}: + {{ lookup('container_healthcheck', service_name) | indent(4) }} + {% include 'roles/sys-svc-container/templates/base.yml.j2' %} + {{ lookup('container_image', application_id, 'tika') }} + {{ lookup('compose_only', 'container_name', TIKA_CONTAINER) }} + expose: + - "{{ TIKA_PORT }}" + {{ lookup('container_networks') | indent(4) }} + +{{ lookup('compose_networks') }} + +{{ lookup('compose_volumes') }} diff --git a/roles/svc-ai-tika/vars/main.yml b/roles/svc-ai-tika/vars/main.yml new file mode 100644 index 0000000000..d6d0a05091 --- /dev/null +++ b/roles/svc-ai-tika/vars/main.yml @@ -0,0 +1,6 @@ +--- +application_id: "svc-ai-tika" +entity_name: "{{ application_id | get_entity_name }}" + +TIKA_CONTAINER: "{{ lookup('config', application_id, 'services.tika.name') }}" +TIKA_PORT: "{{ lookup('config', application_id, 'services.tika.ports.internal.http') }}" diff --git a/roles/svc-db-qdrant/README.md b/roles/svc-db-qdrant/README.md index 63109f97f9..1b0c58e735 100644 --- a/roles/svc-db-qdrant/README.md +++ b/roles/svc-db-qdrant/README.md @@ -38,6 +38,7 @@ flowchart LR end subgraph dependents [Dependents] dpt_web_app_flowise["web-app-flowise 🐳🐝"] + dpt_web_app_openwebui["web-app-openwebui 🐳🐝"] end dep_svc_bkp_volume_2_local -. "0..1" .-> svc_container_backup dep_svc_net_tor -. "0..1" .-> svc_tor @@ -46,6 +47,7 @@ flowchart LR dep_web_app_openclaw -. "0..1" .-> svc_openclaw dep_web_app_openwebui -. "0..1" .-> svc_openwebui svc_qdrant -. "0..1" .-> dpt_web_app_flowise + svc_qdrant -. "0..1" .-> dpt_web_app_openwebui ``` Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. diff --git a/roles/svc-net-tor/files/test/onion_ports.py b/roles/svc-net-tor/files/test/onion_ports.py index adb6a81c66..2312c60f80 100755 --- a/roles/svc-net-tor/files/test/onion_ports.py +++ b/roles/svc-net-tor/files/test/onion_ports.py @@ -23,7 +23,7 @@ carrying a test toolchain. Env (rendered into test.env from templates/test.env.j2): TOR_SOCKS SOCKS proxy, host:port ONION_HOST the node onion host - ONION_PORTS space-separated forwarded ports; empty means there is + ONION_PORTS comma-separated forwarded ports; empty means there is nothing to check ONION_PORT_TIMEOUT seconds per connect attempt. 60 for the same reason the domain probe uses it: Tor's SocksTimeout default is 120s @@ -150,7 +150,11 @@ def main() -> int: timeout = float( _required("ONION_PORT_TIMEOUT", "seconds per connect, flavor-dependent") ) - ports = [int(entry) for entry in os.environ.get("ONION_PORTS", "").split()] + ports = [ + int(entry) + for entry in os.environ.get("ONION_PORTS", "").split(",") + if entry.strip() + ] if not ports: print("[INFO] no onion-forwarded ports declared; nothing to probe") diff --git a/roles/svc-net-tor/templates/test.env.j2 b/roles/svc-net-tor/templates/test.env.j2 index 6e0bec9054..04b03a6fc5 100644 --- a/roles/svc-net-tor/templates/test.env.j2 +++ b/roles/svc-net-tor/templates/test.env.j2 @@ -1,6 +1,6 @@ NGINX_SERVERS_DIR={{ lookup('nginx', 'directories.configuration.servers') }} ONION_HOST={{ lookup('config', 'svc-net-tor', 'services.tor.node') }} -ONION_PORTS={{ lookup('tor_ports') | map(attribute='onion_port') | unique | sort | join(' ') | dotenv_quote }} +ONION_PORTS={{ lookup('tor_ports') | map(attribute='onion_port') | unique | sort | join(',') | dotenv_quote }} ONION_PORT_RETRIES=2 ONION_PORT_TIMEOUT=60 RETRIES=5 diff --git a/roles/sys-svc-compose/tasks/00_core.yml b/roles/sys-svc-compose/tasks/00_core.yml index 9e53702465..a1ee2c33e2 100644 --- a/roles/sys-svc-compose/tasks/00_core.yml +++ b/roles/sys-svc-compose/tasks/00_core.yml @@ -1,6 +1,19 @@ --- - include_tasks: utils/once/flag.yml +- name: "✅ Ensure compose wrapper system deps (PyYAML)" + package_install: + id: python-yaml + +- name: 📦 Install compose wrapper + become: true + ansible.builtin.copy: + src: "python/compose.py" + dest: "{{ BIN_COMPOSE }}" + mode: "0755" + owner: root + group: root + - include_role: name: sys-svc-container public: false @@ -23,19 +36,6 @@ owner: root group: root -- name: "✅ Ensure compose wrapper system deps (PyYAML)" - package_install: - id: python-yaml - -- name: 📦 Install compose wrapper - become: true - ansible.builtin.copy: - src: "python/compose.py" - dest: "{{ BIN_COMPOSE }}" - mode: "0755" - owner: root - group: root - - name: 📦 Install resolve-container-id helper (Swarm task → container ID) become: true ansible.builtin.copy: diff --git a/roles/sys-svc-container/files/shell/install-cli.sh b/roles/sys-svc-container/files/shell/install-cli.sh index 419bd08247..17e9db97b2 100755 --- a/roles/sys-svc-container/files/shell/install-cli.sh +++ b/roles/sys-svc-container/files/shell/install-cli.sh @@ -14,6 +14,16 @@ echo ">>> Installing docker client on ID=${ID} ID_LIKE=${ID_LIKE:-}" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" DOCKER_APT_SANITIZER="${SCRIPT_DIR}/apt_repo.sh" +apt_install_refreshing_on_miss() { + if apt-get install -y --no-install-recommends "$@"; then + return 0 + fi + echo "[docker-cli] install missed the pool, discarding the index and retrying" >&2 + rm -rf /var/lib/apt/lists/* + apt-get update + apt-get install -y --no-install-recommends "$@" +} + sanitize_docker_apt_sources() { local distro_id="$1" local keep_canonical="${2:-1}" @@ -72,7 +82,7 @@ EOF fi apt-get update - apt-get install -y --no-install-recommends \ + apt_install_refreshing_on_miss \ ca-certificates \ curl \ gnupg \ @@ -84,7 +94,7 @@ EOF > /etc/apt/sources.list.d/docker.list apt-get update if [[ "${REPO_ONLY}" != "1" ]]; then - apt-get install -y --no-install-recommends \ + apt_install_refreshing_on_miss \ docker-buildx-plugin \ docker-ce-cli \ docker-compose-plugin diff --git a/roles/test-e2e-cli/files/shared/gateway/probe.py b/roles/test-e2e-cli/files/shared/gateway/probe.py new file mode 100644 index 0000000000..c02fdfce5c --- /dev/null +++ b/roles/test-e2e-cli/files/shared/gateway/probe.py @@ -0,0 +1,83 @@ +"""Check that a workstation agent's configured gateway actually answers it. + +Env (rendered into test.env from the staging role's templates/test.env.j2): + GATEWAY_CONFIG_PATH the config file the role writes + GATEWAY_URL the OpenAI-compatible base URL the agent was pointed at + GATEWAY_KEY the key the agent presents + GATEWAY_MODEL the alias the agent asks for +""" + +from __future__ import annotations + +import json +import os +import sys +import urllib.error +import urllib.request +from pathlib import Path + +TIMEOUT = 300 +PROMPT = "Reply with the single word pong." + + +def failures() -> list[str]: + config_path = Path(os.environ["GATEWAY_CONFIG_PATH"]) + url = os.environ["GATEWAY_URL"].rstrip("/") + model = os.environ["GATEWAY_MODEL"] + found: list[str] = [] + + if not config_path.is_file(): + return [ + f"{config_path} does not exist, so the agent was never pointed anywhere" + ] + + written = config_path.read_text(encoding="utf-8") + if url not in written: + found.append( + f"{config_path} does not name {url}, so the agent would reach the vendor API" + ) + + payload = json.dumps( + { + "model": model, + "messages": [{"role": "user", "content": PROMPT}], + "max_tokens": 16, + } + ).encode() + request = urllib.request.Request( # noqa: S310 the URL is the loopback gateway this deployment published + f"{url}/chat/completions", + data=payload, + headers={ + "Authorization": f"Bearer {os.environ['GATEWAY_KEY']}", + "Content-Type": "application/json", + }, + method="POST", + ) + try: + with urllib.request.urlopen(request, timeout=TIMEOUT) as response: # noqa: S310 loopback gateway + body = json.loads(response.read()) + except urllib.error.HTTPError as error: + return [*found, f"the gateway refused the agent's key with HTTP {error.code}"] + except (urllib.error.URLError, TimeoutError) as error: + return [*found, f"the gateway at {url} could not be reached: {error}"] + + choices = body.get("choices") or [{}] + content = ((choices[0].get("message") or {}).get("content") or "").strip() + if not content: + found.append(f"{model} answered through the gateway without any content") + else: + print(f"[OK] {model} answered through {url}: {content[:40]}") + return found + + +def main() -> int: + found = failures() + for failure in found: + print(f"[FAIL] {failure}", file=sys.stderr) + if not found: + print("[OK] the agent reaches the gateway and gets an answer") + return 1 if found else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/roles/test-e2e-cli/files/shared/gateway/run.sh b/roles/test-e2e-cli/files/shared/gateway/run.sh new file mode 100644 index 0000000000..6680b01c62 --- /dev/null +++ b/roles/test-e2e-cli/files/shared/gateway/run.sh @@ -0,0 +1,21 @@ +#!/usr/bin/env bash +# Env (rendered into test.env from the staging role's templates/test.env.j2): +# GATEWAY_CONFIG_PATH config file the role writes +# GATEWAY_URL OpenAI-compatible base URL the agent was pointed at +# GATEWAY_KEY key the agent presents +# GATEWAY_MODEL alias the agent asks for +set -euo pipefail + +: "${GATEWAY_CONFIG_PATH?}" +: "${GATEWAY_URL?}" +: "${GATEWAY_KEY?}" +: "${GATEWAY_MODEL?}" + +here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +if [ -z "${GATEWAY_URL}" ]; then + echo "SKIP: this workstation was not pointed at a gateway by the deployment" + exit 0 +fi + +python3 - <"${here}/probe.py" diff --git a/roles/test-e2e-playwright/files/service-gating.js b/roles/test-e2e-playwright/files/service-gating.js index 3e03319193..34182af7eb 100644 --- a/roles/test-e2e-playwright/files/service-gating.js +++ b/roles/test-e2e-playwright/files/service-gating.js @@ -76,7 +76,7 @@ function assertKnownService(name) { // enabled and let individual lookups fall through to the default. return; } - const slug = name.toLowerCase(); + const slug = envKey(name).slice(0, -SUFFIX.length).toLowerCase(); if (!registered.has(slug)) { throw new Error( `service-gating: Unknown service "${name}". ` + diff --git a/roles/web-app-dashboard/README.md b/roles/web-app-dashboard/README.md index 5ff3931638..93ebc289ee 100644 --- a/roles/web-app-dashboard/README.md +++ b/roles/web-app-dashboard/README.md @@ -20,7 +20,7 @@ flowchart LR dep_web_app_matomo["web-app-matomo 🐳🐝"] dep_web_app_prometheus["web-app-prometheus 🐳🐝"] dep_web_svc_asset["web-svc-asset 💻"] - dep_web_svc_cdn["web-svc-cdn 🐳🐝"] + dep_web_svc_cdn["web-svc-cdn 🐳🐝💻"] dep_web_svc_css["web-svc-css 💻"] dep_web_svc_logout["web-svc-logout 🐳🐝"] dep_web_svc_simpleicons["web-svc-simpleicons 🐳🐝"] diff --git a/roles/web-app-hermes/README.md b/roles/web-app-hermes/README.md index 9ff7363803..514996d4a0 100644 --- a/roles/web-app-hermes/README.md +++ b/roles/web-app-hermes/README.md @@ -39,6 +39,7 @@ flowchart LR svc_container_backup["container_backup"] end subgraph dependents [Dependents] + dpt_svc_ai_agent_broker["svc-ai-agent-broker 🐳🐝"] dpt_svc_ai_robot["svc-ai-robot 💻"] dpt_svc_db_qdrant["svc-db-qdrant 🐳🐝"] dpt_web_app_baserow["web-app-baserow 🐳🐝"] @@ -50,7 +51,6 @@ flowchart LR dpt_web_app_jellyfin["web-app-jellyfin 🐳🐝"] dpt_web_app_jenkins["web-app-jenkins 🐳🐝"] dpt_web_app_listmonk["web-app-listmonk 🐳🐝"] - dpt_web_app_mattermost["web-app-mattermost 🐳🐝"] dpt_more["..."] end dep_svc_ai_litellm -. "0..1" .-> svc_litellm @@ -63,6 +63,7 @@ flowchart LR dep_web_app_prometheus -. "0..1" .-> svc_prometheus dep_web_svc_css -. "0..1" .-> svc_css svc_hermes -- "1:1" --> dpt_more + svc_hermes -- "1:1" --> dpt_svc_ai_agent_broker svc_hermes -. "0..1" .-> dpt_svc_ai_robot svc_hermes -. "0..1" .-> dpt_svc_db_qdrant svc_hermes -. "0..1" .-> dpt_web_app_baserow @@ -74,7 +75,6 @@ flowchart LR svc_hermes -. "0..1" .-> dpt_web_app_jellyfin svc_hermes -. "0..1" .-> dpt_web_app_jenkins svc_hermes -. "0..1" .-> dpt_web_app_listmonk - svc_hermes -. "0..1" .-> dpt_web_app_mattermost ``` Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. diff --git a/roles/web-app-hermes/meta/rbac.yml b/roles/web-app-hermes/meta/rbac.yml index 70019a9c87..7192b3b419 100644 --- a/roles/web-app-hermes/meta/rbac.yml +++ b/roles/web-app-hermes/meta/rbac.yml @@ -2,3 +2,5 @@ roles: mcp: description: May drive this agent's MCP clients, which reach every admitted provider with the credentials the deployment issued rather than the caller's own + agent-user: + description: May start a personal Hermes agent from Open WebUI through svc-ai-agent-broker diff --git a/roles/web-app-hermes/meta/services.yml b/roles/web-app-hermes/meta/services.yml index eeb8442dc1..d83dd2cc59 100644 --- a/roles/web-app-hermes/meta/services.yml +++ b/roles/web-app-hermes/meta/services.yml @@ -17,6 +17,7 @@ hermes: image: nousresearch/hermes-agent version: "v2026.9.7" name: hermes + min_context: 64000 min_storage: 1146MB cpus: "2.0" mem_reservation: 2g diff --git a/roles/web-app-keycloak/README.md b/roles/web-app-keycloak/README.md index 774e16642c..1dc259afcf 100644 --- a/roles/web-app-keycloak/README.md +++ b/roles/web-app-keycloak/README.md @@ -41,6 +41,7 @@ flowchart LR svc_tor["tor"] end subgraph dependents [Dependents] + dpt_svc_ai_agent_broker["svc-ai-agent-broker 🐳🐝"] dpt_web_app_akaunting["web-app-akaunting 🐳🐝"] dpt_web_app_baserow["web-app-baserow 🐳🐝"] dpt_web_app_bigbluebutton["web-app-bigbluebutton 🐳🐝"] @@ -52,7 +53,6 @@ flowchart LR dpt_web_app_confluence["web-app-confluence 🐳🐝"] dpt_web_app_dashboard["web-app-dashboard 🐳🐝"] dpt_web_app_decidim["web-app-decidim 🐳🐝"] - dpt_web_app_discourse["web-app-discourse 🐳🐝"] dpt_more["..."] end dep_svc_db_openldap -. "0..1" .-> svc_ldap @@ -65,6 +65,7 @@ flowchart LR dep_web_svc_css -. "0..1" .-> svc_css dep_web_svc_logout -. "0..1" .-> svc_javascript svc_keycloak -- "1:1" --> dpt_more + svc_keycloak -- "1:1" --> dpt_svc_ai_agent_broker svc_keycloak -. "0..1" .-> dpt_web_app_akaunting svc_keycloak -. "0..1" .-> dpt_web_app_baserow svc_keycloak -. "0..1" .-> dpt_web_app_bigbluebutton @@ -76,8 +77,7 @@ flowchart LR svc_keycloak -- "0..0" --> dpt_web_app_confluence svc_keycloak -. "0..1" .-> dpt_web_app_dashboard svc_keycloak -. "0..1" .-> dpt_web_app_decidim - svc_keycloak -. "0..1" .-> dpt_web_app_discourse - linkStyle 15,17,18 stroke:red; + linkStyle 16,18,19 stroke:red; ``` Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. diff --git a/roles/web-app-openclaw/README.md b/roles/web-app-openclaw/README.md index 8c2301a9ed..60d1d8d096 100644 --- a/roles/web-app-openclaw/README.md +++ b/roles/web-app-openclaw/README.md @@ -40,6 +40,7 @@ flowchart LR svc_container_backup["container_backup"] end subgraph dependents [Dependents] + dpt_svc_ai_agent_broker["svc-ai-agent-broker 🐳🐝"] dpt_svc_ai_robot["svc-ai-robot 💻"] dpt_svc_db_qdrant["svc-db-qdrant 🐳🐝"] dpt_web_app_baserow["web-app-baserow 🐳🐝"] @@ -51,7 +52,6 @@ flowchart LR dpt_web_app_jellyfin["web-app-jellyfin 🐳🐝"] dpt_web_app_jenkins["web-app-jenkins 🐳🐝"] dpt_web_app_listmonk["web-app-listmonk 🐳🐝"] - dpt_web_app_mattermost["web-app-mattermost 🐳🐝"] dpt_more["..."] end dep_svc_ai_litellm -. "0..1" .-> svc_litellm @@ -65,6 +65,7 @@ flowchart LR dep_web_svc_css -. "0..1" .-> svc_css dep_web_svc_logout -. "0..1" .-> svc_logout svc_openclaw -- "1:1" --> dpt_more + svc_openclaw -- "1:1" --> dpt_svc_ai_agent_broker svc_openclaw -. "0..1" .-> dpt_svc_ai_robot svc_openclaw -. "0..1" .-> dpt_svc_db_qdrant svc_openclaw -. "0..1" .-> dpt_web_app_baserow @@ -76,7 +77,6 @@ flowchart LR svc_openclaw -. "0..1" .-> dpt_web_app_jellyfin svc_openclaw -. "0..1" .-> dpt_web_app_jenkins svc_openclaw -. "0..1" .-> dpt_web_app_listmonk - svc_openclaw -. "0..1" .-> dpt_web_app_mattermost ``` Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`. diff --git a/roles/web-app-openclaw/meta/rbac.yml b/roles/web-app-openclaw/meta/rbac.yml index 70019a9c87..f88b8e88ab 100644 --- a/roles/web-app-openclaw/meta/rbac.yml +++ b/roles/web-app-openclaw/meta/rbac.yml @@ -2,3 +2,5 @@ roles: mcp: description: May drive this agent's MCP clients, which reach every admitted provider with the credentials the deployment issued rather than the caller's own + agent-user: + description: May start a personal OpenClaw agent from Open WebUI through svc-ai-agent-broker diff --git a/roles/web-app-openwebui/README.md b/roles/web-app-openwebui/README.md index 43ba37378a..c8fa0c1498 100644 --- a/roles/web-app-openwebui/README.md +++ b/roles/web-app-openwebui/README.md @@ -8,6 +8,17 @@ End users access a web page, pick a model, and start chatting. Conversations remain on your servers. Admins can enable strict offline behavior so no external network calls occur. The UI can also point at OpenAI-compatible endpoints if needed. +Document extraction (`svc-ai-tika`), web search (`svc-ai-searxng`) and the code interpreter (`svc-ai-jupyter`) are wired but off by default. Deploying those roles is not enough: each is a `services.` flag on this role, and only that flag both attaches Open WebUI to the provider's network and renders the matching environment block. Turning all three on adds 3.5 GB of memory limit to the closure that every role reaches through Open WebUI, which is more than the variant budget in `tests/integration/roles/meta/variants/test_resource_budget.py` has left, so the choice stays with the operator: + +```yaml +applications: + web-app-openwebui: + services: + tika: { enabled: true, shared: true } + searxng: { enabled: true, shared: true } + jupyter: { enabled: true, shared: true } +``` + ## Cosmos The diagram places Open WebUI in the Infinito.Nexus cosmos: the components it deploys (capabilities), the central services it consumes (dependencies), and its outward reach (federation and bridged external networks). @@ -15,9 +26,11 @@ The diagram places Open WebUI in the Infinito.Nexus cosmos: the components it de ```mermaid flowchart LR subgraph deps [Dependencies] + dep_svc_ai_agent_broker["svc-ai-agent-broker 🐳🐝"] dep_svc_ai_litellm["svc-ai-litellm 🐳🐝"] dep_svc_bkp_volume_2_local["svc-bkp-volume-2-local 💻"] dep_svc_db_openldap["svc-db-openldap 🐳🐝"] + dep_svc_db_qdrant["svc-db-qdrant 🐳🐝"] dep_svc_net_tor["svc-net-tor 🐳🐝"] dep_web_app_dashboard["web-app-dashboard 🐳🐝"] dep_web_app_keycloak["web-app-keycloak 🐳🐝"] @@ -41,6 +54,11 @@ flowchart LR svc_css["css"] svc_javascript["javascript"] svc_litellm["litellm"] + svc_agent_broker["agent-broker"] + svc_qdrant["qdrant"] + svc_tika["tika ❌"] + svc_searxng["searxng ❌"] + svc_jupyter["jupyter ❌"] svc_email["email"] svc_prometheus["prometheus"] svc_tor["tor"] @@ -61,9 +79,11 @@ flowchart LR dpt_web_app_moodle["web-app-moodle 🐳🐝"] dpt_more["..."] end + dep_svc_ai_agent_broker -. "0..1" .-> svc_agent_broker dep_svc_ai_litellm -. "0..1" .-> svc_litellm dep_svc_bkp_volume_2_local -. "0..1" .-> svc_container_backup dep_svc_db_openldap -. "0..1" .-> svc_ldap + dep_svc_db_qdrant -. "0..1" .-> svc_qdrant dep_svc_net_tor -. "0..1" .-> svc_tor dep_web_app_dashboard -. "0..1" .-> svc_dashboard dep_web_app_keycloak -. "0..1" .-> svc_sso @@ -97,6 +117,9 @@ Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional ( * Optional **offline mode** for air-gapped environments * File/paste input for summaries and extraction (model dependent) * Suitable for teams: predictable, private, reproducible +* With `svc-ai-agent-broker` deployed, lists `hermes` and `openclaw` only to members of the matching `agent-user` group and grants every LiteLLM model to all users; a `svc-ai-litellm` deploy re-grants the gateway models in a running Open WebUI +* Retrieval, web search and code execution come from platform services (`svc-ai-tika`, `svc-ai-searxng`, `svc-ai-jupyter`, `svc-db-qdrant`) rather than from a third-party API, each switched on per deployment +* `services.openwebui.embedding_model`, `.image_model`, `.speech_to_text_model` and `.text_to_speech_model` route embeddings, images, speech recognition and speech synthesis through the gateway; an embedding model set here also replaces the sentence-transformer the image otherwise downloads from Hugging Face on first start ## Quick Setup diff --git a/roles/web-app-openwebui/files/playwright/playwright.spec.js b/roles/web-app-openwebui/files/playwright/playwright.spec.js index 047e193078..fbd7cba325 100644 --- a/roles/web-app-openwebui/files/playwright/playwright.spec.js +++ b/roles/web-app-openwebui/files/playwright/playwright.spec.js @@ -14,8 +14,10 @@ require("./test-admin-ldap-login").register(shared); require("./test-biber-ldap-login").register(shared); require("./test-guest-persona").register(shared); require("./test-litellm-chat").register(shared); +require("./test-system-one-router").register(shared); require("./test-mcp-servers").register(shared); require("./test-mcp-bridge").register(shared); require("./test-mcp-non-member").register(shared); require("./test-mcp-grant-via-lam").register(shared); +require("./test-agent-broker").register(shared); require("./test-seaweedfs"); diff --git a/roles/web-app-openwebui/files/playwright/test-agent-broker.js b/roles/web-app-openwebui/files/playwright/test-agent-broker.js new file mode 100644 index 0000000000..ee0738f5fc --- /dev/null +++ b/roles/web-app-openwebui/files/playwright/test-agent-broker.js @@ -0,0 +1,282 @@ +const { test, expect, request } = require("@playwright/test"); +const { resolveTimeout } = require("./timeouts"); +const { isServiceEnabled, skipUnlessServiceEnabled } = require("./service-gating"); +const { decodeDotenvQuotedValue } = require("./personas"); + +const kcBaseUrl = decodeDotenvQuotedValue(process.env.KEYCLOAK_BASE_URL || "").replace(/\/$/, ""); +const kcRealm = decodeDotenvQuotedValue(process.env.KEYCLOAK_REALM || ""); +const kcAdminUser = decodeDotenvQuotedValue(process.env.KEYCLOAK_ADMIN_USERNAME || ""); +const kcAdminPw = decodeDotenvQuotedValue(process.env.KEYCLOAK_ADMIN_PASSWORD || ""); +function agentGroups() { + const raw = decodeDotenvQuotedValue(process.env.AGENT_GROUP_PATHS || ""); + expect(raw, "AGENT_GROUP_PATHS must be rendered while the broker is enabled").toBeTruthy(); + return JSON.parse(raw); +} + +async function keycloakAdmin() { + const api = await request.newContext({ ignoreHTTPSErrors: true }); + const tokenResp = await api.post(`${kcBaseUrl}/realms/master/protocol/openid-connect/token`, { + form: { client_id: "admin-cli", username: kcAdminUser, password: kcAdminPw, grant_type: "password" }, + }); + expect(tokenResp.ok(), `Keycloak admin token (HTTP ${tokenResp.status()})`).toBeTruthy(); + const headers = { Authorization: `Bearer ${(await tokenResp.json()).access_token}` }; + return { api, headers }; +} + +/** + * Put one user into, or take it out of, the named agent groups. + * + * Args: + * username: the Keycloak username to change. + * membership: group path -> whether the user must end up a member. + */ +async function applyAgentGroups(username, membership) { + const { api, headers } = await keycloakAdmin(); + try { + const users = await ( + await api.get(`${kcBaseUrl}/admin/realms/${kcRealm}/users?username=${encodeURIComponent(username)}&exact=true`, { headers }) + ).json(); + expect(users, `Keycloak user ${username} must exist`).toHaveLength(1); + for (const [path, member] of Object.entries(membership)) { + const groupResp = await api.get(`${kcBaseUrl}/admin/realms/${kcRealm}/group-by-path${path}`, { headers }); + expect(groupResp.ok(), `Keycloak group ${path} must exist (HTTP ${groupResp.status()})`).toBeTruthy(); + const url = `${kcBaseUrl}/admin/realms/${kcRealm}/users/${users[0].id}/groups/${(await groupResp.json()).id}`; + const resp = member ? await api.put(url, { headers }) : await api.delete(url, { headers }); + expect([200, 204, 404]).toContain(resp.status()); + } + } finally { + await api.dispose(); + } +} + +async function setAgentGroups(username, member) { + await applyAgentGroups( + username, + Object.fromEntries(Object.values(agentGroups()).map((path) => [path, member])), + ); +} + +/** + * Leave the user in exactly one platform's agent group. + * + * Args: + * username: the Keycloak username to change. + * granted: the platform whose group the user keeps; every other is removed. + */ +async function grantOnlyAgentGroup(username, granted) { + const groups = agentGroups(); + await applyAgentGroups( + username, + Object.fromEntries(Object.entries(groups).map(([platform, path]) => [path, platform === granted])), + ); +} + +async function openwebuiSession(shared, page) { + await shared.signInViaDashboardOidc(page, shared.env.biberUsername, shared.env.biberPassword, "biber"); + const token = await page.evaluate(() => window.localStorage.getItem("token")); + expect(token, "OpenWebUI must store a session token for biber").toBeTruthy(); + return { + base: shared.env.openwebuiBaseUrl.replace(/\/+$/, ""), + headers: { Authorization: `Bearer ${token}`, "Content-Type": "application/json" }, + }; +} + +/** + * Run work inside a fresh Open WebUI administrator session. + * + * Args: + * page: any page; its browser opens the administrator context. + * shared: the role's shared spec helpers. + * work: async (adminPage, admin) => result, where admin holds base and headers. + */ +async function asAdministrator(page, shared, work) { + const context = await page.context().browser().newContext({ ignoreHTTPSErrors: true }); + try { + const adminPage = await context.newPage(); + await shared.signInViaDashboardOidc(adminPage, shared.env.adminUsername, shared.env.adminPassword, "administrator"); + const token = await adminPage.evaluate(() => window.localStorage.getItem("token")); + expect(token, "OpenWebUI must store a session token for the administrator").toBeTruthy(); + return await work(adminPage, { + base: shared.env.openwebuiBaseUrl.replace(/\/+$/, ""), + headers: { Authorization: `Bearer ${token}` }, + }); + } finally { + await context.close(); + } +} + +/** + * Take one user out of every Open WebUI agent group. + * + * Args: + * adminPage: a page holding an administrator session. + * admin: the administrator's base URL and headers. + * userId: the Open WebUI user id to remove. + * + * Open WebUI's OIDC sync drops a stale group only while the groups claim is + * non-empty; biber loses its last group here, so the revocation in Keycloak + * never reaches Open WebUI on its own. + */ +async function dropAgentMemberships(adminPage, admin, userId) { + const listed = await adminPage.request.get(`${admin.base}/api/v1/groups/`, { headers: admin.headers }); + expect(listed.ok(), `the administrator must list the groups (HTTP ${listed.status()})`).toBeTruthy(); + const paths = Object.values(agentGroups()); + for (const group of (await listed.json()).filter((g) => paths.includes(g?.name))) { + const removed = await adminPage.request.post(`${admin.base}/api/v1/groups/id/${group.id}/users/remove`, { + headers: admin.headers, + data: { user_ids: [userId] }, + }); + expect(removed.ok(), `the administrator must remove ${userId} from ${group.name} (HTTP ${removed.status()})`).toBeTruthy(); + } +} + +/** + * Run work inside a browser context that has just signed biber in. + * + * A membership change only reaches Open WebUI through a fresh OIDC sign-in, so + * every step of the per-group walk needs its own context rather than a reused + * token that still carries the previous groups claim. + * + * Args: + * page: any page; its browser opens the new context. + * shared: the role's shared spec helpers. + * work: async (biberPage, session) => result. + */ +async function asBiber(page, shared, work) { + const context = await page.context().browser().newContext({ ignoreHTTPSErrors: true }); + try { + const biberPage = await context.newPage(); + const session = await openwebuiSession(shared, biberPage); + return await work(biberPage, session); + } finally { + await context.close(); + } +} + +async function sessionUserId(page, session) { + const me = await page.request.get(`${session.base}/api/v1/auths/`, { headers: session.headers }); + expect(me.ok(), `OpenWebUI must report the signed-in user's id (HTTP ${me.status()})`).toBeTruthy(); + return (await me.json()).id; +} + +async function listedModels(page, session, path) { + const resp = await page.request.get(`${session.base}${path}`, { headers: session.headers }); + expect(resp.ok(), `OpenWebUI ${path} (HTTP ${resp.status()})`).toBeTruthy(); + const body = await resp.json(); + return (Array.isArray(body?.data) ? body.data : []).map((m) => String(m?.id ?? "")); +} + +exports.register = function (shared) { + test("biber: without agent-user no agent model is listed or answers", async ({ page }) => { + skipUnlessServiceEnabled("agent-broker"); + skipUnlessServiceEnabled("sso"); + test.setTimeout(resolveTimeout(180_000)); + + await setAgentGroups(shared.env.biberUsername, false); + const session = await openwebuiSession(shared, page); + const biberId = await sessionUserId(page, session); + const servedModels = await asAdministrator(page, shared, async (adminPage, admin) => { + await dropAgentMemberships(adminPage, admin, biberId); + return listedModels(adminPage, admin, "/openai/models"); + }); + const models = await listedModels(page, session, "/api/models?refresh=true"); + const gatewayModels = servedModels.filter((id) => !(id in agentGroups())); + if (isServiceEnabled("litellm")) { + expect(gatewayModels.length, "the connections must serve the gateway models").toBeGreaterThan(0); + } + for (const model of gatewayModels) { + expect(models, `the gateway model ${model} must be granted to every user`).toContain(model); + } + for (const platform of Object.keys(agentGroups())) { + expect(models, `${platform} is granted to its agent-user group only`).not.toContain(platform); + const chat = await page.request.post(`${session.base}/api/chat/completions`, { + headers: session.headers, + data: { model: platform, messages: [{ role: "user", content: "ping" }], stream: false }, + failOnStatusCode: false, + }); + expect(chat.ok(), `a completion on ${platform} must be refused for biber (HTTP ${chat.status()})`).toBeFalsy(); + } + }); + + test("biber: with agent-user each platform starts a personal agent that answers", async ({ page }) => { + skipUnlessServiceEnabled("agent-broker"); + skipUnlessServiceEnabled("sso"); + test.setTimeout(resolveTimeout(3_600_000)); + + await setAgentGroups(shared.env.biberUsername, true); + try { + const session = await openwebuiSession(shared, page); + const models = await listedModels(page, session, "/api/models?refresh=true"); + for (const platform of Object.keys(agentGroups())) { + expect(models, `${platform} must be listed once biber holds its agent-user group`).toContain(platform); + const chat = await page.request.post(`${session.base}/api/chat/completions`, { + headers: session.headers, + data: { + model: platform, + messages: [{ role: "user", content: "Reply with exactly the word: pong" }], + stream: false, + }, + timeout: resolveTimeout(1_800_000), + }); + expect(chat.ok(), `the ${platform} agent must answer through the broker (HTTP ${chat.status()})`).toBeTruthy(); + const content = ((await chat.json())?.choices?.[0]?.message?.content ?? "").trim(); + expect(content, `the ${platform} agent must return an answer through the broker`).not.toBe(""); + } + } finally { + await setAgentGroups(shared.env.biberUsername, false); + } + }); + + test("biber: each agent-user group grants exactly its own platform, and revoking it takes that platform away", async ({ page }) => { + skipUnlessServiceEnabled("agent-broker"); + skipUnlessServiceEnabled("sso"); + test.setTimeout(resolveTimeout(3_600_000)); + + const platforms = Object.keys(agentGroups()); + expect(platforms.length, "the broker must offer at least one platform to walk").toBeGreaterThan(0); + + try { + for (const granted of platforms) { + await grantOnlyAgentGroup(shared.env.biberUsername, granted); + await asBiber(page, shared, async (biberPage, session) => { + const models = await listedModels(biberPage, session, "/api/models?refresh=true"); + expect(models, `${granted} must be listed while biber holds only its group`).toContain(granted); + const answered = await biberPage.request.post(`${session.base}/api/chat/completions`, { + headers: session.headers, + data: { model: granted, messages: [{ role: "user", content: "ping" }], stream: false }, + timeout: resolveTimeout(1_800_000), + }); + expect(answered.ok(), `the ${granted} agent must answer while its group is held (HTTP ${answered.status()})`).toBeTruthy(); + + for (const withheld of platforms.filter((platform) => platform !== granted)) { + expect(models, `${withheld} must stay hidden while biber holds only the ${granted} group`).not.toContain(withheld); + const refused = await biberPage.request.post(`${session.base}/api/chat/completions`, { + headers: session.headers, + data: { model: withheld, messages: [{ role: "user", content: "ping" }], stream: false }, + failOnStatusCode: false, + }); + expect(refused.ok(), `${withheld} must stay refused while biber holds only the ${granted} group (HTTP ${refused.status()})`).toBeFalsy(); + } + }); + } + } finally { + await setAgentGroups(shared.env.biberUsername, false); + } + + const lastGranted = platforms[platforms.length - 1]; + await asBiber(page, shared, async (biberPage, session) => { + const biberId = await sessionUserId(biberPage, session); + await asAdministrator(biberPage, shared, (adminPage, admin) => dropAgentMemberships(adminPage, admin, biberId)); + }); + await asBiber(page, shared, async (biberPage, session) => { + const models = await listedModels(biberPage, session, "/api/models?refresh=true"); + expect(models, `${lastGranted} must disappear once its group is revoked`).not.toContain(lastGranted); + const refused = await biberPage.request.post(`${session.base}/api/chat/completions`, { + headers: session.headers, + data: { model: lastGranted, messages: [{ role: "user", content: "ping" }], stream: false }, + failOnStatusCode: false, + }); + expect(refused.ok(), `${lastGranted} must be refused once its group is revoked (HTTP ${refused.status()})`).toBeFalsy(); + }); + }); +}; diff --git a/roles/web-app-openwebui/files/playwright/test-system-one-router.js b/roles/web-app-openwebui/files/playwright/test-system-one-router.js new file mode 100644 index 0000000000..bb7fe05056 --- /dev/null +++ b/roles/web-app-openwebui/files/playwright/test-system-one-router.js @@ -0,0 +1,69 @@ +const { test, expect } = require("@playwright/test"); +const { resolveTimeout } = require("./timeouts"); +const { skipUnlessServiceEnabled } = require("./service-gating"); + +const ROUTER_ALIAS = "auto"; + +exports.register = function (shared) { + test("system one router: the router alias answers, served by a model that is not itself", async ({ page }) => { + skipUnlessServiceEnabled("litellm"); + skipUnlessServiceEnabled("sso"); + test.setTimeout(resolveTimeout(120_000)); + + await shared.signInViaDashboardOidc( + page, + shared.env.adminUsername, + shared.env.adminPassword, + "administrator" + ); + + const token = await page.evaluate(() => window.localStorage.getItem("token")); + expect( + token, + "OpenWebUI must store a session token in localStorage after login (used to authenticate its OpenAI-compatible API)" + ).toBeTruthy(); + + const base = shared.env.openwebuiBaseUrl.replace(/\/+$/, ""); + const headers = { Authorization: `Bearer ${token}`, "Content-Type": "application/json" }; + + const modelsResp = await page.request.get(`${base}/api/models`, { headers }); + expect( + modelsResp.ok(), + `OpenWebUI /api/models must respond to the authenticated session (HTTP ${modelsResp.status()})` + ).toBeTruthy(); + + const modelsBody = await modelsResp.json(); + const modelIds = (Array.isArray(modelsBody?.data) ? modelsBody.data : []) + .map((m) => String(m?.id ?? m?.name ?? "")); + expect( + modelIds, + `the gateway must publish the router alias '${ROUTER_ALIAS}' so a consumer can select it like any other model (got ${JSON.stringify(modelIds)})` + ).toContain(ROUTER_ALIAS); + + const chatResp = await page.request.post(`${base}/api/chat/completions`, { + headers, + data: { + model: ROUTER_ALIAS, + messages: [{ role: "user", content: "Reply with exactly the word: pong" }], + stream: false, + }, + }); + expect( + chatResp.ok(), + `'${ROUTER_ALIAS}' must return 200 over OpenWebUI -> LiteLLM; a pre-call hook that raised instead of rewriting the model shows up here (HTTP ${chatResp.status()})` + ).toBeTruthy(); + + const chatBody = await chatResp.json(); + const content = (chatBody?.choices?.[0]?.message?.content ?? "").trim(); + expect( + content.length, + `'${ROUTER_ALIAS}' must return a non-empty assistant response (got ${JSON.stringify(chatBody).slice(0, 300)})` + ).toBeGreaterThan(0); + + const servedBy = String(chatBody?.model ?? ""); + expect( + servedBy && servedBy !== ROUTER_ALIAS, + `the answer must name the model the hook chose rather than the alias itself, which is what distinguishes a routed request from the fallback route (got ${JSON.stringify(servedBy)})` + ).toBeTruthy(); + }); +}; diff --git a/roles/web-app-openwebui/files/python/agent_model_grants.py b/roles/web-app-openwebui/files/python/agent_model_grants.py new file mode 100644 index 0000000000..3dc2a211c9 --- /dev/null +++ b/roles/web-app-openwebui/files/python/agent_model_grants.py @@ -0,0 +1,159 @@ +""" +Environment: + OPENWEBUI_BASE: in-container Open WebUI base URL. + OPENWEBUI_ADMIN_EMAIL: administrator to take or mint an API key for. + OPENWEBUI_ADMIN_NAME: its display name when it has to be created. + OPENWEBUI_ADMIN_PASSWORD: its password when it has to be created. + OPENWEBUI_GATEWAY_URL: LiteLLM base URL whose models become public; empty skips them. + OPENWEBUI_GATEWAY_KEY: Open WebUI's own gateway key. + OPENWEBUI_AGENT_MODELS: JSON model id -> group name. +""" + +import asyncio +import json +import os +import secrets +import sys +import urllib.error +import urllib.request + +sys.path.insert(0, "/app/backend") + +BASE = os.environ["OPENWEBUI_BASE"].rstrip("/") +ADMIN_EMAIL = os.environ["OPENWEBUI_ADMIN_EMAIL"] +ADMIN_NAME = os.environ["OPENWEBUI_ADMIN_NAME"] +ADMIN_PASSWORD = os.environ["OPENWEBUI_ADMIN_PASSWORD"] +GATEWAY_URL = os.environ["OPENWEBUI_GATEWAY_URL"].rstrip("/") +GATEWAY_KEY = os.environ["OPENWEBUI_GATEWAY_KEY"] +AGENT_MODELS = json.loads(os.environ["OPENWEBUI_AGENT_MODELS"]) +PUBLIC = [{"principal_type": "user", "principal_id": "*", "permission": "read"}] + + +async def resolve_api_key(): + from open_webui.models.auths import Auths + from open_webui.models.users import Users + from open_webui.utils.auth import get_password_hash + + existing = (await Users.get_users()).get("users") or [] + admin = next((user for user in existing if user.role == "admin"), None) + if admin is None: + admin = await Auths.insert_new_auth( + email=ADMIN_EMAIL.lower(), + password=await get_password_hash(ADMIN_PASSWORD), + name=ADMIN_NAME, + role="admin", + ) + if admin is None: + sys.exit("FAILED: no administrator exists and one could not be created") + key = await Users.get_user_api_key_by_id(admin.id) + if key: + return admin.id, key, False + key = f"sk-{secrets.token_hex(32)}" + if not await Users.update_user_api_key_by_id(admin.id, key): + sys.exit(f"FAILED: could not mint an API key for {admin.id}") + return admin.id, key, True + + +async def drop_api_key(user_id): + from open_webui.models.users import Users + + await Users.delete_user_api_key_by_id(user_id) + + +def call(url, key, payload=None): + data = json.dumps(payload).encode() if payload is not None else None + request = urllib.request.Request( # noqa: S310 - fixed http:// bases from the environment, no user-supplied scheme + url, data=data, method="POST" if data else "GET" + ) + request.add_header("Content-Type", "application/json") + request.add_header("Authorization", f"Bearer {key}") + try: + with urllib.request.urlopen(request, timeout=30) as response: # noqa: S310 - fixed http:// bases from the environment, no user-supplied scheme + return response.status, json.loads(response.read() or b"null") + except urllib.error.HTTPError as error: + return error.code, error.read().decode(errors="replace") + + +def gateway_models(): + if not GATEWAY_URL: + return [] + status, body = call(f"{GATEWAY_URL}/v1/models", GATEWAY_KEY) + if status != 200: + sys.exit(f"FAILED listing gateway models: {status} {body}") + return [item["id"] for item in body.get("data") or []] + + +def group_id(key, name): + status, body = call(f"{BASE}/api/v1/groups/", key) + if status != 200: + sys.exit(f"FAILED listing groups: {status} {body}") + matches = [group for group in body if group.get("name") == name] + if len(matches) > 1: + sys.exit(f"FAILED: {len(matches)} groups named {name}, refusing to guess") + if matches: + return matches[0]["id"] + status, body = call( + f"{BASE}/api/v1/groups/create", + key, + {"name": name, "description": "Per-user agent access"}, + ) + if status != 200: + sys.exit(f"FAILED creating group {name}: {status} {body}") + return body["id"] + + +async def _stored_grants(model_id): + from open_webui.models.access_grants import AccessGrants + + grants = await AccessGrants.get_grants_by_resource("model", model_id) + return sorted( + (grant.principal_type, grant.principal_id, grant.permission) + for grant in grants or [] + ) + + +def current_grants(model_id): + return asyncio.run(_stored_grants(model_id)) + + +def ensure_grants(key, model_id, grants): + wanted = sorted( + (g["principal_type"], g["principal_id"], g["permission"]) for g in grants + ) + if current_grants(model_id) == wanted: + return False + status, body = call( + f"{BASE}/api/v1/models/model/access/update", + key, + {"id": model_id, "name": model_id, "access_grants": grants}, + ) + if status != 200: + sys.exit(f"FAILED granting {model_id}: {status} {body}") + if current_grants(model_id) != wanted: + sys.exit(f"FAILED: {model_id} does not carry {wanted} after the write") + return True + + +def provision(key): + changed = False + for model_id in gateway_models(): + if model_id not in AGENT_MODELS: + changed |= ensure_grants(key, model_id, PUBLIC) + for model_id, group_name in AGENT_MODELS.items(): + grant = { + "principal_type": "group", + "principal_id": group_id(key, group_name), + "permission": "read", + } + changed |= ensure_grants(key, model_id, [grant]) + return changed + + +if __name__ == "__main__": + admin_id, api_key, minted = asyncio.run(resolve_api_key()) + try: + result = provision(api_key) + finally: + if minted: + asyncio.run(drop_api_key(admin_id)) + print("CHANGED" if result else "OK") diff --git a/roles/web-app-openwebui/meta/secrets.yml b/roles/web-app-openwebui/meta/secrets.yml index 9de984053a..3a2790038f 100644 --- a/roles/web-app-openwebui/meta/secrets.yml +++ b/roles/web-app-openwebui/meta/secrets.yml @@ -6,6 +6,11 @@ credentials: algorithm: alphanumeric validation: "^[A-Za-z0-9]{32,}$" length: 64 + comfyui_api_key: + description: Key of the image-generation backend the operator points this deployment at. Empty unless services.openwebui.comfyui_url names one, because the + platform ships no pinned ComfyUI image of its own. + algorithm: plain + validation: ^.*$ litellm_api_key: description: Per-consumer virtual key for the shared svc-ai-litellm gateway algorithm: sk_prefixed diff --git a/roles/web-app-openwebui/meta/services.yml b/roles/web-app-openwebui/meta/services.yml index 69ea6a7ae6..fb2dcc48a4 100644 --- a/roles/web-app-openwebui/meta/services.yml +++ b/roles/web-app-openwebui/meta/services.yml @@ -42,6 +42,11 @@ openwebui: offline_mode: false hf_hub_offline: false enable_api_keys: true + image_model: "" + embedding_model: "" + speech_to_text_model: "" + text_to_speech_model: "" + comfyui_url: "" cpus: 2.0 mem_reservation: 1g mem_limit: 4g @@ -86,6 +91,30 @@ litellm: bond: 1 enabled: "{{ LITELLM_USABLE }}" shared: true +agent-broker: + bond: 1 + enabled: "{{ 'svc-ai-agent-broker' in group_names }}" + shared: "{{ 'svc-ai-agent-broker' in group_names }}" +# nocheck: playwright-service-flag - DB engine, covered by role-local integration tests +qdrant: + bond: 1 + enabled: "{{ 'svc-db-qdrant' in group_names }}" + shared: "{{ 'svc-db-qdrant' in group_names }}" +# nocheck: playwright-service-flag - extraction engine without a browser surface +tika: + bond: 0 + enabled: false # nocheck: dynamic-flag + shared: false # nocheck: dynamic-flag +# nocheck: playwright-service-flag - search backend consumed over the container network +searxng: + bond: 0 + enabled: false # nocheck: dynamic-flag + shared: false # nocheck: dynamic-flag +# nocheck: playwright-service-flag - kernel backend consumed over the container network +jupyter: + bond: 0 + enabled: false # nocheck: dynamic-flag + shared: false # nocheck: dynamic-flag email: bond: 1 diff --git a/roles/web-app-openwebui/meta/variants.yml b/roles/web-app-openwebui/meta/variants.yml index 13acca7e3c..80f80ac777 100644 --- a/roles/web-app-openwebui/meta/variants.yml +++ b/roles/web-app-openwebui/meta/variants.yml @@ -27,6 +27,12 @@ shared: true litellm: enabled: true + agent-broker: + enabled: true + shared: true + qdrant: + enabled: true + shared: true prometheus: enabled: true shared: true @@ -62,6 +68,12 @@ shared: true litellm: enabled: false + agent-broker: + enabled: false + shared: false + qdrant: + enabled: false + shared: false prometheus: enabled: false shared: false # nocheck: variants-static-override backup shared stays literally true @@ -100,6 +112,12 @@ shared: false # nocheck: variants-static-override backup shared stays literally true litellm: enabled: false + agent-broker: + enabled: false + shared: false + qdrant: + enabled: false + shared: false prometheus: enabled: false shared: false # nocheck: variants-static-override backup shared stays literally true diff --git a/roles/web-app-openwebui/tasks/00_core.yml b/roles/web-app-openwebui/tasks/00_core.yml index f97313fe8c..579327172c 100644 --- a/roles/web-app-openwebui/tasks/00_core.yml +++ b/roles/web-app-openwebui/tasks/00_core.yml @@ -17,3 +17,7 @@ - name: "🔌 Provision MCP tool-server access grants" ansible.builtin.include_tasks: utils/mcp.yml when: OPENWEBUI_MCP_SERVERS | length > 0 + +- name: "🤖 Provision per-group access to the agent models" + ansible.builtin.include_tasks: utils/agent_models.yml + when: OPENWEBUI_AGENT_BROKER_ENABLED | bool diff --git a/roles/web-app-openwebui/tasks/utils/agent_models.yml b/roles/web-app-openwebui/tasks/utils/agent_models.yml new file mode 100644 index 0000000000..1f60301ee9 --- /dev/null +++ b/roles/web-app-openwebui/tasks/utils/agent_models.yml @@ -0,0 +1,40 @@ +--- +- name: "🔍 Resolve {{ application_id }} hosting node + container id" + ansible.builtin.include_tasks: "{{ lookup('path_absolute', 'roles/sys-svc-compose/tasks/utils/swarm/resolve_host_cid.yml') }}" + vars: + resolve_service_key: "{{ entity_name }}" + resolve_output_prefix: OPENWEBUI + resolve_compose_cid: "{{ OPENWEBUI_CONTAINER }}" + +- name: "🔑 Publish the gateway models and bind each agent model to its group" + ansible.builtin.command: + argv: + - container + - exec + - -i + - -e + - "OPENWEBUI_BASE=http://localhost:{{ OPENWEBUI_INTERNAL_HTTP_PORT }}" + - -e + - "OPENWEBUI_ADMIN_EMAIL={{ OPENWEBUI_API_BOT_USER.email }}" + - -e + - "OPENWEBUI_ADMIN_NAME={{ OPENWEBUI_API_BOT_USER.username }}" + - -e + - "OPENWEBUI_ADMIN_PASSWORD={{ OPENWEBUI_API_BOT_USER.password }}" + - -e + - "OPENWEBUI_GATEWAY_URL={{ LITELLM_BASE_LOCAL_URL if (OPENWEBUI_LITELLM_ENABLED | bool) else '' }}" + - -e + - "OPENWEBUI_GATEWAY_KEY={{ OPENWEBUI_LITELLM_API_KEY if (OPENWEBUI_LITELLM_ENABLED | bool) else '' }}" + - -e + - "OPENWEBUI_AGENT_MODELS={{ OPENWEBUI_AGENT_MODELS | to_json }}" + - "{{ OPENWEBUI_LOCAL_CID }}" # nocheck: container-exec-resolver + - python3 + - "-" + stdin: "{{ lookup('file', 'python/agent_model_grants.py') }}" + register: openwebui_model_grants + retries: 30 + delay: 10 + until: openwebui_model_grants.rc == 0 + changed_when: openwebui_model_grants.stdout.startswith('CHANGED') + no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" + delegate_to: "{{ OPENWEBUI_HOST_NODE }}" + run_once: true diff --git a/roles/web-app-openwebui/tasks/utils/refresh_model_grants.yml b/roles/web-app-openwebui/tasks/utils/refresh_model_grants.yml new file mode 100644 index 0000000000..0d5f36b977 --- /dev/null +++ b/roles/web-app-openwebui/tasks/utils/refresh_model_grants.yml @@ -0,0 +1,15 @@ +--- +- name: "🔍 Check whether {{ application_id }} already runs" + ansible.builtin.command: + argv: >- + {{ ['container', 'service', 'ls', '-q', '--filter', 'name=' ~ lookup('container_service', application_id, entity_name)] + if DEPLOYMENT_MODE == 'swarm' + else ['container', 'ps', '-q', '--filter', 'status=running', '--filter', 'name=^' ~ OPENWEBUI_CONTAINER ~ '$'] }} + register: openwebui_running + changed_when: false + delegate_to: "{{ STACK_HOST }}" + run_once: true + +- name: "🔁 Re-grant the gateway models in the running {{ application_id }}" + ansible.builtin.include_tasks: "{{ lookup('path_absolute', 'roles/web-app-openwebui/tasks/utils/agent_models.yml') }}" + when: openwebui_running.stdout | trim | length > 0 diff --git a/roles/web-app-openwebui/templates/env.j2 b/roles/web-app-openwebui/templates/env.j2 index 7a0b8635a5..c181c5d0c5 100644 --- a/roles/web-app-openwebui/templates/env.j2 +++ b/roles/web-app-openwebui/templates/env.j2 @@ -2,9 +2,68 @@ WEBUI_SECRET_KEY={{ OPENWEBUI_SECRET_KEY | dotenv_quote }} -{% if OPENWEBUI_LITELLM_ENABLED | bool %} -OPENAI_API_BASE_URL={{ LITELLM_BASE_LOCAL_URL }}/v1 -OPENAI_API_KEY={{ OPENWEBUI_LITELLM_API_KEY | dotenv_quote }} +{% if OPENWEBUI_OPENAI_CONNECTIONS | length > 0 %} +OPENAI_API_BASE_URLS={{ OPENWEBUI_OPENAI_CONNECTIONS | map(attribute='url') | join(';') }} +OPENAI_API_KEYS={{ OPENWEBUI_OPENAI_CONNECTIONS | map(attribute='key') | join(';') | dotenv_quote }} +{% endif %} +{% if OPENWEBUI_AGENT_BROKER_ENABLED | bool %} +ENABLE_FORWARD_USER_INFO_HEADERS=true +AIOHTTP_CLIENT_TIMEOUT={{ OPENWEBUI_AGENT_BROKER_TIMEOUT }} +{% endif %} +{% if OPENWEBUI_COMFYUI_URL | length > 0 %} +ENABLE_IMAGE_GENERATION=true +IMAGE_GENERATION_ENGINE=comfyui +COMFYUI_BASE_URL={{ OPENWEBUI_COMFYUI_URL }} +COMFYUI_API_KEY={{ OPENWEBUI_COMFYUI_API_KEY | dotenv_quote }} +{% elif OPENWEBUI_LITELLM_ENABLED | bool and OPENWEBUI_IMAGE_MODEL | length > 0 %} +ENABLE_IMAGE_GENERATION=true +IMAGE_GENERATION_ENGINE=openai +IMAGE_GENERATION_MODEL={{ OPENWEBUI_IMAGE_MODEL }} +IMAGES_OPENAI_API_BASE_URL={{ LITELLM_BASE_LOCAL_URL }}/v1 +IMAGES_OPENAI_API_KEY={{ OPENWEBUI_LITELLM_API_KEY | dotenv_quote }} +{% endif %} +{% if OPENWEBUI_LITELLM_ENABLED | bool and OPENWEBUI_EMBEDDING_MODEL | length > 0 %} +RAG_EMBEDDING_ENGINE=openai +RAG_EMBEDDING_MODEL={{ OPENWEBUI_EMBEDDING_MODEL }} +RAG_OPENAI_API_BASE_URL={{ LITELLM_BASE_LOCAL_URL }}/v1 +RAG_OPENAI_API_KEY={{ OPENWEBUI_LITELLM_API_KEY | dotenv_quote }} +{% endif %} +{% if OPENWEBUI_LITELLM_ENABLED | bool and OPENWEBUI_STT_MODEL | length > 0 %} +AUDIO_STT_ENGINE=openai +AUDIO_STT_MODEL={{ OPENWEBUI_STT_MODEL }} +AUDIO_STT_OPENAI_API_BASE_URL={{ LITELLM_BASE_LOCAL_URL }}/v1 +AUDIO_STT_OPENAI_API_KEY={{ OPENWEBUI_LITELLM_API_KEY | dotenv_quote }} +{% endif %} +{% if OPENWEBUI_LITELLM_ENABLED | bool and OPENWEBUI_TTS_MODEL | length > 0 %} +AUDIO_TTS_ENGINE=openai +AUDIO_TTS_MODEL={{ OPENWEBUI_TTS_MODEL }} +AUDIO_TTS_OPENAI_API_BASE_URL={{ LITELLM_BASE_LOCAL_URL }}/v1 +AUDIO_TTS_OPENAI_API_KEY={{ OPENWEBUI_LITELLM_API_KEY | dotenv_quote }} +{% endif %} +{% if OPENWEBUI_JUPYTER_ENABLED | bool %} +CODE_EXECUTION_ENGINE=jupyter +CODE_EXECUTION_JUPYTER_URL={{ OPENWEBUI_JUPYTER_URL }} +CODE_EXECUTION_JUPYTER_AUTH=token +CODE_EXECUTION_JUPYTER_AUTH_TOKEN={{ OPENWEBUI_JUPYTER_TOKEN | dotenv_quote }} +CODE_INTERPRETER_ENGINE=jupyter +CODE_INTERPRETER_JUPYTER_URL={{ OPENWEBUI_JUPYTER_URL }} +CODE_INTERPRETER_JUPYTER_AUTH=token +CODE_INTERPRETER_JUPYTER_AUTH_TOKEN={{ OPENWEBUI_JUPYTER_TOKEN | dotenv_quote }} +{% endif %} +{% if OPENWEBUI_SEARXNG_ENABLED | bool %} +ENABLE_WEB_SEARCH=true +WEB_SEARCH_ENGINE=searxng +SEARXNG_QUERY_URL={{ OPENWEBUI_SEARXNG_URL }} +{% endif %} +{% if OPENWEBUI_TIKA_ENABLED | bool %} +CONTENT_EXTRACTION_ENGINE=tika +TIKA_SERVER_URL={{ OPENWEBUI_TIKA_URL }} +{% endif %} +{% if OPENWEBUI_QDRANT_ENABLED | bool %} +VECTOR_DB=qdrant +QDRANT_URI={{ OPENWEBUI_QDRANT_URL }} +QDRANT_API_KEY={{ OPENWEBUI_QDRANT_API_KEY | dotenv_quote }} +QDRANT_COLLECTION_PREFIX={{ OPENWEBUI_QDRANT_COLLECTION_PREFIX }} {% endif %} {% if OPENWEBUI_MCP_SERVERS | length > 0 %} TOOL_SERVER_CONNECTIONS={{ OPENWEBUI_MCP_SERVERS | mcp_tool_server_connections(OPENWEBUI_MCP_GROUP_IDS) | to_json(separators=(',', ':')) | dotenv_quote }} @@ -64,7 +123,7 @@ OAUTH_GROUPS_CLAIM={{ RBAC.GROUP.CLAIM }} BYPASS_ADMIN_ACCESS_CONTROL=false {% endif %} -BYPASS_MODEL_ACCESS_CONTROL=true +BYPASS_MODEL_ACCESS_CONTROL={{ (not (OPENWEBUI_AGENT_BROKER_ENABLED | bool)) | string | lower }} {% if OPENWEBUI_LDAP_ENABLED | bool %} ENABLE_LDAP=true diff --git a/roles/web-app-openwebui/templates/playwright.env.j2 b/roles/web-app-openwebui/templates/playwright.env.j2 index d72e8209da..c89ebb65ce 100644 --- a/roles/web-app-openwebui/templates/playwright.env.j2 +++ b/roles/web-app-openwebui/templates/playwright.env.j2 @@ -1,5 +1,9 @@ ADMIN_EMAIL={{ lookup('users', 'administrator').email | dotenv_quote }} +AGENT_BROKER_SERVICE_ENABLED={{ lookup('config', application_id, 'services.agent-broker.enabled') | string | lower }} + +AGENT_GROUP_PATHS={{ (OPENWEBUI_AGENT_MODELS if (OPENWEBUI_AGENT_BROKER_ENABLED | bool) else {}) | to_json | dotenv_quote }} + ADMIN_PASSWORD={{ lookup('users', 'administrator').password | dotenv_quote }} ADMIN_USERNAME={{ lookup('users', 'administrator').username | dotenv_quote }} @@ -15,6 +19,14 @@ CANONICAL_DOMAIN={{ lookup('domain', application_id) | dotenv_quote }} # nocheck: playwright-service-gate — covered by sys-svc-mail integration tests, not the persona surface EMAIL_SERVICE_ENABLED={{ lookup('config', application_id, 'services.email.enabled') | string | lower }} +KEYCLOAK_ADMIN_PASSWORD={{ lookup('users', 'administrator').password | dotenv_quote }} + +KEYCLOAK_ADMIN_USERNAME={{ lookup('users', 'administrator').username | dotenv_quote }} + +KEYCLOAK_BASE_URL={{ lookup('tls', 'web-app-keycloak', 'url.base') | dotenv_quote }} + +KEYCLOAK_REALM={{ OIDC.CLIENT.REALM | dotenv_quote }} + LAM_BASE_URL={{ (lookup('tls', 'web-app-lam', 'url.base') if 'web-app-lam' in group_names else '') | dotenv_quote }} LDAP_BIND_DN={{ (LDAP.DN.ADMINISTRATOR.DATA if 'web-app-lam' in group_names else '') | dotenv_quote }} diff --git a/roles/web-app-openwebui/vars/main.yml b/roles/web-app-openwebui/vars/main.yml index d6bcdcd866..474adb9bae 100644 --- a/roles/web-app-openwebui/vars/main.yml +++ b/roles/web-app-openwebui/vars/main.yml @@ -10,6 +10,34 @@ OPENWEBUI_ENABLE_API_KEYS: "{{ lookup('config', application_id, 'services.openwe OPENWEBUI_LDAP_ENABLED: "{{ lookup('config', application_id, 'services.ldap.enabled') }}" OPENWEBUI_LITELLM_ENABLED: "{{ lookup('config', application_id, 'services.litellm.enabled') }}" OPENWEBUI_LITELLM_API_KEY: "{{ lookup('config', application_id, 'secrets.credentials.litellm_api_key') }}" +OPENWEBUI_COMFYUI_URL: "{{ lookup('config', application_id, 'services.openwebui.comfyui_url') }}" +OPENWEBUI_COMFYUI_API_KEY: "{{ lookup('config', application_id, 'secrets.credentials.comfyui_api_key') }}" +OPENWEBUI_IMAGE_MODEL: "{{ lookup('config', application_id, 'services.openwebui.image_model') }}" +OPENWEBUI_EMBEDDING_MODEL: "{{ lookup('config', application_id, 'services.openwebui.embedding_model') }}" +OPENWEBUI_STT_MODEL: "{{ lookup('config', application_id, 'services.openwebui.speech_to_text_model') }}" +OPENWEBUI_TTS_MODEL: "{{ lookup('config', application_id, 'services.openwebui.text_to_speech_model') }}" +OPENWEBUI_JUPYTER_ENABLED: "{{ lookup('config', application_id, 'services.jupyter.enabled') }}" +OPENWEBUI_JUPYTER_URL: "http://{{ 'svc-ai-jupyter' | get_entity_name }}:{{ lookup('config', 'svc-ai-jupyter', 'services.jupyter.ports.internal.http') }}" +OPENWEBUI_JUPYTER_TOKEN: "{{ lookup('config', 'svc-ai-jupyter', 'secrets.credentials.auth_token') }}" +OPENWEBUI_SEARXNG_ENABLED: "{{ lookup('config', application_id, 'services.searxng.enabled') }}" +OPENWEBUI_SEARXNG_URL: "http://{{ 'svc-ai-searxng' | get_entity_name }}:{{ lookup('config', 'svc-ai-searxng', 'services.searxng.ports.internal.http') }}/search?q=" +OPENWEBUI_TIKA_ENABLED: "{{ lookup('config', application_id, 'services.tika.enabled') }}" +OPENWEBUI_TIKA_URL: "http://{{ 'svc-ai-tika' | get_entity_name }}:{{ lookup('config', 'svc-ai-tika', 'services.tika.ports.internal.http') }}" +OPENWEBUI_QDRANT_ENABLED: "{{ lookup('config', application_id, 'services.qdrant.enabled') }}" +OPENWEBUI_QDRANT_URL: "{{ lookup('engine', 'qdrant', application_id, 'url') }}" +OPENWEBUI_QDRANT_API_KEY: "{{ lookup('engine', 'qdrant', application_id, 'password') }}" +OPENWEBUI_QDRANT_COLLECTION_PREFIX: "{{ lookup('engine', 'qdrant', application_id, 'prefix') }}" +OPENWEBUI_AGENT_BROKER_ENABLED: "{{ lookup('config', application_id, 'services.agent-broker.enabled') }}" +OPENWEBUI_AGENT_BROKER_URL: "http://{{ 'svc-ai-agent-broker' | get_entity_name }}:{{ lookup('config', 'svc-ai-agent-broker', 'services.agent-broker.ports.internal.http') + }}/v1" +OPENWEBUI_AGENT_BROKER_KEY: "{{ lookup('config', 'svc-ai-agent-broker', 'secrets.credentials.api_key') }}" +OPENWEBUI_AGENT_BROKER_TIMEOUT: "{{ lookup('config', 'svc-ai-agent-broker', 'services.agent-broker.request_timeout') }}" +OPENWEBUI_AGENT_MODELS: + hermes: "/{{ lookup('rbac_group_path', application_id='web-app-hermes', role='agent-user') }}" + openclaw: "/{{ lookup('rbac_group_path', application_id='web-app-openclaw', role='agent-user') }}" +OPENWEBUI_OPENAI_CONNECTIONS: >- + {{ ([{'url': LITELLM_BASE_LOCAL_URL ~ '/v1', 'key': OPENWEBUI_LITELLM_API_KEY}] if (OPENWEBUI_LITELLM_ENABLED | bool) else []) + + ([{'url': OPENWEBUI_AGENT_BROKER_URL, 'key': OPENWEBUI_AGENT_BROKER_KEY}] if (OPENWEBUI_AGENT_BROKER_ENABLED | bool) else []) }} OPENWEBUI_SECRET_KEY: "{{ lookup('config', application_id, 'secrets.credentials.webui_secret_key') }}" OPENWEBUI_MCP_SERVERS: "{{ MCP_DISCOVERED_SERVERS if (lookup('config', application_id, 'mcp.enabled') | bool) else [] }}" OPENWEBUI_MCP_GROUP_OWNER: "openwebui-mcp-groups" diff --git a/roles/web-svc-cdn/README.md b/roles/web-svc-cdn/README.md index 545b50420f..4201d1551d 100644 --- a/roles/web-svc-cdn/README.md +++ b/roles/web-svc-cdn/README.md @@ -21,7 +21,7 @@ flowchart LR dep_web_app_matomo["web-app-matomo 🐳🐝"] dep_web_app_prometheus["web-app-prometheus 🐳🐝"] end - subgraph role [web-svc-cdn 🐳🐝] + subgraph role [web-svc-cdn 🐳🐝💻] svc_dashboard["dashboard"] svc_cdn["cdn"] svc_build["build"] diff --git a/roles/web-svc-css/README.md b/roles/web-svc-css/README.md index d8812b4d70..025095a313 100644 --- a/roles/web-svc-css/README.md +++ b/roles/web-svc-css/README.md @@ -20,7 +20,7 @@ flowchart LR subgraph deps [Dependencies] dep_svc_net_tor["svc-net-tor 🐳🐝"] dep_web_app_prometheus["web-app-prometheus 🐳🐝"] - dep_web_svc_cdn["web-svc-cdn 🐳🐝"] + dep_web_svc_cdn["web-svc-cdn 🐳🐝💻"] end subgraph role [web-svc-css 💻] svc_css["css"] diff --git a/roles/web-svc-html/README.md b/roles/web-svc-html/README.md index fa67901abf..c7d9177703 100644 --- a/roles/web-svc-html/README.md +++ b/roles/web-svc-html/README.md @@ -36,7 +36,7 @@ flowchart LR svc_tor["tor"] end subgraph dependents [Dependents] - dpt_web_svc_legal["web-svc-legal"] + dpt_web_svc_legal["web-svc-legal 🐳🐝"] end dep_svc_net_tor -. "0..1" .-> svc_tor dep_web_app_matomo -. "0..1" .-> svc_matomo diff --git a/roles/web-svc-legal/README.md b/roles/web-svc-legal/README.md index 92bc5fbab6..bb07ea912e 100644 --- a/roles/web-svc-legal/README.md +++ b/roles/web-svc-legal/README.md @@ -19,7 +19,7 @@ flowchart LR dep_web_app_prometheus["web-app-prometheus 🐳🐝"] dep_web_svc_html["web-svc-html 💻"] end - subgraph role [web-svc-legal] + subgraph role [web-svc-legal 🐳🐝] svc_legal["legal ❌"] svc_html["html"] svc_prometheus["prometheus"] diff --git a/roles/web-svc-legal/meta/services.yml b/roles/web-svc-legal/meta/services.yml index ad0c8e2d41..1ae841bec2 100644 --- a/roles/web-svc-legal/meta/services.yml +++ b/roles/web-svc-legal/meta/services.yml @@ -2,6 +2,10 @@ # nocheck: dynamic-flag legal: modes: + compose: + enabled: true + swarm: + enabled: true host: enabled: false enabled: false diff --git a/scripts/tests/deploy/local/deploy/main.sh b/scripts/tests/deploy/local/deploy/main.sh index 225182099a..e549164776 100755 --- a/scripts/tests/deploy/local/deploy/main.sh +++ b/scripts/tests/deploy/local/deploy/main.sh @@ -31,6 +31,10 @@ echo $$ >build/deploy.pid MODE="${mode:-initialize}" # nocheck: deploy router knob; routes to apps// subscripts PURGE="${purge:-false}" # nocheck: deploy router knob; gates entity pre-purge +printf '=== compose-deploy: mode=%s apps=%s bundles=%s variant=%s full_cycle=%s purge=%s disable=%s playwright_keep=%s\n' \ + "${MODE}" "${apps:-}" "${bundles:-}" "${variant:-}" "${full_cycle:-false}" \ + "${PURGE}" "${disable:-}" "${INFINITO_PLAYWRIGHT_KEEP:-}" + case "${MODE}" in initialize | reinstall | update) ;; *) diff --git a/skills/i8-ci-trigger-priority/SKILL.md b/skills/i8-ci-trigger-priority/SKILL.md new file mode 100644 index 0000000000..4d6693fe10 --- /dev/null +++ b/skills/i8-ci-trigger-priority/SKILL.md @@ -0,0 +1,10 @@ +--- +name: i8-ci-trigger-priority +description: > + Lead a manual CI run with a priority line that verifies the branch's still-open questions: up to 20 selections, every axis pinned in every token (role, variant, mode, onion, distro, filesystem) and its value chosen by measured runtime, workspace and chunk gate off. Infinito.Nexus specific. +--- + +Follow the instructions from AGENTS.md, then build and dispatch the priority line by following the +procedure in `docs/agents/action/ci-trigger-priority.md` (relative to the Infinito.Nexus repository root) +exactly. That document is the source of truth; this skill only routes you +there. Begin by clarifying every open requirement with the `active-listening` skill (escalating any root-cause or design question you are not ~99% sure of to the `dialectic` skill), then work under the `robot` skill. Every token MUST pin all six axes; a token that leaves one to the rotation is not a verification row and MUST NOT be dispatched. Never dispatch a line the validator refuses, and report per candidate what each pin rests on and which question the run will not answer. diff --git a/skills/i8-verify-parallel/SKILL.md b/skills/i8-verify-parallel/SKILL.md new file mode 100644 index 0000000000..8853077dee --- /dev/null +++ b/skills/i8-verify-parallel/SKILL.md @@ -0,0 +1,10 @@ +--- +name: i8-verify-parallel +description: > + Verify the branch from both ends at once: trigger the priority line through the i8-ci-trigger-priority skill, keep triaging that run on a loop, and iterate the same roles on the local stack in parallel. Infinito.Nexus specific. +--- + +Follow the instructions from AGENTS.md, then run both verification lanes by following the +procedure in `docs/agents/action/verify-parallel.md` (relative to the Infinito.Nexus repository root) +exactly. That document is the source of truth; this skill only routes you +there. Begin by clarifying every open requirement with the `active-listening` skill (escalating any root-cause or design question you are not ~99% sure of to the `dialectic` skill), then work under the `robot` skill. The CI lane MUST be dispatched by invoking the `i8-ci-trigger-priority` skill rather than by building a line here, so the two skills cannot drift on what a verification row is. The two lanes run at once because they run on different machines; inside the local lane deploys stay serial. Never dispatch a second CI run while the triage loop is armed, and close each question by naming which lane answered it. diff --git a/tasks/utils/setup/group.yml b/tasks/utils/setup/group.yml index 71e1982beb..05f4a4197e 100644 --- a/tasks/utils/setup/group.yml +++ b/tasks/utils/setup/group.yml @@ -1,7 +1,7 @@ --- - name: "🧩 setup {{ role_group }} roles in run_after order" ansible.builtin.include_tasks: ./tasks/utils/setup/role.yml - loop: "{{ lookup('group_roles', role_group) }}" + loop: "{{ lookup('group_roles', role_group, inventory_groups=groups) }}" loop_control: loop_var: role_entry label: "{{ role_entry.app }}" diff --git a/tasks/utils/setup/role.yml b/tasks/utils/setup/role.yml index b618b6195d..26ca195d3b 100644 --- a/tasks/utils/setup/role.yml +++ b/tasks/utils/setup/role.yml @@ -2,6 +2,8 @@ - name: "🔧 setup {{ role_entry.app }}" ansible.builtin.include_role: name: "{{ role_entry.role }}" + vars: + application_id: "{{ role_entry.app }}" when: (role_entry.app | application_allowed(group_names, lookup('deployment').whitelist)) - name: "🚿 flush handlers after {{ role_entry.app }}" diff --git a/templates/roles/README.md.j2.tmpl b/templates/roles/README.md.j2.tmpl index d945704aa1..d74e423b90 100644 --- a/templates/roles/README.md.j2.tmpl +++ b/templates/roles/README.md.j2.tmpl @@ -6,6 +6,8 @@ Required context: application_id Role id used in the Quick Setup commands (e.g. web-app-matrix). + application_deploy_ids That id plus meta/services.yml guide_companions, comma-joined, + because --include filters the inventory literally. application_name Human-readable software name (H1, no role id). application_url Official website; linked on first use in Description. application_description One paragraph about the SOFTWARE (not the role). @@ -67,7 +69,7 @@ Clone, set up the workstation, and deploy {{ application_name }} onto the local git clone {{ software_repository }} cd core make onboard -make compose-deploy mode=reinstall apps={{ application_id }} full_cycle=false +make compose-deploy mode=reinstall apps={{ application_deploy_ids }} full_cycle=false ``` ### Production @@ -82,7 +84,7 @@ bash scripts/install/package.sh make install source scripts/meta/env/load.sh -APP={{ application_id }} +APP={{ application_deploy_ids }} DOMAIN= {% if application_needs_clearnet_resolver -%} DNS= @@ -103,7 +105,7 @@ infinito administration deploy dedicated "$INVENTORY/devices.yml" \ Run the published image to provision the inventory and deploy {{ application_name }} to a managed server (the mounted volume persists the inventory): ```bash -APP={{ application_id }} +APP={{ application_deploy_ids }} HOST= DOMAIN= {% if application_needs_clearnet_resolver -%} diff --git a/tests/integration/ansible/vars/test_application_id_survives_a_foreign_load.py b/tests/integration/ansible/vars/test_application_id_survives_a_foreign_load.py new file mode 100644 index 0000000000..63995f205b --- /dev/null +++ b/tests/integration/ansible/vars/test_application_id_survives_a_foreign_load.py @@ -0,0 +1,163 @@ +"""A role must see its own application_id even after a foreign role's vars load. + +``include_vars`` without ``name:`` puts every key of the loaded file into play +scope as a host fact, and a host fact outranks the role vars a role declares for +itself. The E2E stages load each tested role's ``vars/main.yml`` that way to +render that role's own templates, so the last tested role's ``application_id`` +outlives the loop and every role set up afterwards resolves +``lookup('config', application_id, ...)`` under the wrong id. + +``tasks/utils/setup/role.yml`` defeats that by passing ``application_id`` as an +include param, which outranks the fact. These cases run real Ansible: the first +reproduces the shadowing so the second cannot pass vacuously. +""" + +import shutil +import subprocess +import tempfile +import unittest +from pathlib import Path +from typing import ClassVar + +from utils.cache.files import read_text +from utils.cache.yaml import load_yaml_all_str +from utils.roles.mapping import ROLE_FILE_TASKS_MAIN, ROLE_FILE_VARS_MAIN + +from . import PROJECT_ROOT + +INCLUDE_SITE = PROJECT_ROOT / "tasks/utils/setup/role.yml" +FOREIGN_ID = "web-svc-mirror" +OWN_ID = "dsk-gnt-claude" + +PROBE_TASK = """--- +- name: Report the application_id the role sees + ansible.builtin.copy: + dest: "{{ probe_out }}" + content: "{{ application_id }}" + mode: "0644" +""" + +PLAYBOOK = """--- +- name: Prove an include param survives a foreign vars load + hosts: localhost + gather_facts: false + connection: local + tasks: + - name: Leak a foreign application_id the way the E2E stages do + ansible.builtin.include_vars: + file: "{{ poison_file }}" + + - name: Without the include param the role sees the foreign id + ansible.builtin.include_role: + name: probe + vars: + probe_out: "{{ shadowed_out }}" + + - name: With the include param the role sees its own id + ansible.builtin.include_role: + name: probe + vars: + probe_out: "{{ bound_out }}" + application_id: "{{ own_id }}" +""" + + +def _run_play(workdir: Path) -> tuple[str, str]: + """Render the fixture tree, run the play once, return both observed ids. + + Args: + workdir: an empty directory to build the role and the play in. + + Returns: + ``(shadowed, bound)``: what the role saw without and with the param. + + Raises: + RuntimeError: when ansible-playbook exits non-zero. + """ + role = workdir / "roles" / "probe" + for relative, body in ( + (ROLE_FILE_TASKS_MAIN, PROBE_TASK), + (ROLE_FILE_VARS_MAIN, "---\napplication_id: probe\n"), + ): + target = role / relative + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text(body) + (workdir / "poison.yml").write_text(f"---\napplication_id: {FOREIGN_ID}\n") + (workdir / "play.yml").write_text(PLAYBOOK) + + shadowed = workdir / "shadowed.txt" + bound = workdir / "bound.txt" + result = subprocess.run( + [ + "ansible-playbook", + "-i", + "localhost,", + "-c", + "local", + str(workdir / "play.yml"), + "-e", + f"poison_file={workdir / 'poison.yml'}", + "-e", + f"shadowed_out={shadowed}", + "-e", + f"bound_out={bound}", + "-e", + f"own_id={OWN_ID}", + ], + cwd=str(workdir), + capture_output=True, + text=True, + check=False, + ) + if result.returncode != 0: + raise RuntimeError( + f"ansible-playbook failed\nSTDOUT:\n{result.stdout}\nSTDERR:\n{result.stderr}" + ) + return read_text(str(shadowed)), read_text(str(bound)) + + +class TestApplicationIdSurvivesAForeignLoad(unittest.TestCase): + observed: ClassVar[dict[str, str]] = {} + + @classmethod + def setUpClass(cls) -> None: + """One play answers both cases, so it runs once rather than per case.""" + if not shutil.which("ansible-playbook"): + return + with tempfile.TemporaryDirectory() as tmp: + shadowed, bound = _run_play(Path(tmp)) + cls.observed = {"shadowed": shadowed, "bound": bound} + + @unittest.skipUnless(shutil.which("ansible-playbook"), "ansible-playbook not found") + def test_a_foreign_load_shadows_a_role_that_takes_no_param(self) -> None: + self.assertEqual( + self.observed["shadowed"], + FOREIGN_ID, + "the hazard this guards against must be reproducible, or the case " + "below passes without proving anything", + ) + + @unittest.skipUnless(shutil.which("ansible-playbook"), "ansible-playbook not found") + def test_the_include_param_outranks_the_leaked_fact(self) -> None: + self.assertEqual(self.observed["bound"], OWN_ID) + + def test_the_shared_include_site_binds_the_application_id(self) -> None: + tasks = [ + task + for doc in load_yaml_all_str(read_text(str(INCLUDE_SITE))) + if doc + for task in doc + if isinstance(task, dict) and "ansible.builtin.include_role" in task + ] + self.assertTrue(tasks, f"no include_role task in {INCLUDE_SITE}") + for task in tasks: + self.assertEqual( + (task.get("vars") or {}).get("application_id"), + "{{ role_entry.app }}", + f"{INCLUDE_SITE} must bind application_id, or every role set up " + f"after an E2E stage resolves its config under a foreign id", + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/integration/roles/dashboard/test_integration_scope.py b/tests/integration/roles/dashboard/test_integration_scope.py index a36f9f0b75..c29ad2ee96 100644 --- a/tests/integration/roles/dashboard/test_integration_scope.py +++ b/tests/integration/roles/dashboard/test_integration_scope.py @@ -13,7 +13,7 @@ Why --- The dashboard is the user-facing tile grid. Only web-facing surfaces -belong in it. A system role (``sys-*``), desktop role (``desk-*``), +belong in it. A system role (``sys-*``), desktop role (``dsk-*``), service role (``svc-*``), or driver role (``drv-*``) embedding the dashboard would spam the tile grid with non-clickable infrastructure entries. diff --git a/tests/integration/roles/meta/variants/test_resource_budget.py b/tests/integration/roles/meta/variants/test_resource_budget.py index 3d462bed5d..45485fa4fb 100644 --- a/tests/integration/roles/meta/variants/test_resource_budget.py +++ b/tests/integration/roles/meta/variants/test_resource_budget.py @@ -86,6 +86,7 @@ def _collect_findings(root: Path) -> list[BudgetFinding]: rows=rows, warnings=[], dedup=True, + dynamic_enabled=False, ) totals = aggregate(rows) mem_reservation = totals["mem_reservation_bytes"] or 0 diff --git a/tests/lint/ansible/jinja/test_compose_template_no_bare_dollar.py b/tests/lint/ansible/jinja/test_compose_template_no_bare_dollar.py new file mode 100644 index 0000000000..252ab54bc0 --- /dev/null +++ b/tests/lint/ansible/jinja/test_compose_template_no_bare_dollar.py @@ -0,0 +1,102 @@ +"""Flag a ``$`` in a compose template that docker cannot read as a variable. + +``docker stack config`` refuses the whole stack file with ``invalid +interpolation format for services..command.[]: "..."; you may need to +escape any $ with another $``. ``docker compose config`` accepts the same file +silently and even re-emits the character as ``$$``, so the compose rows of a CI +run are green while every swarm row of the same role dies at validation. Only +the swarm path can catch this, which is why it is caught here instead. + +A regex anchor is the usual source: ``-allowGET=^(/v[0-9.]+)?/(...)$`` ends in a +``$`` that starts no variable name. Escaping it as ``$$`` is correct for both +runtimes: ``docker stack config`` unescapes it to one ``$``, and a container run +under compose receives one ``$`` in its argv. + +Per-line opt-out: ``# nocheck: compose-bare-dollar`` on the offending line or +the immediately preceding non-empty line. +""" + +from __future__ import annotations + +import re +import unittest +from pathlib import Path + +from utils.annotations.suppress import is_suppressed_at +from utils.cache.files import iter_project_files_with_content + +from . import PROJECT_ROOT, is_main_compose_template + +_RULE = "compose-bare-dollar" + +_JINJA = re.compile(r"\{\{.*?\}\}|\{%.*?%\}|\{#.*?#\}", re.DOTALL) +_BARE_DOLLAR = re.compile(r"\$(?![A-Za-z_{$])") + + +def bare_dollar_columns(line: str) -> list[int]: + """The 1-based columns of every ``$`` docker would refuse, Jinja removed. + + Args: + line: one raw template line. + + Returns: + The columns, empty when the line is clean. A ``$`` inside a Jinja + expression is consumed at render time and never reaches docker, so the + expressions are blanked rather than skipped, which keeps the columns + pointing at the raw line. + """ + masked = _JINJA.sub(lambda match: " " * len(match.group(0)), line) + columns = [] + index = 0 + while index < len(masked): + if masked[index] != "$": + index += 1 + continue + if masked.startswith("$$", index): + index += 2 + continue + if _BARE_DOLLAR.match(masked, index): + columns.append(index + 1) + index += 1 + return columns + + +class TestComposeTemplateNoBareDollar(unittest.TestCase): + def test_no_bare_dollar_in_compose_template(self) -> None: + findings: list[tuple[str, int, str]] = [] + for path_str, content in iter_project_files_with_content( + extensions=(".j2",), + exclude_tests=True, + ): + rel = Path(path_str).relative_to(PROJECT_ROOT).as_posix() + if not is_main_compose_template(rel): + continue + lines = content.splitlines() + for idx, line in enumerate(lines): + if not bare_dollar_columns(line): + continue + if is_suppressed_at(lines, idx + 1, _RULE, mode="same-or-above"): + continue + findings.append((rel, idx + 1, line.strip())) + + if findings: + formatted = "\n".join( + f"- {p}:{n}: {s}" + for p, n, s in sorted(set(findings), key=lambda i: (i[0], i[1])) + ) + self.fail( + "Found a `$` in a compose template that docker cannot read as " + "a variable reference. `docker stack config` refuses the whole " + "stack file over it while `docker compose config` accepts it " + "silently, so every swarm row of the role fails and no compose " + "row reports anything.\n\n" + "Fix: double it (`$$`). Both runtimes unescape it back to one " + "`$` in the container. Mark with `# nocheck: " + "compose-bare-dollar` only where docker really is meant to " + "interpolate.\n\n" + f"Offending lines:\n{formatted}" + ) + + +if __name__ == "__main__": # pragma: no cover + unittest.main() diff --git a/tests/lint/ansible/jinja/test_model_backend_via_gateway.py b/tests/lint/ansible/jinja/test_model_backend_via_gateway.py index e3700ab0ce..66b2c76cc2 100644 --- a/tests/lint/ansible/jinja/test_model_backend_via_gateway.py +++ b/tests/lint/ansible/jinja/test_model_backend_via_gateway.py @@ -33,7 +33,7 @@ _RULE = "model-backend-via-gateway" _GATEWAY_ROLE = "svc-ai-litellm" -_PROVIDER_URL = re.compile(r"\{\{[^}]*\b(OLLAMA|LMSTUDIO)_BASE_LOCAL_URL\b") +_PROVIDER_URL = re.compile(r"\{[{%][^}]*\b(OLLAMA|LMSTUDIO)_BASE_LOCAL_URL\b") class TestModelBackendViaGateway(unittest.TestCase): diff --git a/tests/lint/ansible/services/test_addons_playwright_spec.py b/tests/lint/ansible/services/test_addons_playwright_spec.py index 4f04335db5..7d1ae48b82 100644 --- a/tests/lint/ansible/services/test_addons_playwright_spec.py +++ b/tests/lint/ansible/services/test_addons_playwright_spec.py @@ -6,9 +6,9 @@ no per-addon exemption: no mechanism auto-exempt and no ``# nocheck`` escape; every such addon carries its own spec (gated with ``skipUnlessAddonEnabled`` so it skips cleanly when the addon is not enabled in the current variant). -Desktop (``desk-*``) roles are the one categorical exception: they drive no +Desktop (``dsk-*``) roles are the one categorical exception: they drive no web surface a Playwright test could exercise, so they MUST NOT ship addon -specs at all. The lint forbids spec files under ``desk-*`` roles. +specs at all. The lint forbids spec files under ``dsk-*`` roles. """ from __future__ import annotations @@ -19,7 +19,7 @@ from utils.update.addons import iter_addon_files from . import PROJECT_ROOT -DESKTOP_ROLE_PREFIX = "desk-" +DESKTOP_ROLE_PREFIX = "dsk-" class TestAddonsPlaywrightSpec(unittest.TestCase): diff --git a/tests/lint/ansible/variables/test_local_model_quant_pinned.py b/tests/lint/ansible/variables/test_local_model_quant_pinned.py new file mode 100644 index 0000000000..255ca79527 --- /dev/null +++ b/tests/lint/ansible/variables/test_local_model_quant_pinned.py @@ -0,0 +1,107 @@ +"""An ``AI_LOCAL_MODELS`` entry names one quantization in every field. + +Rationale +========= +``roles/svc-ai-lmstudio/tasks/utils/preload/pull.yml`` downloads with +``lms get --yes --gguf @``. Without the ``@`` suffix LM +Studio picks the variant itself, and the ``file``, ``sha256`` and ``bytes`` the +entry declares become a guess about that pick rather than a description of it. + +Two shipped checks measure against those fields, so the guess is not harmless. +``roles/svc-ai-lmstudio/files/test/test.sh`` looks for +``//`` and reports ``is not in the model store`` when the +store holds a different variant. ``pull.yml`` derives the download budget from +``bytes``, so a declaration smaller than the real file cuts the transfer off. + +Run 36107421422 hit both at once: the entry declared ``Q4_K_M`` at 808 MB while +``lms get`` fetched ``Q8_0`` at 1.32 GB. The declared quantization is therefore +pinned here against the other three fields, because an entry whose fields +disagree fails far from the line that introduced the disagreement. + +Per-line opt-out +================ +Add ``# nocheck: local-model-quant-pinned`` on the offending line or the one +above it, with a comment naming why the fields may diverge. +""" + +from __future__ import annotations + +import unittest +from typing import ClassVar + +from utils.annotations.suppress import is_suppressed_at +from utils.cache.files import read_text +from utils.cache.yaml import load_yaml + +from . import PROJECT_ROOT + +_RULE = "local-model-quant-pinned" +_VARS_FILE = PROJECT_ROOT / "group_vars" / "all" / "16_ai.yml" +_KEY = "AI_LOCAL_MODELS" + + +def _entries() -> list[dict]: + declared = load_yaml(_VARS_FILE).get(_KEY) + return [entry for entry in (declared or []) if isinstance(entry, dict)] + + +def _line_of(alias: str) -> int: + for number, line in enumerate(read_text(str(_VARS_FILE)).splitlines(), 1): + if f"alias: {alias}" in line: + return number + return 0 + + +class TestLocalModelQuantPinned(unittest.TestCase): + rel: ClassVar[str] = _VARS_FILE.relative_to(PROJECT_ROOT).as_posix() + + def _unsuppressed(self) -> list[dict]: + return [ + entry + for entry in _entries() + if not is_suppressed_at( + str(_VARS_FILE), _line_of(str(entry.get("alias", ""))), _RULE + ) + ] + + def test_every_entry_pins_a_quantization(self) -> None: + missing = [ + entry.get("alias") + for entry in self._unsuppressed() + if not str(entry.get("quant") or "").strip() + ] + self.assertFalse( + missing, + f"{self.rel}: {missing} declare no 'quant', so lms get picks the " + "variant and the declared file, sha256 and bytes describe a guess", + ) + + def test_the_declared_file_carries_the_pinned_quantization(self) -> None: + drifted = [ + (entry.get("alias"), entry.get("quant"), entry.get("file")) + for entry in self._unsuppressed() + if str(entry.get("quant") or "").strip() + and str(entry.get("quant")).lower() + not in str(entry.get("file") or "").lower() + ] + self.assertFalse( + drifted, + f"{self.rel}: {drifted} name a quantization the file does not " + "carry; the store check looks for that file and reports it missing", + ) + + def test_the_url_resolves_the_declared_file(self) -> None: + drifted = [ + (entry.get("alias"), entry.get("file")) + for entry in self._unsuppressed() + if not str(entry.get("url") or "").endswith(str(entry.get("file") or "")) + ] + self.assertFalse( + drifted, + f"{self.rel}: {drifted} declare a url that does not end in the " + "declared file, so the digest is pinned against a different blob", + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/lint/filesystem/env/test_env_value_space_quoted.py b/tests/lint/filesystem/env/test_env_value_space_quoted.py new file mode 100644 index 0000000000..35e1a7188c --- /dev/null +++ b/tests/lint/filesystem/env/test_env_value_space_quoted.py @@ -0,0 +1,85 @@ +"""An env value that renders more than one word must be quoted. + +``KEY=one two`` is not an assignment of two words. The shell reads it as +``KEY=one`` prefixed to the command ``two``, so the variable is set for that +command only and is unset afterwards. Under ``set -u`` the consumer then dies +on ``KEY: unbound variable`` while the rendered file plainly shows the value, +which is how ``AGENT_PLATFORMS_OFFERED=hermes openclaw`` took down the compose +rows of svc-ai-agent-broker and web-app-openwebui in run 35889056152. + +Only ``join`` with a space separator is flagged. A comma or a semicolon +produces one word and is safe unquoted, and ``to_json`` output is already +covered by the repository's quoting convention. + +``dotenv_quote`` does NOT fix this. In swarm mode it delivers the value +verbatim, because ``docker stack deploy`` keeps the quotes it would add and +the container would then read them as part of the value. + +Per-line opt-out: ``# nocheck: env-space-value-unquoted``. +""" + +from __future__ import annotations + +import re +import unittest +from pathlib import Path + +from utils.annotations.suppress import is_suppressed_at +from utils.cache.files import PROJECT_ROOT, iter_project_files_with_content + +_RULE = "env-space-value-unquoted" + +_ASSIGNMENT = re.compile(r"^(?P[A-Za-z_][A-Za-z0-9_]*)=(?P.*)$") +_SPACE_JOIN = re.compile(r"join\(\s*(['\"]) \1\s*\)") +_QUOTED = re.compile(r"^(['\"]).*\1$") + + +def renders_unquoted_words(line: str) -> bool: + """Whether this assignment can put a space into an unquoted value.""" + match = _ASSIGNMENT.match(line) + if not match: + return False + value = match.group("value").strip() + if not _SPACE_JOIN.search(value): + return False + return not _QUOTED.match(value) + + +class TestEnvValueSpaceQuoted(unittest.TestCase): + def test_no_unquoted_space_joined_env_value(self) -> None: + findings: list[tuple[str, int, str]] = [] + for path_str, content in iter_project_files_with_content( + extensions=(".j2",), + exclude_tests=True, + ): + rel = Path(path_str).relative_to(PROJECT_ROOT).as_posix() + if not rel.endswith("env.j2"): + continue + lines = content.splitlines() + for idx, line in enumerate(lines): + if not renders_unquoted_words(line): + continue + if is_suppressed_at(lines, idx + 1, _RULE, mode="same-or-above"): + continue + findings.append((rel, idx + 1, line.strip())) + + if findings: + formatted = "\n".join( + f"- {p}:{n}: {s}" + for p, n, s in sorted(set(findings), key=lambda i: (i[0], i[1])) + ) + self.fail( + "Found env assignments whose value joins on a space without " + "being quoted. The shell treats the second word as a command " + "and leaves the variable unset, so a consumer running under " + "`set -u` dies on an unbound variable that the rendered file " + "appears to contain.\n\n" + "Fix: join on a comma and split on one in the consumer. " + "`| dotenv_quote` does not help: swarm passes the value " + "verbatim, so the quotes would land inside it.\n\n" + f"Offending lines:\n{formatted}" + ) + + +if __name__ == "__main__": # pragma: no cover + unittest.main() diff --git a/tests/unit/python/cli/administration/deploy/ci/test_timings.py b/tests/unit/python/cli/administration/deploy/ci/test_timings.py new file mode 100644 index 0000000000..3f03447c79 --- /dev/null +++ b/tests/unit/python/cli/administration/deploy/ci/test_timings.py @@ -0,0 +1,124 @@ +from __future__ import annotations + +import unittest +from typing import ClassVar + +from cli.administration.deploy.ci import timings +from tests.utils.ci.job_names import deploy_job_name +from utils.github.variant.pools import DISTROS, FILESYSTEMS + + +def _job( + name: str, + *, + minutes: float, + conclusion: str = "success", + status: str = "completed", +) -> dict: + return { + "name": name, + "status": status, + "conclusion": conclusion, + "startedAt": "2026-09-23T10:00:00Z", + "completedAt": f"2026-09-23T{10 + int(minutes) // 60:02d}:{int(minutes) % 60:02d}:00Z", + } + + +class TestSamples(unittest.TestCase): + """Which jobs a duration is read off, and which are left out.""" + + def test_a_failed_job_is_not_measured(self) -> None: + jobs = [ + _job(deploy_job_name("docker", "web-app-matomo", "0"), minutes=10), + _job( + deploy_job_name("swarm", "web-app-matomo", "0"), + minutes=90, + conclusion="failure", + ), + ] + grouped = timings.samples(jobs) + self.assertEqual( + sorted(grouped["mode"]), + ["compose"], + "a failed job's duration measures the failure, not the axis", + ) + + def test_a_job_without_both_timestamps_is_left_out(self) -> None: + job = _job(deploy_job_name("docker", "web-app-matomo", "0"), minutes=10) + del job["completedAt"] + self.assertEqual(timings.samples([job])["mode"], {}) + + def test_a_title_that_carries_no_row_is_left_out(self) -> None: + self.assertEqual( + timings.samples([_job("🧪 Unit tests", minutes=5)])["mode"], {} + ) + + def test_every_axis_of_one_title_is_attributed(self) -> None: + job = _job( + deploy_job_name( + "swarm", + "web-app-matomo", + "0", + tor=True, + distro=DISTROS[0], + filesystem=FILESYSTEMS[0], + ), + minutes=30, + ) + grouped = timings.samples([job]) + self.assertEqual(sorted(grouped["mode"]), ["swarm"]) + self.assertEqual(sorted(grouped["tor"]), ["tor"]) + self.assertEqual(sorted(grouped["distro"]), [DISTROS[0]]) + self.assertEqual(sorted(grouped["filesystem"]), [FILESYSTEMS[0]]) + + +class TestRanking(unittest.TestCase): + """Cheapest first, and only for values that were seen often enough.""" + + _GROUPED: ClassVar[dict[str, dict[str, list[float]]]] = { + "mode": { + "swarm": [600.0, 600.0, 600.0], + "compose": [300.0, 300.0, 300.0], + "host": [1.0], + }, + "tor": {}, + "distro": {}, + "filesystem": {}, + } + + def test_the_cheapest_measured_value_wins(self) -> None: + self.assertEqual(timings.fastest(self._GROUPED, "mode"), "compose") + + def test_a_single_observation_does_not_steer_the_run(self) -> None: + self.assertNotIn( + "host", + [ + value + for value, _median, _count in timings.ranking(self._GROUPED, "mode") + ], + "one job is a queue wait, not a measurement", + ) + + def test_the_median_ignores_one_long_tail(self) -> None: + grouped = { + "mode": {"compose": [300.0, 300.0, 9000.0], "swarm": [400.0] * 3}, + "tor": {}, + "distro": {}, + "filesystem": {}, + } + self.assertEqual(timings.fastest(grouped, "mode"), "compose") + + def test_among_restricts_the_choice(self) -> None: + self.assertEqual( + timings.fastest(self._GROUPED, "mode", among=("swarm",)), "swarm" + ) + + def test_nothing_measured_enough_returns_none(self) -> None: + self.assertIsNone( + timings.fastest(self._GROUPED, "mode", minimum=99), + "None tells the caller to leave the axis to the rotation", + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/unit/python/cli/administration/deploy/ci/test_trigger.py b/tests/unit/python/cli/administration/deploy/ci/test_trigger.py index 6dde4f43c4..e999a1efed 100644 --- a/tests/unit/python/cli/administration/deploy/ci/test_trigger.py +++ b/tests/unit/python/cli/administration/deploy/ci/test_trigger.py @@ -98,6 +98,38 @@ class TestTriggerMain(unittest.TestCase): calls, [("entry-manual-steer.yml", "feature/x", "__ALL__", "", {}, "o/r")] ) + def test_priority_is_dispatched_with_the_axis_overrides(self) -> None: + rc, calls = self._run( + [ + "--priority", + "web-app-b#1@compose web-app-a#0@swarm", + "--workspace", + "false", + "--chunk-gate", + "false", + "--filesystem", + "btrfs", + ] + ) + self.assertEqual(rc, 0) + self.assertEqual(calls[0][3], "web-app-a#0@swarm web-app-b#1@compose") + self.assertEqual( + calls[0][4], + {"workspace": "false", "chunk_gate": "false", "filesystem": "btrfs"}, + ) + + def test_an_unusable_token_is_refused_before_anything_is_dispatched(self) -> None: + with mock.patch.object( + trigger.validate, "problems", return_value=(["bad token"], []) + ): + rc, calls = self._run(["--priority", "web-app-a#9"]) + self.assertEqual(rc, 1) + self.assertEqual(calls, [], "a refused line must not reach the workflow") + + def test_priority_and_failed_cannot_both_build_the_line(self) -> None: + with self.assertRaises(SystemExit): + self._run(["--failed", "--priority", "web-app-a#0"]) + def test_apps_explicit_list(self) -> None: rc, calls = self._run(["--apps", "web-app-a web-app-b"]) self.assertEqual(rc, 0) diff --git a/tests/unit/python/cli/meta/roles/services/called/test_called.py b/tests/unit/python/cli/meta/roles/services/called/test_called.py index 37a6bf6954..09aff5aff9 100644 --- a/tests/unit/python/cli/meta/roles/services/called/test_called.py +++ b/tests/unit/python/cli/meta/roles/services/called/test_called.py @@ -43,7 +43,7 @@ class TestCategoriesOf(unittest.TestCase): self.assertEqual(categories_of("sys-ctl-hlth-csp"), {"sys", "sys-ctl"}) def test_single_segment(self) -> None: - self.assertEqual(categories_of("desk"), {"desk"}) + self.assertEqual(categories_of("dsk"), {"dsk"}) def test_empty(self) -> None: self.assertEqual(categories_of(""), set()) diff --git a/tests/unit/python/group_vars/test_litellm_chat_model.py b/tests/unit/python/group_vars/test_litellm_chat_model.py index 67a8c4c902..f97a33d99c 100644 --- a/tests/unit/python/group_vars/test_litellm_chat_model.py +++ b/tests/unit/python/group_vars/test_litellm_chat_model.py @@ -1,3 +1,4 @@ +import ast import re import unittest from pathlib import Path @@ -47,14 +48,27 @@ class TestLitellmChatModel(unittest.TestCase): cls.env.filters["bool"] = _ansible_bool cls.source = load_yaml(_AI_VARS) - def _render(self, name, *, roles, api_key, preload_models=(), lmstudio_models=()): + def _render( + self, + name, + *, + roles, + api_key, + preload_models=(), + lmstudio_models=(), + remote_aliases=None, + ): return ( self.env.from_string(self.source[name]) .render( LITELLM_BACKEND_ROLES=list(roles), LITELLM_OLLAMA_BACKEND=str("svc-ai-ollama" in roles), LITELLM_LMSTUDIO_BACKEND=str("svc-ai-lmstudio" in roles), - AI_REMOTE_ALIASES=(["openrouter/auto"] if api_key else []), + AI_REMOTE_ALIASES=( + list(remote_aliases) + if remote_aliases is not None + else (["openrouter/auto"] if api_key else []) + ), lookup=_stub_lookup( [{"alias": alias, "name": alias} for alias in preload_models], [ @@ -82,6 +96,43 @@ class TestLitellmChatModel(unittest.TestCase): self.assertEqual(model, "openrouter/auto") self.assertEqual(served, "True") + def test_a_declared_mock_answers_when_no_backend_preloaded_anything(self): + model = self._render( + "LITELLM_CHAT_MODEL", + roles=["svc-ai-ollama"], + api_key="", + preload_models=[], + remote_aliases=["mock/deterministic"], + ) + self.assertEqual( + model, + "mock/deterministic", + "a CI round that deploys the backend without preloading must fall to the mock", + ) + + def test_a_preloaded_model_wins_over_the_mock(self): + model = self._render( + "LITELLM_CHAT_MODEL", + roles=["svc-ai-ollama"], + api_key="", + preload_models=["qwen2.5:0.5b"], + remote_aliases=["mock/deterministic"], + ) + self.assertEqual( + model, + "qwen2.5:0.5b", + "the round that loads a real model must prove that model, not the mock", + ) + + def test_without_a_mock_the_chain_is_unchanged(self): + self.assertEqual( + self._render( + "LITELLM_CHAT_MODEL", roles=["web-app-mattermost"], api_key="sk-test" + ), + "openrouter/auto", + "production declares no mock, so it must resolve exactly as before", + ) + def test_lmstudio_wins_over_the_openrouter_fallback(self): model, served = self._both( roles=["svc-ai-lmstudio"], api_key="", lmstudio_models=["qwen2.5:0.5b"] @@ -149,5 +200,90 @@ class TestLitellmChatModel(unittest.TestCase): self.assertEqual(bool(model), served == "True") +class TestAiRemoteAliases(unittest.TestCase): + models: ClassVar[list] + template: ClassVar[str] + + @classmethod + def setUpClass(cls): + cls.models = [ + {"alias": "openrouter/auto", "provider": "openrouter"}, + {"alias": "openrouter/ternary-bonsai-2-27b", "provider": "openrouter"}, + {"alias": "openai/gpt-4o-mini", "provider": "openai"}, + ] + cls.template = load_yaml(_AI_VARS)["AI_REMOTE_ALIASES"] + + def _render(self, enabled): + models = self.models + + def lookup(kind, *terms): + if kind == "config": + self.assertEqual( + terms, ("svc-ai-litellm", "services.litellm.remote_models") + ) + return models + if kind == "api_enabled": + return terms[0] in enabled + raise AssertionError(f"unexpected lookup({kind!r}, {terms!r})") + + env = Environment(undefined=StrictUndefined, autoescape=False) # noqa: S701 - renders a Python list literal, not markup + return ast.literal_eval( + env.from_string(self.template).render( + lookup=lookup, AI_MOCK_PROVIDER="mock" + ) + ) + + def test_every_model_of_an_enabled_provider_is_listed(self): + self.assertEqual( + self._render({"openrouter"}), + ["openrouter/auto", "openrouter/ternary-bonsai-2-27b"], + ) + + def test_no_enabled_provider_lists_nothing(self): + self.assertEqual(self._render(set()), []) + + +class TestAiAgentModel(unittest.TestCase): + """Every agent the broker starts asks the gateway for this alias, and + roles/svc-ai-agent-broker/tasks/00_core.yml stops the deploy when it is + empty. A deployment that carries neither a mock nor a backend nor a + provider key resolves to '', so the two sources are pinned together. + """ + + template: ClassVar[str] + + @classmethod + def setUpClass(cls): + cls.template = load_yaml(_AI_VARS)["AI_AGENT_MODEL"] + + def _render(self, *, mock_aliases, chat_model): + env = Environment(undefined=StrictUndefined, autoescape=False) # noqa: S701 - renders a bare alias, not markup + return env.from_string(self.template).render( + AI_MOCK_ALIASES=list(mock_aliases), LITELLM_CHAT_MODEL=chat_model + ) + + def test_a_declared_mock_is_preferred_over_the_served_model(self): + self.assertEqual( + self._render( + mock_aliases=["mock/deterministic"], chat_model="qwen2.5:0.5b" + ), + "mock/deterministic", + ) + + def test_without_a_mock_the_served_model_answers(self): + self.assertEqual( + self._render(mock_aliases=[], chat_model="qwen2.5:0.5b"), "qwen2.5:0.5b" + ) + + def test_neither_a_mock_nor_a_served_model_names_nothing(self): + self.assertEqual( + self._render(mock_aliases=[], chat_model=""), + "", + "a broker deployed without a backend role and without a provider " + "key has no model to start its agents on, which 00_core.yml turns " + "into a stopped deploy rather than agents that all answer 400", + ) + + if __name__ == "__main__": unittest.main() diff --git a/tests/unit/python/plugins/action/test_package_install.py b/tests/unit/python/plugins/action/test_package_install.py index 4635705b38..ade6efd690 100644 --- a/tests/unit/python/plugins/action/test_package_install.py +++ b/tests/unit/python/plugins/action/test_package_install.py @@ -61,12 +61,12 @@ class TestOwningRole(unittest.TestCase): def test_foreign_role_id_fails(self): with self.assertRaises(AnsibleActionFail) as caught: - self._spec_for("nfs-ganesha", "desk-micro") + self._spec_for("nfs-ganesha", "dsk-micro") self.assertIn("svc-storage-nfs-server", str(caught.exception)) def test_unknown_id_fails(self): with self.assertRaises(AnsibleActionFail): - self._spec_for("nope", "desk-micro") + self._spec_for("nope", "dsk-micro") def test_own_id_resolves(self): with mock.patch( @@ -78,7 +78,7 @@ class TestOwningRole(unittest.TestCase): def test_shared_id_resolves_for_any_role(self): with mock.patch("plugins.action.package_install.resolve", return_value="spec"): - spec = self._spec_for("git", "desk-micro") + spec = self._spec_for("git", "dsk-micro") self.assertEqual(spec, "spec") def test_no_role_context_skips_the_check(self): diff --git a/tests/unit/python/plugins/filter/get/test_all_invokable_apps.py b/tests/unit/python/plugins/filter/get/test_all_invokable_apps.py index 891cac8397..dc765bd203 100644 --- a/tests/unit/python/plugins/filter/get/test_all_invokable_apps.py +++ b/tests/unit/python/plugins/filter/get/test_all_invokable_apps.py @@ -30,7 +30,7 @@ class TestGetAllInvokableApps(unittest.TestCase): "util": { "title": "utils", "invokable": False, - "desk": {"title": "Desktop utils", "invokable": True}, + "dsk": {"title": "Desktop utils", "invokable": True}, }, } } diff --git a/tests/unit/python/plugins/filter/get/test_deployment_types_from_groups.py b/tests/unit/python/plugins/filter/get/test_deployment_types_from_groups.py index 57f152be94..575af281ff 100644 --- a/tests/unit/python/plugins/filter/get/test_deployment_types_from_groups.py +++ b/tests/unit/python/plugins/filter/get/test_deployment_types_from_groups.py @@ -22,7 +22,7 @@ class TestGetDeploymentTypesFromGroups(unittest.TestCase): return [ "web-app", "web-svc", - "desk", + "dsk", "update", ] @@ -34,7 +34,7 @@ class TestGetDeploymentTypesFromGroups(unittest.TestCase): get_deployment_types_from_groups( [ "web-app-nextcloud", - "desk-firefox", + "dsk-firefox", ] ), ["server", "workstation"], @@ -48,7 +48,7 @@ class TestGetDeploymentTypesFromGroups(unittest.TestCase): get_deployment_types_from_groups( [ "web-svc-logout", - "desk-nextcloud", + "dsk-nextcloud", ] ), ["server", "workstation"], diff --git a/tests/unit/python/plugins/filter/get/test_entity_name.py b/tests/unit/python/plugins/filter/get/test_entity_name.py index 6c1ad2f97c..4cb076cb75 100644 --- a/tests/unit/python/plugins/filter/get/test_entity_name.py +++ b/tests/unit/python/plugins/filter/get/test_entity_name.py @@ -22,9 +22,7 @@ class TestGetEntityNameFilter(unittest.TestCase): "app": {"title": "Applications", "invokable": True}, "svc": {"title": "Services", "invokable": True}, }, - "util": { - "desk": {"dev": {"title": "Dev Utilities", "invokable": True}} - }, + "util": {"dsk": {"dev": {"title": "Dev Utilities", "invokable": True}}}, "sys": { "ctl": { "bkp": {"title": "Backup", "invokable": True}, diff --git a/tests/unit/python/plugins/filter/litellm/__init__.py b/tests/unit/python/plugins/filter/litellm/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/unit/python/plugins/filter/litellm/test_model_routes.py b/tests/unit/python/plugins/filter/litellm/test_model_routes.py new file mode 100644 index 0000000000..237572857a --- /dev/null +++ b/tests/unit/python/plugins/filter/litellm/test_model_routes.py @@ -0,0 +1,118 @@ +from __future__ import annotations + +import unittest + +from plugins.filter.litellm.model_routes import litellm_model_routes + +OLLAMA_URL = "http://ollama:11434" +LMSTUDIO_URL = "http://lmstudio:1234" + +SHARED = {"alias": "qwen2.5:0.5b", "name": "qwen2.5-0.5b-instruct", "context": 32768} +LMSTUDIO_ONLY = {"alias": "mistral:latest", "name": "mistral-7b-instruct-v0.3"} +REMOTE = { + "alias": "openrouter/auto", + "model": "openrouter/openrouter/auto", + "provider": "openrouter", +} +MOCK = { + "alias": "mock/deterministic", + "provider": "mock", + "response": "pong", + "context": 131072, +} + + +def routes(ollama=(), lmstudio=(), declared=()): + return litellm_model_routes( + list(ollama), + list(lmstudio), + list(declared), + ollama_url=OLLAMA_URL, + lmstudio_url=LMSTUDIO_URL, + max_tokens=512, + timeout=60, + mock_provider=MOCK["provider"], + ) + + +def by_alias(built): + return {route["alias"]: route for route in built} + + +class TestPublicationOrder(unittest.TestCase): + """The first route is what LITELLM_CHAT_MODEL names, so order is a contract.""" + + def test_a_local_model_outranks_a_declared_one(self) -> None: + built = routes(ollama=[SHARED], declared=[MOCK, REMOTE]) + self.assertEqual(next(route["alias"] for route in built), SHARED["alias"]) + + def test_declared_models_keep_their_declaration_order(self) -> None: + built = routes(declared=[MOCK, REMOTE]) + self.assertEqual( + [route["alias"] for route in built], [MOCK["alias"], REMOTE["alias"]] + ) + + def test_nothing_declared_publishes_nothing(self) -> None: + self.assertEqual(routes(), []) + + +class TestLocalBackends(unittest.TestCase): + def test_ollama_is_called_through_its_own_scheme(self) -> None: + params = by_alias(routes(ollama=[SHARED]))[SHARED["alias"]]["params"] + self.assertEqual(params["model"], f"ollama/{SHARED['alias']}") + self.assertEqual(params["api_base"], OLLAMA_URL) + + def test_a_declared_window_reaches_the_backend_and_the_model_info(self) -> None: + route = by_alias(routes(ollama=[SHARED]))[SHARED["alias"]] + self.assertEqual(route["params"]["num_ctx"], SHARED["context"]) + self.assertEqual(route["context"], SHARED["context"]) + + def test_a_model_without_a_window_sets_no_num_ctx(self) -> None: + route = by_alias(routes(lmstudio=[LMSTUDIO_ONLY]))[LMSTUDIO_ONLY["alias"]] + self.assertNotIn("num_ctx", route["params"]) + self.assertIsNone(route["context"]) + + def test_lmstudio_publishes_the_window_it_declares(self) -> None: + route = by_alias(routes(lmstudio=[SHARED]))[SHARED["alias"]] + self.assertEqual( + route["context"], + SHARED["context"], + "the LM Studio branch used to drop the window its README promised", + ) + + def test_an_alias_both_backends_serve_is_published_once_by_ollama(self) -> None: + built = routes(ollama=[SHARED], lmstudio=[SHARED, LMSTUDIO_ONLY]) + self.assertEqual( + [route["alias"] for route in built], + [SHARED["alias"], LMSTUDIO_ONLY["alias"]], + ) + self.assertEqual( + by_alias(built)[SHARED["alias"]]["params"]["api_base"], OLLAMA_URL + ) + + +class TestDeclaredModels(unittest.TestCase): + def test_a_provider_route_reads_its_key_from_the_environment(self) -> None: + params = by_alias(routes(declared=[REMOTE]))[REMOTE["alias"]]["params"] + self.assertEqual(params["api_key"], "os.environ/OPENROUTER_API_KEY") + self.assertEqual(params["model"], REMOTE["model"]) + + def test_a_mock_answers_from_its_own_response(self) -> None: + params = by_alias(routes(declared=[MOCK]))[MOCK["alias"]]["params"] + self.assertEqual(params["mock_response"], MOCK["response"]) + + def test_a_mock_names_no_upstream_to_call(self) -> None: + params = by_alias(routes(declared=[MOCK]))[MOCK["alias"]]["params"] + self.assertNotIn("api_base", params) + self.assertNotIn( + "timeout", params, "a mock cannot time out against a backend it never calls" + ) + + def test_a_mock_publishes_its_window(self) -> None: + self.assertEqual( + by_alias(routes(declared=[MOCK]))[MOCK["alias"]]["context"], MOCK["context"] + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/unit/python/plugins/filter/test_dotenv_quote.py b/tests/unit/python/plugins/filter/test_dotenv_quote.py index 9d61bddbdd..904e0bd89d 100644 --- a/tests/unit/python/plugins/filter/test_dotenv_quote.py +++ b/tests/unit/python/plugins/filter/test_dotenv_quote.py @@ -94,14 +94,14 @@ class TestDotenvQuoteModeAware(unittest.TestCase): def test_swarm_non_string_is_stringified(self): self.assertEqual(self._render(42, mode="swarm"), "42") - def test_swarm_quotes_value_with_interior_whitespace(self): + def test_swarm_passes_a_value_with_spaces_through_unchanged(self): self.assertEqual( - self._render("My Org Helpdesk", mode="swarm"), '"My Org Helpdesk"' + self._render('{"hermes": {"group": "/x"}}', mode="swarm"), + '{"hermes": {"group": "/x"}}', + "swarm delivers the value verbatim, so quoting it makes the " + "container read the quotes as part of the JSON and parse a string", ) - def test_swarm_whitespace_value_escapes_quotes_not_dollars(self): - self.assertEqual(self._render('a "b" $c', mode="swarm"), '"a \\"b\\" $c"') - def test_compose_mode_quotes_like_default(self): self.assertEqual(self._render("abc", mode="compose"), '"abc"') self.assertEqual(self._render("$ecret", mode="compose"), '"$$ecret"') diff --git a/tests/unit/python/plugins/lookup/test_ca_injected.py b/tests/unit/python/plugins/lookup/test_ca_injected.py index 5680300214..ce658492f2 100644 --- a/tests/unit/python/plugins/lookup/test_ca_injected.py +++ b/tests/unit/python/plugins/lookup/test_ca_injected.py @@ -47,12 +47,23 @@ class TestCaInjectedLookup(unittest.TestCase): def setUpClass(cls): cls.mod = _load_lookup() - def _resolve(self, *, domains, enabled, mode, app="web-app-moodle"): + def _resolve( + self, + *, + domains, + enabled, + mode, + app="web-app-moodle", + services=None, + tls_mode="self_signed", + ): calls: list[tuple[str, list]] = [] def fake_get(name, **_kwargs): def run(args, variables=None): calls.append((name, list(args))) + if name == "config": + return [services or {}] if name == "domains": return [domains] return [enabled if args[1] == "enabled" else mode] @@ -63,7 +74,8 @@ class TestCaInjectedLookup(unittest.TestCase): plugin._loader = None plugin._templar = _DummyTemplar() with mock.patch.object(self.mod.lookup_loader, "get", side_effect=fake_get): - result = plugin.run([app], variables={}) + variables = {} if tls_mode is None else {"TLS_MODE": tls_mode} + result = plugin.run([app], variables=variables) return result[0], calls def test_a_self_signed_app_with_a_domain_gets_the_ca(self): @@ -93,7 +105,64 @@ class TestCaInjectedLookup(unittest.TestCase): def test_an_app_without_a_domain_never_asks_about_tls(self): verdict, calls = self._resolve(domains={}, enabled=True, mode="self_signed") self.assertIs(verdict, False) - self.assertEqual([name for name, _ in calls], ["domains"]) + self.assertEqual([name for name, _ in calls], ["domains", "config"]) + + def test_a_domainless_ca_client_gets_the_ca_on_the_deployment_mode(self): + verdict, calls = self._resolve( + domains={}, + enabled=True, + mode="self_signed", + services={"agent-broker": {"ca_client": True}, "socket-proxy": {}}, + ) + self.assertIs( + verdict, + True, + "a container that only calls the self-signed endpoint has no domain " + "of its own, and without the CA every outbound call fails the " + "handshake", + ) + self.assertEqual([name for name, _ in calls], ["domains", "config"]) + + def test_a_ca_client_with_a_domain_follows_its_own_tls_mode(self): + verdict, calls = self._resolve( + domains={"web-app-moodle": "moodle.example"}, + enabled=True, + mode="letsencrypt", + services={"agent-broker": {"ca_client": True}}, + tls_mode="self_signed", + ) + self.assertIs( + verdict, + False, + "an app that has a domain is served over its own resolved mode, which " + "may override the deployment default; the ca_client shortcut must not " + "answer for it on the global TLS_MODE", + ) + self.assertEqual([name for name, _ in calls], ["domains", "tls", "tls"]) + + def test_a_ca_client_gets_nothing_when_the_ca_is_not_self_signed(self): + verdict, _ = self._resolve( + domains={}, + enabled=True, + mode="self_signed", + services={"agent-broker": {"ca_client": True}}, + tls_mode="letsencrypt", + ) + self.assertIs(verdict, False) + + def test_a_ca_client_refuses_an_absent_tls_mode(self): + with self.assertRaises( + AnsibleError, + msg="a missing mode read as 'not self signed' withholds the CA in " + "silence, and every handshake then fails far from here", + ): + self._resolve( + domains={}, + enabled=True, + mode="self_signed", + services={"agent-broker": {"ca_client": True}}, + tls_mode=None, + ) def test_the_term_is_required(self): plugin = self.mod.LookupModule() diff --git a/tests/unit/python/plugins/lookup/test_compose_file_args.py b/tests/unit/python/plugins/lookup/test_compose_file_args.py index 8ad1a27a86..38116e0d0b 100644 --- a/tests/unit/python/plugins/lookup/test_compose_file_args.py +++ b/tests/unit/python/plugins/lookup/test_compose_file_args.py @@ -30,6 +30,14 @@ class _DomainsResolveStub: return [(variables or {}).get("domains", {})] +class _CaInjectedStub: + def __init__(self, injected: bool): + self._injected = injected + + def run(self, terms, variables=None, **kwargs): + return [self._injected] + + class TestComposeFArgs(unittest.TestCase): def setUp(self): self.m = _load_module( @@ -48,18 +56,48 @@ class TestComposeFArgs(unittest.TestCase): }, } - def _loader_dispatch(self, tls_stub): - # "domains" resolves through variables['domains'] to keep tests hermetic; - # "tls" resolves through the per-test stub. + def _loader_dispatch(self, tls_stub, ca_injected=None): + verdict = tls_stub.run(None)[0] + injected = ( + (verdict["enabled"] and verdict["mode"] == "self_signed") + if ca_injected is None + else ca_injected + ) + def _get(name, *args, **kwargs): if name == "domains": return _DomainsResolveStub() if name == "tls": return tls_stub + if name == "ca_injected": + return _CaInjectedStub(injected) raise AssertionError(f"unexpected lookup name: {name}") return _get + def test_a_domainless_ca_client_still_gets_the_ca_override_file(self): + with ( + patch.object( + self.m.lookup_loader, + "get", + side_effect=self._loader_dispatch( + _TlsResolveStub(False, "off"), ca_injected=True + ), + ), + patch.object( + self.m, "get_docker_paths", side_effect=self._stub_get_docker_paths + ), + patch.object(self.m, "_role_provides_override", return_value=False), + ): + out = self.lookup.run(["web-app-a"], variables={**self.vars, "domains": {}}) + self.assertIn( + "/x/compose.ca.override.yml", + out[0], + "ca_injected grants a domainless ca_client the override and the " + "handler writes it; a domain gate here leaves that file on disk " + "while the container runs without a trust store", + ) + def _stub_get_docker_paths(self, application_id: str, base_dir: str) -> dict: self.assertEqual(application_id, "web-app-a") self.assertEqual(base_dir, "/x/") diff --git a/tests/unit/python/plugins/lookup/test_group_roles.py b/tests/unit/python/plugins/lookup/test_group_roles.py new file mode 100644 index 0000000000..67c7b406d0 --- /dev/null +++ b/tests/unit/python/plugins/lookup/test_group_roles.py @@ -0,0 +1,84 @@ +"""The stage loop skips roles no host carries, and gives every host one list. + +Dropping what nobody deploys is free: 132 of 139 entries were skipped one task +at a time on a single-app deploy, 278 task deltas, 41.8 seconds measured on +2026-09-23. + +Dropping what *this* host does not carry is not free, and these cases forbid +it. 8aea5a9b35 measured that a per-host condition above the loop makes ansible +insert blocks in host order rather than loop order, after which +svc-storage-nfs-client mounted an export svc-storage-nfs-server had not +deployed yet (run 31645153977). The list therefore has to come out identical +for every host, and per-host narrowing stays in the body's ``when``. +""" + +from __future__ import annotations + +import unittest +from typing import ClassVar +from unittest import mock + +from plugins.lookup.group_roles import LookupModule + +ENTRIES = [ + {"role": "web-app-alpha", "app": "alpha"}, + {"role": "web-app-beta", "app": "beta"}, + {"role": "web-app-gamma", "app": "gamma"}, +] + + +def run(**kwargs): + with mock.patch( + "plugins.lookup.group_roles.ordered_roles", return_value=list(ENTRIES) + ): + return LookupModule().run(["web-app"], **kwargs)[0] + + +class TestWithoutAnInventory(unittest.TestCase): + def test_no_groups_returns_the_whole_group(self) -> None: + self.assertEqual(run(), ENTRIES) + + +class TestDroppingWhatNobodyDeploys(unittest.TestCase): + def test_a_role_no_host_carries_is_dropped(self) -> None: + selected = run(inventory_groups={"alpha": ["h1"], "beta": []}) + self.assertEqual([entry["app"] for entry in selected], ["alpha"]) + + def test_an_empty_inventory_deploys_nothing(self) -> None: + self.assertEqual(run(inventory_groups={}), []) + + def test_run_after_order_survives_the_filter(self) -> None: + selected = run(inventory_groups={"gamma": ["h1"], "alpha": ["h1"]}) + self.assertEqual( + [entry["app"] for entry in selected], + ["alpha", "gamma"], + "run_after order is the contract; the inventory's key order is not", + ) + + +class TestSameListForEveryHost(unittest.TestCase): + """The property 8aea5a9b35 paid for; losing it inverts deploy order.""" + + GROUPS: ClassVar[dict] = {"alpha": ["h1"], "beta": ["h2"], "gamma": ["h1", "h2"]} + + def test_a_role_only_one_host_carries_still_enters_the_loop(self) -> None: + selected = [entry["app"] for entry in run(inventory_groups=self.GROUPS)] + self.assertEqual( + selected, + ["alpha", "beta", "gamma"], + "beta belongs to h2 only; dropping it for h1 would give the two " + "hosts different loops and let host order decide item order", + ) + + def test_the_lookup_reads_no_per_host_variable(self) -> None: + selected = run(inventory_groups=self.GROUPS, group_names=["alpha"]) + self.assertEqual( + [entry["app"] for entry in selected], + ["alpha", "beta", "gamma"], + "a host's own groups must not reach this lookup; narrowing to one " + "host belongs in the body's when", + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/unit/python/plugins/lookup/test_roles_with_service.py b/tests/unit/python/plugins/lookup/test_roles_with_service.py index 6fa2d754e4..6e760c6e60 100644 --- a/tests/unit/python/plugins/lookup/test_roles_with_service.py +++ b/tests/unit/python/plugins/lookup/test_roles_with_service.py @@ -245,6 +245,29 @@ class RolesWithServiceLookupTests(unittest.TestCase): } self.assertEqual(self._run(["dashboard"], applications), [[]]) + def test_include_headless_keeps_a_role_without_canonical_domain(self): + applications = { + "svc-ai-foo": { + "services": { + "litellm": {"enabled": True, "shared": True}, + }, + "domains": {}, + }, + } + self.assertEqual( + self._run(["litellm"], applications, include_headless=True), + [ + [ + { + "id": "svc-ai-foo", + "canonical_domain": "", + "canonical_url": "", + "iframe": True, + } + ] + ], + ) + def test_canonical_as_string_is_accepted(self): applications = { "web-app-foo": { diff --git a/tests/unit/python/roles/svc-ai-agent-broker/__init__.py b/tests/unit/python/roles/svc-ai-agent-broker/__init__.py new file mode 100644 index 0000000000..8ee16d7567 --- /dev/null +++ b/tests/unit/python/roles/svc-ai-agent-broker/__init__.py @@ -0,0 +1,3 @@ +from pathlib import Path + +PROJECT_ROOT: Path = Path(__file__).resolve().parents[5] diff --git a/tests/unit/python/roles/svc-ai-agent-broker/files/__init__.py b/tests/unit/python/roles/svc-ai-agent-broker/files/__init__.py new file mode 100644 index 0000000000..9f69c70b47 --- /dev/null +++ b/tests/unit/python/roles/svc-ai-agent-broker/files/__init__.py @@ -0,0 +1,3 @@ +from pathlib import Path + +PROJECT_ROOT: Path = Path(__file__).resolve().parents[6] diff --git a/tests/unit/python/roles/svc-ai-agent-broker/files/test_access.py b/tests/unit/python/roles/svc-ai-agent-broker/files/test_access.py new file mode 100644 index 0000000000..f02932cabc --- /dev/null +++ b/tests/unit/python/roles/svc-ai-agent-broker/files/test_access.py @@ -0,0 +1,58 @@ +from __future__ import annotations + +import sys +import unittest + +from . import PROJECT_ROOT + +sys.path.insert(0, str(PROJECT_ROOT / "roles/svc-ai-agent-broker/files/python")) + +import access + +GROUP = "/roles/web-app-hermes/agent-user" + + +class FakeKeycloak(access.Keycloak): + def __init__(self, users, groups): + super().__init__( + "https://auth", "realm", "admin", "pw", cache_seconds=60, timeout=5 + ) + self.users = users + self.groups = groups + self.lookups = 0 + + def _admin_token(self): + return "token" + + def _get(self, path, query): + self.lookups += 1 + if path == "/users": + return self.users.get(query["email"], []) + user_id = path.split("/")[2] + return [{"path": path} for path in self.groups.get(user_id, [])] + + +class TestAllows(unittest.TestCase): + def test_only_a_member_of_the_platform_group_is_allowed(self): + keycloak = FakeKeycloak( + {"a@x": [{"id": "1"}], "b@x": [{"id": "2"}]}, + {"1": [GROUP], "2": ["/roles/web-app-hermes/mcp"]}, + ) + self.assertTrue(keycloak.allows("A@x", GROUP)) + self.assertFalse(keycloak.allows("b@x", GROUP)) + + def test_an_unknown_or_empty_email_is_refused(self): + keycloak = FakeKeycloak({}, {}) + self.assertFalse(keycloak.allows("", GROUP)) + self.assertFalse(keycloak.allows("nobody@x", GROUP)) + + def test_an_answer_is_reused_within_the_cache_window(self): + keycloak = FakeKeycloak({"a@x": [{"id": "1"}]}, {"1": [GROUP]}) + keycloak.allows("a@x", GROUP) + lookups = keycloak.lookups + keycloak.allows("a@x", GROUP) + self.assertEqual(keycloak.lookups, lookups) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/unit/python/roles/svc-ai-agent-broker/files/test_agents.py b/tests/unit/python/roles/svc-ai-agent-broker/files/test_agents.py new file mode 100644 index 0000000000..e10b671dd4 --- /dev/null +++ b/tests/unit/python/roles/svc-ai-agent-broker/files/test_agents.py @@ -0,0 +1,205 @@ +from __future__ import annotations + +import json +import sys +import unittest + +import yaml + +from . import PROJECT_ROOT + +sys.path.insert(0, str(PROJECT_ROOT / "roles/svc-ai-agent-broker/files/python")) + +import agents +import engine + +PLATFORMS = { + "hermes": { + "image": "nousresearch/hermes-agent:v1", + "command": ["gateway"], + "env": {"API_SERVER_PORT": "8642"}, + "key_env": "API_SERVER_KEY", + "config_path": "/opt/data/config.yaml", + "data": "/opt/data", + "user": "", + "cpus": "2.0", + "mem_limit": 4294967296, + "mem_reservation": 2147483648, + "pids_limit": "2048", + "port": "8642", + "health": "/health", + "request_model": "hermes-agent", + } +} + + +class FakeEngine: + def __init__(self, runtime="runsc"): + self.calls = [] + self.runtime = runtime + + def image_config(self, ref): + return ["/entrypoint.sh"], ["serve"] + + def call(self, method, path, body=None, query=None, expect=(200,)): + self.calls.append((method, path)) + return {"Id": "abc", "Name": "/agent", "HostConfig": {"Runtime": self.runtime}} + + +class FakeBackend: + def __init__(self, details): + self.details = details + self.stopped = [] + + def list_agents(self): + return list(self.details.values()) + + def find(self, name): + return self.details.get(name) + + def is_running(self, detail): + return detail["running"] + + def started_at(self, detail): + return detail["started_at"] + + def stop(self, detail): + self.stopped.append(detail["Name"].lstrip("/")) + detail["running"] = False + + +def make_agents(backend=None, idle_stop=True, context=0): + return agents.Agents( + backend=backend, + engine=FakeEngine(), + platforms=PLATFORMS, + self_container="self", + broker_alias="agent-broker", + relay_url="http://agent-broker:8080/llm/v1", + model="qwen2.5:0.5b", + context=context, + idle_stop=idle_stop, + idle_seconds=60, + max_running=2, + start_timeout=10, + ) + + +class TestNaming(unittest.TestCase): + def test_one_owner_one_name_two_owners_two_names(self): + self.assertEqual( + agents.agent_name("hermes", "u1"), agents.agent_name("hermes", "u1") + ) + self.assertNotEqual( + agents.agent_name("hermes", "u1"), agents.agent_name("hermes", "u2") + ) + self.assertNotEqual( + agents.agent_name("hermes", "u1"), agents.agent_name("openclaw", "u1") + ) + + +class TestConfigs(unittest.TestCase): + def test_hermes_points_its_custom_provider_at_the_relay(self): + config = yaml.safe_load( + agents.hermes_config("m", "http://relay/v1", "k", 0) + ) # nocheck: direct-yaml - parses this test's own render + self.assertEqual( + config["model"], + { + "default": "m", + "provider": "custom", + "base_url": "http://relay/v1", + "api_key": "k", + }, + ) + + def test_openclaw_enables_chat_completions_and_the_relay_provider(self): + config = json.loads(agents.openclaw_config("m", "http://relay/v1", "k", 0)) + self.assertTrue( + config["gateway"]["http"]["endpoints"]["chatCompletions"]["enabled"] + ) + self.assertEqual( + config["models"]["providers"]["broker"]["baseUrl"], "http://relay/v1" + ) + self.assertEqual(config["agents"]["defaults"]["model"]["primary"], "broker/m") + + +class TestContextWindow(unittest.TestCase): + def test_a_known_window_reaches_both_agent_configs(self): + hermes = yaml.safe_load( + agents.hermes_config("m", "http://relay/v1", "k", 32768) + ) # nocheck: direct-yaml - parses this test's own render + self.assertEqual(hermes["model"]["context_length"], 32768) + openclaw = json.loads( + agents.openclaw_config("m", "http://relay/v1", "k", 32768) + ) + entry = openclaw["models"]["providers"]["broker"]["models"][0] + self.assertEqual(entry["contextWindow"], 32768) + + def test_an_unknown_window_leaves_the_agents_to_their_own_default(self): + hermes = yaml.safe_load( + agents.hermes_config("m", "http://relay/v1", "k", 0) + ) # nocheck: direct-yaml - parses this test's own render + self.assertNotIn("context_length", hermes["model"]) + openclaw = json.loads(agents.openclaw_config("m", "http://relay/v1", "k", 0)) + self.assertNotIn( + "contextWindow", openclaw["models"]["providers"]["broker"]["models"][0] + ) + + +class TestReap(unittest.TestCase): + def detail(self, name, running=True, started="2020-01-01T00:00:00.000000000Z"): + return {"Name": f"/{name}", "running": running, "started_at": started} + + def test_an_idle_agent_is_stopped(self): + backend = FakeBackend({"agent-hermes-1": self.detail("agent-hermes-1")}) + make_agents(backend).reap() + self.assertEqual(backend.stopped, ["agent-hermes-1"]) + + def test_idle_stop_false_stops_nothing(self): + backend = FakeBackend({"agent-hermes-1": self.detail("agent-hermes-1")}) + make_agents(backend, idle_stop=False).reap() + self.assertEqual(backend.stopped, []) + + def test_an_agent_serving_a_request_survives_its_idle_time(self): + backend = FakeBackend({"agent-hermes-1": self.detail("agent-hermes-1")}) + broker = make_agents(backend) + broker._busy["agent-hermes-1"] = 1 + broker.reap() + self.assertEqual(backend.stopped, []) + + def test_a_prompt_that_lands_during_the_sweep_keeps_the_agent(self): + backend = FakeBackend({"agent-hermes-1": self.detail("agent-hermes-1")}) + broker = make_agents(backend) + original = broker._lock_for + + def touch_then_lock(name): + broker.end(name) + return original(name) + + broker._lock_for = touch_then_lock + broker.reap() + self.assertEqual(backend.stopped, []) + + +class TestSpec(unittest.TestCase): + def test_the_config_is_written_before_the_image_entrypoint_runs(self): + spec = make_agents()._spec("hermes", "owner", "secret") + self.assertEqual(spec["entrypoint"][:2], ["/bin/sh", "-c"]) + self.assertEqual(spec["cmd"], ["/entrypoint.sh", "gateway"]) + self.assertIn("API_SERVER_KEY=secret", spec["env"]) + self.assertIn(f"{agents.CONFIG_PATH_ENV}=/opt/data/config.yaml", spec["env"]) + self.assertEqual(spec["labels"][engine.LABEL_OWNER], "owner") + self.assertEqual(spec["nano_cpus"], 2_000_000_000) + + def test_a_numeric_command_argument_reaches_the_engine_as_a_string(self): + broker = make_agents() + broker.platforms = { + "hermes": {**PLATFORMS["hermes"], "command": ["run", "--port", 18789]} + } + spec = broker._spec("hermes", "owner", "secret") + self.assertEqual(spec["cmd"], ["/entrypoint.sh", "run", "--port", "18789"]) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/unit/python/roles/svc-ai-agent-broker/files/test_engine.py b/tests/unit/python/roles/svc-ai-agent-broker/files/test_engine.py new file mode 100644 index 0000000000..ac2e09e41b --- /dev/null +++ b/tests/unit/python/roles/svc-ai-agent-broker/files/test_engine.py @@ -0,0 +1,177 @@ +from __future__ import annotations + +import http.server +import socketserver +import sys +import tempfile +import threading +import unittest +from pathlib import Path + +from . import PROJECT_ROOT + +sys.path.insert(0, str(PROJECT_ROOT / "roles/svc-ai-agent-broker/files/python")) + +import engine + + +class RecordingEngine: + def __init__(self, runtime): + self.runtime = runtime + self.calls = [] + + def call(self, method, path, body=None, query=None, expect=(200,)): + self.calls.append((method, path, body)) + if method == "GET" and path in ("/containers/json", "/services"): + return [] + return {"Id": "abc", "Name": "/agent", "HostConfig": {"Runtime": self.runtime}} + + +SPEC = { + "name": "agent-hermes-1", + "network": "agent-hermes-1", + "volume": "agent-hermes-1", + "image": "hermes:1", + "entrypoint": ["/bin/sh", "-c", 'exec "$@"', "agent"], + "cmd": ["gateway"], + "env": ["A=1"], + "labels": {engine.LABEL_OWNER: "u1"}, + "data": "/opt/data", + "user": "", + "nano_cpus": 1, + "memory": 2, + "memory_reservation": 1, + "pids": 3, +} + + +class TestComposeBackend(unittest.TestCase): + def test_an_agent_on_the_host_kernel_is_stopped_and_refused(self): + fake = RecordingEngine(runtime="runc") + backend = engine.ComposeBackend(fake, "runsc") + with self.assertRaises(engine.EngineError): + backend.start({"Id": "abc"}) + self.assertIn(("POST", "/containers/abc/stop", None), fake.calls) + + def test_a_created_agent_is_pinned_to_the_runtime_and_its_own_volume(self): + fake = RecordingEngine(runtime="runsc") + engine.ComposeBackend(fake, "runsc").create(SPEC, "net-1") + body = next( + body for method, path, body in fake.calls if path == "/containers/create" + ) + self.assertEqual(body["HostConfig"]["Runtime"], "runsc") + self.assertEqual( + body["HostConfig"]["Mounts"], + [{"Type": "volume", "Source": "agent-hermes-1", "Target": "/opt/data"}], + ) + self.assertNotIn("Privileged", body["HostConfig"]) + self.assertNotIn("Binds", body["HostConfig"]) + self.assertEqual(list(body["NetworkingConfig"]["EndpointsConfig"]), ["net-1"]) + + +class _Handler(http.server.BaseHTTPRequestHandler): + def do_GET(self): + status, body = (200, b'{"ok": true}') if self.path == "/_ping" else (403, b"no") + self.send_response(status) + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def log_message(self, *_args): + return + + +class _UnixHTTPServer(socketserver.ThreadingMixIn, socketserver.UnixStreamServer): + daemon_threads = True + + +class TestUnixTransport(unittest.TestCase): + def setUp(self): + self.directory = tempfile.TemporaryDirectory() + self.path = str(Path(self.directory.name) / "engine.sock") + self.server = _UnixHTTPServer(self.path, _Handler) + threading.Thread(target=self.server.serve_forever, daemon=True).start() + + def tearDown(self): + self.server.shutdown() + self.server.server_close() + self.directory.cleanup() + + def test_a_json_answer_travels_over_the_unix_socket(self): + self.assertEqual( + engine.Engine(self.path, timeout=5).call("GET", "/_ping"), {"ok": True} + ) + + def test_a_refused_path_raises(self): + with self.assertRaises(engine.EngineError): + engine.Engine(self.path, timeout=5).call("GET", "/containers/x/exec") + + +class _ConnectEngine(engine.Engine): + """Engine whose connect answers 404 for the first ``misses`` attempts.""" + + def __init__(self, misses): + self.misses = misses + self.attempts = 0 + self.slept = 0.0 + + def call(self, method, path, body=None, query=None, expect=(200,), timeout=None): + if method == "GET": + return {"Containers": {}} + self.attempts += 1 + if self.attempts <= self.misses: + raise engine.EngineError(f"POST {path} -> 404: not found", 404) + return None + + +class TestConnect(unittest.TestCase): + def setUp(self): + self.real_sleep = engine.time.sleep + engine.time.sleep = lambda seconds: None + + def tearDown(self): + engine.time.sleep = self.real_sleep + + def test_a_swarm_overlay_is_joined_once_its_allocator_realizes_it(self): + fake = _ConnectEngine(misses=3) + fake.connect("net-1", "broker", "broker-alias") + self.assertEqual(fake.attempts, 4) + + def test_a_network_that_stays_missing_is_not_waited_on_forever(self): + fake = _ConnectEngine(misses=10**6) + with self.assertRaises(engine.EngineError): + fake.connect("net-1", "broker", "broker-alias", realize_timeout=0) + self.assertEqual(fake.attempts, 1) + + def test_a_refusal_that_is_not_a_missing_network_is_raised_at_once(self): + fake = _ConnectEngine(misses=0) + + def deny(method, path, body=None, query=None, expect=(200,), timeout=None): + if method == "GET": + return {"Containers": {}} + raise engine.EngineError(f"POST {path} -> 403: denied", 403) + + fake.call = deny + with self.assertRaises(engine.EngineError) as caught: + fake.connect("net-1", "broker", "broker-alias") + self.assertEqual(caught.exception.status, 403) + + +class TestSwarmBackend(unittest.TestCase): + def test_a_service_is_placed_on_the_sandbox_nodes_and_starts_scaled_down(self): + fake = RecordingEngine(runtime="runsc") + engine.SwarmBackend(fake, "node.labels.kata-capable == true").create( + SPEC, "net-1" + ) + body = next( + body for method, path, body in fake.calls if path == "/services/create" + ) + self.assertEqual( + body["TaskTemplate"]["Placement"]["Constraints"], + ["node.labels.kata-capable == true"], + ) + self.assertEqual(body["Mode"], {"Replicated": {"Replicas": 0}}) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/unit/python/roles/svc-ai-litellm/files/test_ensure_keys.py b/tests/unit/python/roles/svc-ai-litellm/files/test_ensure_keys.py index 98d9d6440c..51dc54adf0 100644 --- a/tests/unit/python/roles/svc-ai-litellm/files/test_ensure_keys.py +++ b/tests/unit/python/roles/svc-ai-litellm/files/test_ensure_keys.py @@ -116,20 +116,19 @@ class TestHttpCall(unittest.TestCase): class TestMain(unittest.TestCase): ENV: ClassVar[dict[str, str]] = { "LITELLM_PORT": "4000", - "LITELLM_MK": "sk-master", - "LITELLM_KEYS_PAYLOAD": json.dumps( - [ - {"alias": "web-app-flowise", "key": "sk-new"}, - {"alias": "web-app-hermes", "key": "sk-known"}, - ] - ), + "LITELLM_MASTER_KEY": "sk-master", } + PAYLOAD: ClassVar[list[dict[str, str]]] = [ + {"alias": "web-app-flowise", "key": "sk-new"}, + {"alias": "web-app-hermes", "key": "sk-known"}, + ] def test_only_the_created_key_is_announced_as_changed(self): proxy = RecordingProxy({"sk-known"}, {"web-app-flowise"}) buffer = io.StringIO() with ( patch.dict("os.environ", self.ENV, clear=True), + patch.object(MODULE.sys, "stdin", io.StringIO(json.dumps(self.PAYLOAD))), patch.object(MODULE, "http_call", return_value=proxy), redirect_stdout(buffer), ): @@ -139,6 +138,7 @@ class TestMain(unittest.TestCase): def test_the_master_key_reaches_the_proxy_as_a_bearer_header(self): with ( patch.dict("os.environ", self.ENV, clear=True), + patch.object(MODULE.sys, "stdin", io.StringIO(json.dumps(self.PAYLOAD))), patch.object(MODULE, "http_call") as http_call, ): http_call.return_value = RecordingProxy({"sk-new", "sk-known"}, set()) diff --git a/tests/unit/python/roles/svc-ai-litellm/files/test_measure_speed.py b/tests/unit/python/roles/svc-ai-litellm/files/test_measure_speed.py new file mode 100644 index 0000000000..af71c8754d --- /dev/null +++ b/tests/unit/python/roles/svc-ai-litellm/files/test_measure_speed.py @@ -0,0 +1,242 @@ +"""The deploy measures a rate per model, because no catalogue carries one. + +Speed is a property of this machine under this load, not of the model, so the +numbers the router ranks on come from requests the deploy actually sent. These +cases pin what the script does with them: it measures only a model that has no +rate yet, discards the cold first sample, takes a median rather than a mean, +refuses to rewrite a rate that barely moved, and never drops a model because +one deploy could not reach it. +""" + +from __future__ import annotations + +import importlib.util +import io +import json +import unittest +from contextlib import redirect_stdout +from typing import ClassVar, Self +from unittest.mock import patch + +from . import PROJECT_ROOT + +SCRIPT_PATH = PROJECT_ROOT / "roles/svc-ai-litellm/files/python/measure_speed.py" + +SAMPLE_TIMEOUT = 60.0 +WARMUP_TIMEOUT = 240.0 + +ENVIRONMENT = { + "LITELLM_MASTER_KEY": "sk-master-key", + "LITELLM_PORT": "4000", + "LITELLM_SAMPLES": "12", + "LITELLM_TIMEOUT": str(SAMPLE_TIMEOUT), + "LITELLM_WARMUP": str(WARMUP_TIMEOUT), + "LITELLM_EXCLUDE": "auto", + "LITELLM_REMEASURE": "", + "LITELLM_PREVIOUS": "{}", +} + + +class FakeResponse: + def __init__(self, body: bytes) -> None: + self.body = body + + def __enter__(self) -> Self: + return self + + def __exit__(self, *exc_info: object) -> bool: + return False + + def read(self) -> bytes: + return self.body + + +class Gateway: + """A gateway that answers /v1/models and bills each completion a duration. + + Args: + served: the model ids /v1/models publishes. + durations: model -> the seconds each successive call takes; the last + value repeats once the list runs out. + tokens: completion tokens every answer reports. + """ + + def __init__(self, served, durations, tokens=60) -> None: + self.served = served + self.durations = durations + self.tokens = tokens + self.clock = 0.0 + self.calls: dict[str, int] = {} + self.timeouts: dict[str, list[float]] = {} + + def urlopen(self, request, timeout=None) -> FakeResponse: + if request.full_url.endswith("/v1/models"): + return FakeResponse( + json.dumps({"data": [{"id": name} for name in self.served]}).encode() + ) + model = json.loads(request.data.decode())["model"] + index = self.calls.get(model, 0) + self.calls[model] = index + 1 + self.timeouts.setdefault(model, []).append(timeout) + schedule = self.durations[model] + self.clock += schedule[min(index, len(schedule) - 1)] + return FakeResponse( + json.dumps({"usage": {"completion_tokens": self.tokens}}).encode() + ) + + def monotonic(self) -> float: + return self.clock + + +def run(gateway, **environment): + """The script against *gateway*, returning the JSON it printed.""" + spec = importlib.util.spec_from_file_location("measure_speed", SCRIPT_PATH) + module = importlib.util.module_from_spec(spec) + with ( + patch.dict("os.environ", {**ENVIRONMENT, **environment}, clear=False), + patch("urllib.request.urlopen", gateway.urlopen), + patch("time.monotonic", gateway.monotonic), + redirect_stdout(io.StringIO()) as captured, + ): + spec.loader.exec_module(module) + module.main() + return json.loads(captured.getvalue()) + + +class TestMeasureSpeed(unittest.TestCase): + def test_a_rate_is_output_tokens_over_the_wall_clock(self) -> None: + gateway = Gateway(["m"], {"m": [2.0]}, tokens=60) + self.assertEqual(run(gateway)["m"], 30.0) + + def test_it_sends_the_configured_number_of_requests_plus_one_warm_up(self) -> None: + gateway = Gateway(["m"], {"m": [2.0]}) + run(gateway, LITELLM_SAMPLES="12") + self.assertEqual(gateway.calls["m"], 13) + + def test_a_single_sample_run_still_discards_the_cold_load(self) -> None: + gateway = Gateway(["m"], {"m": [600.0, 2.0]}, tokens=60) + self.assertEqual( + run(gateway, LITELLM_SAMPLES="1")["m"], + 30.0, + "a one-sample run that kept its cold load would report load time as " + "a rate, ranking a small model below a larger one", + ) + + def test_the_cold_first_sample_is_discarded(self) -> None: + gateway = Gateway(["m"], {"m": [600.0, 2.0]}, tokens=60) + self.assertEqual( + run(gateway)["m"], + 30.0, + "a backend pays its load cost on the first call, which is not the " + "rate any later caller sees", + ) + + def test_the_cold_first_sample_gets_the_longer_budget(self) -> None: + gateway = Gateway(["m"], {"m": [2.0]}, tokens=60) + run(gateway) + self.assertEqual( + gateway.timeouts["m"][:2], + [WARMUP_TIMEOUT, SAMPLE_TIMEOUT], + "the cold call pays the model load, so holding it to the sample " + "budget would time out exactly the models worth measuring", + ) + + def test_one_slow_sample_does_not_move_the_median(self) -> None: + schedule = [600.0] + [2.0] * 5 + [120.0] + [2.0] * 5 + gateway = Gateway(["m"], {"m": schedule}, tokens=60) + self.assertEqual(run(gateway)["m"], 30.0) + + def test_a_single_sample_keeps_its_own_measurement(self) -> None: + gateway = Gateway(["m"], {"m": [2.0]}, tokens=60) + self.assertEqual( + run(gateway, LITELLM_SAMPLES="1")["m"], + 30.0, + "discarding the only sample would return nothing; at this setting " + "the number is worth less than the proof that the path ran", + ) + + def test_the_router_alias_is_not_measured(self) -> None: + gateway = Gateway(["auto", "m"], {"m": [2.0]}) + self.assertEqual(list(run(gateway)), ["m"]) + + def test_every_excluded_alias_is_left_unranked(self) -> None: + gateway = Gateway(["auto", "mock/deterministic", "m"], {"m": [2.0]}) + measured = run(gateway, LITELLM_EXCLUDE="auto,mock/deterministic") + self.assertEqual( + list(measured), + ["m"], + "a mock generates nothing, so its rate is the proxy's echo latency; " + "ranking on it routes every request to a fixture", + ) + self.assertEqual(gateway.calls.get("mock/deterministic", 0), 0) + + def test_every_served_model_gets_its_own_rate(self) -> None: + gateway = Gateway(["slow", "fast"], {"slow": [6.0], "fast": [1.0]}, tokens=60) + self.assertEqual(run(gateway), {"slow": 10.0, "fast": 60.0}) + + +class TestOnlyUnrankedIsMeasured(unittest.TestCase): + """The traffic is the live surface, so a stored rate sends none of it.""" + + STORED: ClassVar[dict] = {"m": 31.0} + + def test_a_model_that_already_has_a_rate_is_not_measured(self) -> None: + gateway = Gateway(["m"], {"m": [2.0]}, tokens=60) + measured = run(gateway, LITELLM_PREVIOUS=json.dumps(self.STORED)) + self.assertEqual(gateway.calls.get("m", 0), 0) + self.assertEqual(measured["m"], 31.0) + + def test_an_unranked_model_beside_a_ranked_one_is_measured(self) -> None: + gateway = Gateway(["m", "new"], {"m": [2.0], "new": [1.0]}, tokens=60) + measured = run(gateway, LITELLM_PREVIOUS=json.dumps(self.STORED)) + self.assertEqual(gateway.calls.get("m", 0), 0) + self.assertEqual(measured, {"m": 31.0, "new": 60.0}) + + def test_the_remeasure_switch_sends_the_requests_again(self) -> None: + gateway = Gateway(["m"], {"m": [2.0]}, tokens=60) + run(gateway, LITELLM_PREVIOUS=json.dumps(self.STORED), LITELLM_REMEASURE="1") + self.assertEqual(gateway.calls["m"], 13) + + +class TestRemeasuring(unittest.TestCase): + """Only a re-measurement can move a stored number, and barely moving does not.""" + + def remeasure(self, gateway, stored): + return run(gateway, LITELLM_PREVIOUS=json.dumps(stored), LITELLM_REMEASURE="1") + + def test_a_rate_within_tolerance_keeps_the_stored_number(self) -> None: + gateway = Gateway(["m"], {"m": [2.0]}, tokens=60) + self.assertEqual( + self.remeasure(gateway, {"m": 31.0})["m"], + 31.0, + "an unclamped result rewrites the config and restarts the gateway " + "on every deploy, because no two runs measure the same machine", + ) + + def test_a_rate_beyond_tolerance_replaces_the_stored_number(self) -> None: + gateway = Gateway(["m"], {"m": [2.0]}, tokens=60) + self.assertEqual(self.remeasure(gateway, {"m": 10.0})["m"], 30.0) + + def test_a_model_that_cannot_be_reached_keeps_its_stored_rate(self) -> None: + gateway = Gateway(["m"], {"m": [2.0]}) + listing = gateway.urlopen + + def refuse(request, timeout=None): + if request.full_url.endswith("/v1/models"): + return listing(request, timeout) + raise OSError("connection refused") + + gateway.urlopen = refuse + self.assertEqual( + self.remeasure(gateway, {"m": 12.5})["m"], + 12.5, + "one bad minute must not unrank a model against its peers", + ) + + def test_a_model_that_answers_no_tokens_is_left_unranked(self) -> None: + gateway = Gateway(["m"], {"m": [2.0]}, tokens=0) + self.assertEqual(run(gateway), {}) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/unit/python/roles/svc-ai-litellm/files/test_probe_routing.py b/tests/unit/python/roles/svc-ai-litellm/files/test_probe_routing.py index b2073361b1..b4f93fda40 100644 --- a/tests/unit/python/roles/svc-ai-litellm/files/test_probe_routing.py +++ b/tests/unit/python/roles/svc-ai-litellm/files/test_probe_routing.py @@ -4,6 +4,10 @@ from __future__ import annotations import importlib.util import unittest +from typing import ClassVar + +from utils.cache.yaml import load_yaml +from utils.roles.mapping import ROLE_FILE_META_SERVICES from . import PROJECT_ROOT @@ -212,5 +216,148 @@ class TestNoBackendDeployment(unittest.TestCase): self.assertEqual(verdict(chat_model=ALIAS), []) +class TestRouterAlias(unittest.TestCase): + """The router alias is served by no backend, so it is accounted separately.""" + + def test_a_served_router_alias_is_not_read_as_an_unbacked_route(self) -> None: + self.assertEqual( + verdict( + served={ALIAS, "auto"}, + expected=[ALIAS, "auto"], + chat_model=ALIAS, + chat_model_served=True, + ollama_enabled=True, + router_alias="auto", + ), + [], + ) + + def test_a_declared_router_alias_the_gateway_withholds_fails(self) -> None: + failures = verdict( + served={ALIAS}, + expected=[ALIAS], + chat_model=ALIAS, + chat_model_served=True, + ollama_enabled=True, + router_alias="auto", + ) + self.assertTrue( + any("router alias 'auto'" in failure for failure in failures), failures + ) + + +class TestRouteVerdict(unittest.TestCase): + """An answer is only evidence of routing when it names a model that fits.""" + + WINDOWS: ClassVar[dict] = {"small": 4096, "medium": 16384, "large": 65536} + + MOCKS: ClassVar[tuple] = ("medium",) + + def verdict(self, served, needed=5461, counted=0, sent=0, mocks=()): + return probe.route_verdict( + "auto", self.WINDOWS, served, needed, counted, sent, mocks + ) + + def test_a_backend_that_evaluated_less_than_was_sent_fails(self) -> None: + self.assertIn( + "silently dropped the rest", + self.verdict("medium", counted=4098, sent=8192), + "ollama truncates a prompt past the trained window instead of " + "refusing it, and reports only the tokens it kept, so the declared " + "window cannot show the loss", + ) + + def test_a_backend_that_evaluated_everything_passes(self) -> None: + self.assertEqual(self.verdict("large", counted=8227, sent=8192), "") + + def test_a_backend_that_reports_no_usage_is_not_accused(self) -> None: + self.assertEqual(self.verdict("large", counted=0, sent=8192), "") + + def test_a_tokenizer_disagreeing_by_a_few_tokens_is_not_truncation(self) -> None: + self.assertEqual( + self.verdict("large", counted=8185, sent=8192), + "", + "a chat template shifts the total either way, so an exact " + "comparison would fail a healthy route", + ) + + def test_a_mock_is_not_accused_of_truncating(self) -> None: + self.assertEqual( + self.verdict("medium", counted=10, sent=8192, mocks=self.MOCKS), + "", + "litellm answers a mock route from a canned string and reports its " + "own constant prompt token count, so the number carries nothing to " + "compare the prompt against", + ) + + def test_a_mock_is_still_held_to_its_declared_window(self) -> None: + self.assertIn( + "did not exclude it", + self.verdict("small", needed=8192, counted=10, sent=8192, mocks=("small",)), + "the exemption covers the truncation number only; a mock routed a " + "prompt its declared window cannot hold is still a routing failure, " + "which is why the exemption sits after the window check", + ) + + def test_an_alias_outside_the_mock_list_is_still_accused(self) -> None: + self.assertIn( + "silently dropped the rest", + self.verdict("medium", counted=10, sent=8192, mocks=("other",)), + ) + + def test_a_model_whose_window_holds_the_prompt_passes(self) -> None: + self.assertEqual(self.verdict("medium"), "") + + def test_answering_as_the_alias_itself_fails(self) -> None: + self.assertIn("did not rewrite", self.verdict("auto")) + + def test_naming_no_model_fails(self) -> None: + self.assertIn("without naming a model", self.verdict("")) + + def test_a_window_too_small_for_the_prompt_fails(self) -> None: + self.assertIn("did not exclude it", self.verdict("small")) + + def test_an_undeclared_window_is_not_second_guessed(self) -> None: + self.assertEqual(self.verdict("openrouter/auto"), "") + + def test_a_provider_prefixed_answer_is_matched_to_its_window(self) -> None: + self.assertIn( + "did not exclude it", + self.verdict("ollama/small"), + "litellm names the resolved model, not the alias, so a literal " + "lookup would miss the window and pass every answer", + ) + + def test_a_provider_prefixed_answer_that_fits_passes(self) -> None: + self.assertEqual(self.verdict("ollama/large"), "") + + +SHIPPED_MIN_CHARS_PER_TOKEN = load_yaml( + PROJECT_ROOT / "roles/svc-ai-litellm" / ROLE_FILE_META_SERVICES +)["litellm"]["router"]["min_chars_per_token"] + + +class TestOversizedPrompt(unittest.TestCase): + """The prompt has to clear the smallest window by the hook's own floor.""" + + def test_the_demand_exceeds_the_window_it_targets(self) -> None: + _, needed = probe.oversized_prompt(4096, SHIPPED_MIN_CHARS_PER_TOKEN) + self.assertGreater(needed, 4096) + + def test_the_demand_stays_inside_the_next_window_up(self) -> None: + _, needed = probe.oversized_prompt(4096, SHIPPED_MIN_CHARS_PER_TOKEN) + self.assertLess(needed, 65536) + + def test_the_demand_matches_what_the_prompt_really_tokenizes_to(self) -> None: + prompt, needed = probe.oversized_prompt(4096, SHIPPED_MIN_CHARS_PER_TOKEN) + self.assertGreaterEqual( + needed, + len(prompt) // 2, + "'x ' costs one token per two characters on every tokenizer served " + "here, so a demand computed on a larger divisor understates the " + "prompt and admits a model that cannot hold it", + ) + + if __name__ == "__main__": unittest.main() diff --git a/tests/unit/python/roles/svc-ai-litellm/files/test_warm_model.py b/tests/unit/python/roles/svc-ai-litellm/files/test_warm_model.py index 94ae49d7ad..63d7913e7a 100644 --- a/tests/unit/python/roles/svc-ai-litellm/files/test_warm_model.py +++ b/tests/unit/python/roles/svc-ai-litellm/files/test_warm_model.py @@ -13,7 +13,7 @@ from . import PROJECT_ROOT SCRIPT_PATH = PROJECT_ROOT / "roles/svc-ai-litellm/files/python/warm_model.py" ENVIRONMENT = { - "LITELLM_MK": "sk-master-key", + "LITELLM_MASTER_KEY": "sk-master-key", "LITELLM_PORT": "4000", "LITELLM_MODEL": "smollm2:135m", } diff --git a/tests/unit/python/roles/svc-ai-litellm/templates/test_config_model_list.py b/tests/unit/python/roles/svc-ai-litellm/templates/test_config_model_list.py index e1728d0e72..73a3990f76 100644 --- a/tests/unit/python/roles/svc-ai-litellm/templates/test_config_model_list.py +++ b/tests/unit/python/roles/svc-ai-litellm/templates/test_config_model_list.py @@ -9,16 +9,37 @@ template is rendered here rather than read. from __future__ import annotations +import json import unittest +from pathlib import Path import yaml from jinja2 import Environment, StrictUndefined +from plugins.filter.litellm.model_routes import litellm_model_routes +from plugins.filter.merge.with_defaults import merge_with_defaults from utils.cache.files import read_text +from utils.cache.yaml import load_yaml +from utils.roles.mapping import ROLE_FILE_META_SERVICES, ROLE_FILE_META_VOLUMES from . import PROJECT_ROOT TEMPLATE = PROJECT_ROOT / "roles/svc-ai-litellm/templates/config.yaml.j2" +REMOTE_MODELS = load_yaml( + PROJECT_ROOT / "roles/svc-ai-litellm" / ROLE_FILE_META_SERVICES +)["litellm"]["remote_models"] +BONSAI = { + "alias": "openrouter/ternary-bonsai-2-27b", + "model": "openrouter/prism-ml/ternary-bonsai-2-27b", + "provider": "openrouter", +} + +MOCK = { + "alias": "mock/deterministic", + "provider": "mock", + "context": 131072, + "response": "pong", +} SHARED = {"alias": "qwen2.5:0.5b", "name": "qwen2.5-0.5b-instruct"} OLLAMA_ONLY = {"alias": "llama3:latest", "name": "llama-3.2-3b-instruct"} @@ -45,44 +66,45 @@ def _stub_lookup(ollama_models, lmstudio_models): return lookup -REMOTE_BACKENDS = ( - ("openai/gpt-4o-mini", "openai/gpt-4o-mini", "OPENAI_API_KEY"), - ("anthropic/claude-sonnet-4-5", "anthropic/claude-sonnet-4-5", "ANTHROPIC_API_KEY"), - ("openrouter/auto", "openrouter/openrouter/auto", "OPENROUTER_API_KEY"), -) - - -def _remote(keys): - """The LITELLM_REMOTE_BACKENDS list vars/main.yml builds, with these keys. - - Args: - keys: env-name -> key value; a provider absent from it stays unkeyed. - """ - return [ - {"alias": alias, "model": model, "env": env, "key": keys.get(env, "")} - for alias, model, env in REMOTE_BACKENDS - ] - - -def render(*, ollama=(), lmstudio=(), keys=None): +def render( + *, + ollama=(), + lmstudio=(), + keys=None, + remote_models=REMOTE_MODELS, + router_alias="", + measured_speed=None, +): """The rendered config as a parsed mapping. Args: ollama: preload entries svc-ai-ollama declares; empty means not deployed. lmstudio: preload entries svc-ai-lmstudio declares; empty means not deployed. - keys: env-name -> provider key for the remote backends. + keys: provider -> key value; a provider absent from it stays unkeyed. + remote_models: the services.litellm.remote_models list in effect. + router_alias: the router's alias; empty publishes no router route, which + is what every case that predates the router expects. + measured_speed: alias -> tokens per second from the last deploy's + measurement; empty means nothing has been measured yet. """ env = Environment(undefined=StrictUndefined, autoescape=False) # noqa: S701 - YAML, not markup env.filters["bool"] = _ansible_bool + env.filters["to_json"] = json.dumps + env.filters["litellm_model_routes"] = litellm_model_routes rendered = env.from_string(read_text(str(TEMPLATE))).render( LITELLM_OLLAMA_BACKEND=str(bool(ollama)), LITELLM_LMSTUDIO_BACKEND=str(bool(lmstudio)), lookup=_stub_lookup(list(ollama), list(lmstudio)), OLLAMA_BASE_LOCAL_URL=OLLAMA_URL, LMSTUDIO_BASE_LOCAL_URL=LMSTUDIO_URL, + AI_MOCK_PROVIDER=MOCK["provider"], LITELLM_MAX_OUTPUT_TOKENS=512, LITELLM_UPSTREAM_TIMEOUT=60, - LITELLM_REMOTE_BACKENDS=_remote(keys or {}), + LITELLM_REMOTE_MODELS=list(remote_models), + LITELLM_SERVED_PROVIDERS=[name for name, key in (keys or {}).items() if key] + + [MOCK["provider"]], + LITELLM_ROUTER_ALIAS=router_alias, + LITELLM_MEASURED_SPEED=measured_speed or {}, ) return yaml.safe_load( rendered @@ -106,6 +128,27 @@ class TestOllamaOnly(unittest.TestCase): self.assertEqual(published[SHARED["alias"]]["api_base"], OLLAMA_URL) +class TestContextWindow(unittest.TestCase): + def test_a_declared_window_reaches_ollama_and_the_model_info(self) -> None: + config = render(ollama=[{**SHARED, "context": 32768}]) + entry = (config.get("model_list") or [])[0] + self.assertEqual(entry["litellm_params"]["num_ctx"], 32768) + self.assertEqual(entry["model_info"]["max_input_tokens"], 32768) + + def test_a_remote_model_publishes_its_window_without_num_ctx(self) -> None: + config = render( + keys={"openrouter": "sk-or"}, remote_models=[{**BONSAI, "context": 128000}] + ) + entry = (config.get("model_list") or [])[0] + self.assertEqual(entry["model_info"]["max_input_tokens"], 128000) + self.assertNotIn("num_ctx", entry["litellm_params"]) + + def test_a_model_without_a_window_publishes_none(self) -> None: + entry = (render(ollama=[SHARED]).get("model_list") or [])[0] + self.assertNotIn("num_ctx", entry["litellm_params"]) + self.assertNotIn("model_info", entry) + + class TestLmstudioOnly(unittest.TestCase): def test_the_alias_is_published_against_the_lmstudio_name(self) -> None: published = routes(render(lmstudio=[SHARED, LMSTUDIO_ONLY])) @@ -142,7 +185,7 @@ class TestNoBackend(unittest.TestCase): self.assertEqual(render()["model_list"], []) def test_one_provider_key_alone_publishes_its_route(self) -> None: - published = routes(render(keys={"OPENROUTER_API_KEY": "sk-or-test"})) + published = routes(render(keys={"openrouter": "sk-or-test"})) self.assertEqual(set(published), {"openrouter/auto"}) self.assertEqual( published["openrouter/auto"]["api_key"], "os.environ/OPENROUTER_API_KEY" @@ -150,19 +193,197 @@ class TestNoBackend(unittest.TestCase): def test_each_keyed_provider_gets_its_own_route(self) -> None: published = routes( - render( - keys={ - "OPENAI_API_KEY": "sk-test", - "ANTHROPIC_API_KEY": "sk-ant-test", - } - ) + render(keys={"openai": "sk-test", "anthropic": "sk-ant-test"}) ) self.assertEqual( set(published), {"openai/gpt-4o-mini", "anthropic/claude-sonnet-4-5"} ) def test_an_unkeyed_provider_publishes_nothing(self) -> None: - self.assertEqual(render(keys={"OPENAI_API_KEY": ""})["model_list"], []) + self.assertEqual(render(keys={"openai": ""})["model_list"], []) + + +class TestConfiguredRemoteModels(unittest.TestCase): + def test_an_added_model_shares_its_provider_key(self) -> None: + published = routes( + render( + keys={"openrouter": "sk-or-test"}, + remote_models=[*REMOTE_MODELS, BONSAI], + ) + ) + self.assertEqual(set(published), {"openrouter/auto", BONSAI["alias"]}) + self.assertEqual(published[BONSAI["alias"]]["model"], BONSAI["model"]) + self.assertEqual( + published[BONSAI["alias"]]["api_key"], "os.environ/OPENROUTER_API_KEY" + ) + + def test_an_inventory_list_replaces_the_defaults(self) -> None: + merged = merge_with_defaults( + { + "svc-ai-litellm": { + "services": {"litellm": {"remote_models": REMOTE_MODELS}} + } + }, + {"svc-ai-litellm": {"services": {"litellm": {"remote_models": [BONSAI]}}}}, + ) + published = routes( + render( + keys={"openrouter": "sk-or-test"}, + remote_models=merged["svc-ai-litellm"]["services"]["litellm"][ + "remote_models" + ], + ) + ) + self.assertEqual(set(published), {BONSAI["alias"]}) + + +class TestMockModels(unittest.TestCase): + """A mock is declared like any other model; the routing shape is the filter's.""" + + def test_the_mock_is_published_although_no_provider_holds_a_key(self) -> None: + published = routes(render(remote_models=[MOCK])) + self.assertEqual( + set(published), + {MOCK["alias"]}, + "an empty model_list makes the gateway answer 400 for the chat model " + "while every name-level variable still resolves to it", + ) + self.assertEqual(published[MOCK["alias"]]["mock_response"], MOCK["response"]) + + def test_a_response_carrying_yaml_punctuation_survives_the_render(self) -> None: + awkward = {**MOCK, "response": 'a: b\n"c" #d'} + published = routes(render(remote_models=[awkward])) + self.assertEqual(published[MOCK["alias"]]["mock_response"], awkward["response"]) + + +class TestRouterAlias(unittest.TestCase): + """The router is one more alias, and the hook is what makes it mean anything.""" + + def test_no_alias_publishes_no_router_route(self) -> None: + self.assertNotIn("auto", routes(render(ollama=[SHARED]))) + + def test_the_alias_is_published_beside_every_other_route(self) -> None: + published = routes(render(ollama=[SHARED, OLLAMA_ONLY], router_alias="auto")) + self.assertEqual( + set(published), {SHARED["alias"], OLLAMA_ONLY["alias"], "auto"} + ) + + def test_the_alias_raises_rather_than_serving_a_default(self) -> None: + published = routes(render(ollama=[SHARED, OLLAMA_ONLY], router_alias="auto")) + self.assertEqual( + published["auto"]["mock_response"], + "litellm.InternalServerError", + "the hook rewrites this alias before it is ever routed, so reaching " + "the entry at all means the router did not run; answering from some " + "default would hide that behind a plausible reply", + ) + + def test_the_alias_carries_no_backend_of_its_own(self) -> None: + published = routes(render(ollama=[SHARED], router_alias="auto")) + self.assertNotIn( + "api_base", + published["auto"], + "pointing the alias at a backend is what made a broken router look " + "like a working one", + ) + + def test_an_empty_model_list_publishes_no_router_route(self) -> None: + self.assertEqual(render(router_alias="auto")["model_list"], []) + + +class TestTraits(unittest.TestCase): + """What the hook filters on has to survive the render.""" + + def test_declared_traits_reach_the_model_info(self) -> None: + traits = {"vision": True, "tools": True, "cost_tier": 3} + config = render( + keys={"openrouter": "sk-or"}, remote_models=[{**BONSAI, "traits": traits}] + ) + entry = (config.get("model_list") or [])[0] + self.assertEqual(entry["model_info"]["traits"], traits) + + def test_a_backend_model_carries_its_traits(self) -> None: + config = render(ollama=[{**SHARED, "traits": {"tools": True}}]) + entry = (config.get("model_list") or [])[0] + self.assertEqual(entry["model_info"]["traits"], {"tools": True}) + + def test_a_model_without_traits_publishes_none(self) -> None: + entry = (render(ollama=[SHARED]).get("model_list") or [])[0] + self.assertNotIn("model_info", entry) + + def test_traits_and_a_context_window_coexist(self) -> None: + config = render( + ollama=[{**SHARED, "context": 32768, "traits": {"tools": True}}] + ) + info = (config.get("model_list") or [])[0]["model_info"] + self.assertEqual(info["max_input_tokens"], 32768) + self.assertEqual(info["traits"], {"tools": True}) + + +class TestMeasuredSpeed(unittest.TestCase): + """A rate has no catalogue, so the deploy measures it and renders it back.""" + + def _entry(self, **kwargs): + return (render(ollama=[SHARED], **kwargs).get("model_list") or [])[0] + + def test_a_measured_rate_becomes_a_trait(self) -> None: + alias = self._entry()["model_name"] + entry = self._entry(measured_speed={alias: 41.7}) + self.assertEqual(entry["model_info"]["traits"], {"speed": 41.7}) + + def test_a_measured_rate_joins_the_declared_traits(self) -> None: + config = render( + ollama=[{**SHARED, "traits": {"tools": True}}], + measured_speed={SHARED["alias"]: 41.7}, + ) + traits = (config.get("model_list") or [])[0]["model_info"]["traits"] + self.assertEqual(traits, {"tools": True, "speed": 41.7}) + + def test_a_declared_rate_outranks_the_measured_one(self) -> None: + config = render( + ollama=[{**SHARED, "traits": {"speed": 5}}], + measured_speed={SHARED["alias"]: 41.7}, + ) + traits = (config.get("model_list") or [])[0]["model_info"]["traits"] + self.assertEqual( + traits, + {"speed": 5}, + "a declaration is the operator overriding the measurement, so the " + "measurement must not overwrite it back on the next deploy", + ) + + def test_a_rate_for_a_model_that_is_gone_publishes_nothing(self) -> None: + entry = self._entry(measured_speed={"a-model-nobody-serves": 41.7}) + self.assertNotIn("model_info", entry) + + +class TestHookRegistration(unittest.TestCase): + """The callback string and the mounted file are one name in two files.""" + + def _mounted_module(self) -> str: + volumes = load_yaml( + PROJECT_ROOT / "roles/svc-ai-litellm" / ROLE_FILE_META_VOLUMES + ) + target = volumes["litellm_router_hook"]["mounts"][0]["target"] + return Path(target).stem + + def test_the_callback_names_the_file_the_role_mounts(self) -> None: + settings = render(ollama=[SHARED], router_alias="auto")["litellm_settings"] + self.assertEqual( + settings["callbacks"], + [f"{self._mounted_module()}.instance"], + "get_instance_fn resolves the callback against the config file's " + "directory, so renaming the mount target without the callback " + "leaves the proxy unable to start, at the next deploy and not here", + ) + + def test_without_a_router_alias_no_callback_is_loaded(self) -> None: + self.assertNotIn( + "callbacks", + render(ollama=[SHARED])["litellm_settings"], + "a gateway that publishes no router alias must not make the hook a " + "condition of starting its proxy", + ) if __name__ == "__main__": diff --git a/tests/unit/python/roles/svc-ai-litellm/templates/test_router_hook.py b/tests/unit/python/roles/svc-ai-litellm/templates/test_router_hook.py new file mode 100644 index 0000000000..4d501fcf3d --- /dev/null +++ b/tests/unit/python/roles/svc-ai-litellm/templates/test_router_hook.py @@ -0,0 +1,985 @@ +"""The router picks a model that can serve the request, or nothing at all. + +The hook ships as a template because the alias it answers to has to match the +one the route builder publishes. It is rendered here the way the role renders +it into the container, then exercised as plain Python: a wrong estimate or a +missing filter does not fail the render, it dispatches a prompt to a model that +cannot hold it and fails at the backend, after the choice was already made. + +The catalog is passed in rather than taken from the installed litellm, so these +cases state which facts they rely on instead of inheriting 1595 of them. +""" + +from __future__ import annotations + +import json +import sys +import types +import unittest +import unittest.mock +from pathlib import Path +from typing import ClassVar + +from jinja2 import Environment, StrictUndefined + +from utils.cache.files import read_text +from utils.cache.yaml import load_yaml +from utils.roles.mapping import ROLE_FILE_META_SERVICES + +from . import PROJECT_ROOT + +TEMPLATE = PROJECT_ROOT / "roles/svc-ai-litellm/templates/router_hook.py.j2" +ALIAS = "auto" +TODAY = "2026-09-24" +S1_URL = "http://s1:8000" + +SHIPPED = load_yaml(PROJECT_ROOT / "roles/svc-ai-litellm" / ROLE_FILE_META_SERVICES)[ + "litellm" +] +SHIPPED_ROUTER = SHIPPED["router"] +SHIPPED_WEIGHTS = SHIPPED_ROUTER["weights"] +SHIPPED_MIN_CHARS_PER_TOKEN = SHIPPED_ROUTER["min_chars_per_token"] +SHIPPED_STATE_CHARS = SHIPPED_ROUTER["state_chars"] +SHIPPED_SAMPLE = SHIPPED_ROUTER["sample"] + +CATALOG = { + "gpt-4o-mini": { + "mode": "chat", + "max_input_tokens": 128000, + "max_output_tokens": 16384, + "supports_vision": True, + "supports_function_calling": True, + "supports_tool_choice": True, + "supports_response_schema": True, + "supports_pdf_input": True, + "supports_prompt_caching": True, + "input_cost_per_token": 1.5e-07, + "output_cost_per_token": 6e-07, + "tpm": 4000000, + }, + "claude-sonnet-4-5": { + "mode": "chat", + "max_input_tokens": 200000, + "supports_vision": True, + "supports_function_calling": True, + "input_cost_per_token": 3e-06, + "output_cost_per_token": 1.5e-05, + }, + "ollama/llama3": { + "mode": "chat", + "max_input_tokens": 8192, + "input_cost_per_token": 0.0, + "output_cost_per_token": 0.0, + }, + "text-embedding-3-small": {"mode": "embedding", "max_input_tokens": 8191}, + "no-system": {"mode": "chat", "supports_system_messages": False}, + "retired": {"mode": "chat", "deprecation_date": "2025-01-01"}, + "outlives-us": {"mode": "chat", "deprecation_date": "2099-01-01"}, + "prose-dated": {"mode": "chat", "deprecation_date": "date when the model dies"}, + "one-image": {"mode": "chat", "supports_vision": True, "max_images_per_prompt": 1}, +} + + +def _stub_litellm(): + """The hook imports litellm at module scope; it is not a test dependency.""" + if "litellm" in sys.modules: + return + package = types.ModuleType("litellm") + integrations = types.ModuleType("litellm.integrations") + custom_logger = types.ModuleType("litellm.integrations.custom_logger") + + class CustomLogger: + pass + + custom_logger.CustomLogger = CustomLogger + package.model_cost = {} + sys.modules["litellm"] = package + sys.modules["litellm.integrations"] = integrations + sys.modules["litellm.integrations.custom_logger"] = custom_logger + + +def load( + *, + remote_fallback=False, + weights=None, + strategy="weighted", + sample_enabled=False, + sample_mode=None, + sample_trigger=None, + sample_interval=None, + sample_width=None, +): + """The rendered hook, imported as the container would import it.""" + _stub_litellm() + env = Environment(undefined=StrictUndefined, autoescape=False) # noqa: S701 - Python source, not markup + env.filters["to_json"] = json.dumps + env.filters["bool"] = bool + source = env.from_string(read_text(str(TEMPLATE))).render( + LITELLM_ROUTER_ALIAS=ALIAS, + LITELLM_ROUTER_REMOTE_FALLBACK=remote_fallback, + LITELLM_ROUTER_WEIGHTS=SHIPPED_WEIGHTS if weights is None else weights, + LITELLM_ROUTER_STRATEGY=strategy, + LITELLM_ROUTER_MIN_CHARS_PER_TOKEN=SHIPPED_MIN_CHARS_PER_TOKEN, + LITELLM_ROUTER_STATE_CHARS=SHIPPED_STATE_CHARS, + LITELLM_S1_URL=S1_URL, + LITELLM_S1_KEY="s1-test-key", + LITELLM_S1_MODEL="s1-latest", + LITELLM_S1_TIMEOUT=20, + LITELLM_ROUTER_SAMPLE_ENABLED=sample_enabled, + LITELLM_ROUTER_SAMPLE_MODE=SHIPPED_SAMPLE["mode"] + if sample_mode is None + else sample_mode, + LITELLM_ROUTER_SAMPLE_TRIGGER=SHIPPED_SAMPLE["trigger"] + if sample_trigger is None + else sample_trigger, + LITELLM_ROUTER_SAMPLE_INTERVAL=SHIPPED_SAMPLE["interval"] + if sample_interval is None + else sample_interval, + LITELLM_ROUTER_SAMPLE_WIDTH=SHIPPED_SAMPLE["width"] + if sample_width is None + else sample_width, + ) + module = types.ModuleType("router_hook_under_test") + module.__file__ = str(Path(TEMPLATE).with_suffix("")) + exec(compile(source, str(TEMPLATE), "exec"), module.__dict__) + return module + + +def local(alias, *, model=None, context=None, traits=None): + """A route served by a backend of this deployment, so it costs nothing.""" + return _route( + alias, {"model": model or alias, "api_base": "http://backend"}, context, traits + ) + + +def remote(alias, *, model=None, context=None, traits=None): + """A route served by a keyed provider.""" + params = {"model": model or alias, "api_key": "os.environ/X_API_KEY"} + return _route(alias, params, context, traits) + + +def mock(alias, *, context=None, traits=None): + """A route that answers from its own declared response.""" + params = {"model": f"openai/{alias}", "mock_response": "pong"} + return _route(alias, params, context, traits) + + +def _route(alias, params, context, traits): + entry = {"model_name": alias, "litellm_params": params} + info = {} + if context: + info["max_input_tokens"] = context + if traits: + info["traits"] = traits + if info: + entry["model_info"] = info + return entry + + +class HookCase: + def setUp(self) -> None: + self.hook = load() + self.need = self.hook.summarize({}) + + def eligible(self, routes, need=None, catalog=None): + return self.hook.eligible( + routes, + need or self.need, + ALIAS, + CATALOG if catalog is None else catalog, + TODAY, + ) + + def aliases(self, routes, need=None, catalog=None): + return [entry["model_name"] for entry in self.eligible(routes, need, catalog)] + + def decide(self, routes, need=None, **kwargs): + need = need or self.need + return self.hook.decide(self.eligible(routes, need), need, **kwargs) + + +class TestSummarize(HookCase, unittest.TestCase): + def test_plain_text_is_counted_and_rounded_up(self) -> None: + self.assertEqual( + self.hook.summarize({"messages": [{"content": "abcd"}]})["input_tokens"], 2 + ) + + def test_an_image_part_raises_the_vision_demand(self) -> None: + need = self.hook.summarize( + {"messages": [{"content": [{"type": "image_url", "image_url": {}}]}]} + ) + self.assertTrue(need["vision"]) + self.assertEqual(need["images"], 1) + + def test_an_audio_part_raises_the_audio_demand(self) -> None: + need = self.hook.summarize( + {"messages": [{"content": [{"type": "input_audio", "input_audio": {}}]}]} + ) + self.assertTrue(need["audio"]) + + def test_a_file_part_raises_the_document_demand(self) -> None: + need = self.hook.summarize( + {"messages": [{"content": [{"type": "file", "file": {}}]}]} + ) + self.assertTrue(need["documents"]) + + def test_a_system_role_raises_the_system_demand(self) -> None: + need = self.hook.summarize( + {"messages": [{"role": "system", "content": "be brief"}]} + ) + self.assertTrue(need["system"]) + + def test_a_developer_role_counts_as_a_system_message(self) -> None: + need = self.hook.summarize( + {"messages": [{"role": "developer", "content": "x"}]} + ) + self.assertTrue(need["system"]) + + def test_a_response_format_raises_the_schema_demand(self) -> None: + need = self.hook.summarize({"response_format": {"type": "json_schema"}}) + self.assertTrue(need["schema"]) + + def test_plain_text_response_format_demands_no_schema(self) -> None: + self.assertFalse( + self.hook.summarize({"response_format": {"type": "text"}})["schema"] + ) + + def test_tools_raise_the_tool_demand(self) -> None: + self.assertTrue(self.hook.summarize({"tools": [{"type": "function"}]})["tools"]) + + def test_parallel_tool_calls_raise_their_own_demand(self) -> None: + self.assertTrue( + self.hook.summarize({"parallel_tool_calls": True})["parallel_tools"] + ) + + def test_an_empty_request_demands_nothing(self) -> None: + need = self.hook.summarize({}) + self.assertEqual(need["input_tokens"], 0) + self.assertFalse( + any( + need[key] + for key in ("vision", "audio", "documents", "system", "tools", "schema") + ) + ) + + +class TestCapabilities(HookCase, unittest.TestCase): + """What a route can do comes from the catalog unless it says otherwise.""" + + def caps(self, entry): + return self.hook.capabilities(entry, CATALOG, TODAY) + + def test_a_catalogued_model_needs_no_declaration(self) -> None: + caps = self.caps(remote("m", model="openai/gpt-4o-mini")) + self.assertEqual( + (caps["context"], caps["vision"], caps["tools"], caps["input_cost"]), + (128000, True, True, 1.5e-07), + "the catalog ships these facts in the image, so declaring them in " + "meta/services.yml would only duplicate them and rot", + ) + + def test_the_output_price_is_read_too(self) -> None: + self.assertEqual( + self.caps(remote("m", model="openai/gpt-4o-mini"))["output_cost"], 6e-07 + ) + + def test_a_prefixed_key_is_tried_before_the_bare_one(self) -> None: + self.assertEqual( + self.caps(local("llama3", model="ollama/llama3"))["context"], 8192 + ) + + def test_a_declaration_overrides_the_catalog(self) -> None: + caps = self.caps( + remote("m", model="openai/gpt-4o-mini", traits={"vision": False}) + ) + self.assertFalse(caps["vision"]) + self.assertEqual(caps["context"], 128000) + + def test_an_uncatalogued_local_route_is_free(self) -> None: + caps = self.caps(local("qwen2.5:0.5b")) + self.assertEqual((caps["input_cost"], caps["output_cost"]), (0.0, 0.0)) + + def test_an_uncatalogued_keyed_route_sorts_behind_every_priced_one(self) -> None: + self.assertGreater(self.caps(remote("openrouter/auto"))["input_cost"], 1e-05) + + def test_an_uncatalogued_route_is_still_servable(self) -> None: + self.assertTrue( + self.caps(local("qwen2.5:0.5b"))["servable"], + "a local tag is in no catalog, so reading silence as a non-chat mode " + "would drop every locally served model", + ) + + def test_an_embedding_model_is_not_servable(self) -> None: + self.assertFalse( + self.caps(remote("e", model="text-embedding-3-small"))["servable"] + ) + + def test_system_messages_are_assumed_unless_denied(self) -> None: + self.assertTrue(self.caps(remote("m", model="claude-sonnet-4-5"))["system"]) + self.assertFalse(self.caps(remote("m", model="no-system"))["system"]) + + def test_a_past_deprecation_date_marks_the_route(self) -> None: + self.assertTrue(self.caps(remote("m", model="retired"))["deprecated"]) + + def test_a_future_deprecation_date_does_not(self) -> None: + self.assertFalse(self.caps(remote("m", model="outlives-us"))["deprecated"]) + + def test_a_prose_deprecation_value_is_not_a_date(self) -> None: + self.assertFalse( + self.caps(remote("m", model="prose-dated"))["deprecated"], + "the catalog ships a sample_spec stub whose date field is prose, so " + "parsing it as a date would retire a model on documentation", + ) + + +class TestEligible(HookCase, unittest.TestCase): + def test_the_router_never_picks_itself(self) -> None: + self.assertEqual(self.aliases([local(ALIAS), local("real")]), ["real"]) + + def test_a_window_smaller_than_the_prompt_is_dropped(self) -> None: + need = {**self.need, "input_tokens": 9000} + self.assertEqual( + self.aliases([local("s", context=4096), local("l", context=65536)], need), + ["l"], + ) + + def test_a_catalogued_window_is_enforced_without_a_declaration(self) -> None: + need = {**self.need, "input_tokens": 9000} + self.assertEqual( + self.aliases([local("llama3", model="ollama/llama3")], need), [] + ) + + def test_the_requested_output_counts_against_the_window(self) -> None: + need = {**self.need, "input_tokens": 4000, "max_output": 500} + self.assertEqual(self.aliases([local("s", context=4096)], need), []) + + def test_a_vision_demand_drops_a_model_without_it(self) -> None: + need = {**self.need, "vision": True} + self.assertEqual( + self.aliases( + [local("plain"), remote("v", model="openai/gpt-4o-mini")], need + ), + ["v"], + ) + + def test_an_audio_demand_drops_every_model_without_it(self) -> None: + need = {**self.need, "audio": True} + self.assertEqual( + self.aliases([remote("v", model="openai/gpt-4o-mini")], need), [] + ) + + def test_a_document_demand_needs_pdf_support(self) -> None: + need = {**self.need, "documents": True} + self.assertEqual( + self.aliases( + [ + remote("c", model="claude-sonnet-4-5"), + remote("o", model="openai/gpt-4o-mini"), + ], + need, + ), + ["o"], + ) + + def test_a_schema_demand_needs_response_schema_support(self) -> None: + need = {**self.need, "schema": True} + self.assertEqual( + self.aliases( + [ + remote("c", model="claude-sonnet-4-5"), + remote("o", model="openai/gpt-4o-mini"), + ], + need, + ), + ["o"], + ) + + def test_more_images_than_the_route_takes_is_dropped(self) -> None: + need = {**self.need, "vision": True, "images": 3} + self.assertEqual(self.aliases([remote("one", model="one-image")], need), []) + + def test_a_system_message_drops_a_route_that_denies_them(self) -> None: + need = {**self.need, "system": True} + self.assertEqual(self.aliases([remote("n", model="no-system")], need), []) + + def test_an_embedding_route_is_dropped_even_when_it_would_fit(self) -> None: + self.assertEqual( + self.aliases([remote("e", model="text-embedding-3-small")]), + [], + "an embedding model answers no chat request, and reaching it fails " + "at the backend after the choice was made", + ) + + def test_a_deprecated_route_is_dropped(self) -> None: + self.assertEqual( + self.aliases([remote("r", model="retired"), local("ok")]), ["ok"] + ) + + +class TestExpectedCost(HookCase, unittest.TestCase): + """Ranking on the input price alone reads the cheaper half of the bill.""" + + def test_the_output_price_enters_the_estimate(self) -> None: + need = {**self.need, "input_tokens": 100, "max_output": 1000} + caps = self.hook.capabilities( + remote("m", model="openai/gpt-4o-mini"), CATALOG, TODAY + ) + self.assertAlmostEqual( + self.hook.expected_cost(caps, need), 100 * 1.5e-07 + 1000 * 6e-07 + ) + + def test_a_request_without_max_tokens_still_prices_an_answer(self) -> None: + caps = self.hook.capabilities( + remote("m", model="openai/gpt-4o-mini"), CATALOG, TODAY + ) + self.assertGreater(self.hook.expected_cost(caps, self.need), 0.0) + + def test_a_long_answer_can_invert_the_input_price_order(self) -> None: + catalog = { + "wordy": { + "mode": "chat", + "input_cost_per_token": 1e-07, + "output_cost_per_token": 9e-06, + }, + "terse": { + "mode": "chat", + "input_cost_per_token": 5e-07, + "output_cost_per_token": 5e-07, + }, + } + need = {**self.need, "input_tokens": 10, "max_output": 4000} + routes = [remote("wordy", model="wordy"), remote("terse", model="terse")] + self.assertEqual( + self.hook.decide( + self.eligible(routes, need, catalog), need, strategy="weighted" + ), + "terse", + "the cheaper input price belongs to the route that would bill far " + "more once the answer is counted", + ) + + +class TestLocalOnly(HookCase, unittest.TestCase): + """A prompt can carry a credential, so it stays in the cluster by default.""" + + BIG: ClassVar[dict] = {"model": ALIAS, "messages": [{"content": "x" * 30000}]} + SECRET: ClassVar[dict] = { + "model": ALIAS, + "messages": [{"content": "deploy with sk-abcdefghijklmnopqrstuvwxyz012345"}], + } + + def route(self, hook, data, routes): + import asyncio + + proxy = types.ModuleType("litellm.proxy.proxy_server") + proxy.llm_router = types.SimpleNamespace(model_list=routes) + sys.modules["litellm.proxy"] = types.ModuleType("litellm.proxy") + sys.modules["litellm.proxy.proxy_server"] = proxy + return asyncio.run( + hook.instance.async_pre_call_hook(None, None, dict(data), "completion") + ) + + def test_a_secret_shape_is_recognised(self) -> None: + self.assertTrue(self.hook.carries_secret(self.SECRET)) + + def test_ordinary_prose_is_not(self) -> None: + self.assertFalse( + self.hook.carries_secret( + {"messages": [{"content": "my password is weak"}]} + ), + "a pattern that matches prose teaches callers to ignore the refusal", + ) + + def test_a_secret_stays_local_when_a_local_route_can_take_it(self) -> None: + routes = [local("home", context=99999), remote("big", context=99999)] + self.assertEqual(self.route(self.hook, self.SECRET, routes)["model"], "home") + + def test_by_default_a_big_prompt_does_not_leave_the_cluster(self) -> None: + routes = [local("tiny", context=10), remote("big", context=99999)] + verdict = self.route(self.hook, self.BIG, routes) + self.assertIsInstance(verdict, str) + self.assertIn("router.remote_fallback", verdict) + + def test_the_switch_lets_a_big_prompt_reach_a_remote_model(self) -> None: + hook = load(remote_fallback=True) + routes = [local("tiny", context=10), remote("big", context=99999)] + self.assertEqual(self.route(hook, self.BIG, routes)["model"], "big") + + def test_the_switch_does_not_unlock_a_secret(self) -> None: + hook = load(remote_fallback=True) + routes = [local("tiny", context=10), remote("big", context=99999)] + verdict = self.route(hook, self.SECRET, routes) + self.assertIsInstance(verdict, str) + self.assertIn("structured secret", verdict) + + +class TestLoudFailure(HookCase, unittest.TestCase): + """A router that cannot route says so; it never answers from a default.""" + + def route(self, routes, data): + import asyncio + + proxy = types.ModuleType("litellm.proxy.proxy_server") + proxy.llm_router = types.SimpleNamespace(model_list=routes) + sys.modules["litellm.proxy"] = types.ModuleType("litellm.proxy") + sys.modules["litellm.proxy.proxy_server"] = proxy + return asyncio.run( + self.hook.instance.async_pre_call_hook(None, None, dict(data), "completion") + ) + + def test_a_request_for_another_model_is_not_touched(self) -> None: + self.assertIsNone(self.route([local("x")], {"model": "x"})) + + def test_no_eligible_model_is_rejected_with_the_demand(self) -> None: + verdict = self.route( + [local("tiny", context=1)], + {"model": ALIAS, "messages": [{"content": "x" * 9000}]}, + ) + self.assertIsInstance(verdict, str) + self.assertIn("cannot route", verdict) + + def test_a_routable_request_is_rewritten(self) -> None: + verdict = self.route( + [local("big", context=99999)], + {"model": ALIAS, "messages": [{"content": "hi"}]}, + ) + self.assertEqual(verdict["model"], "big") + + +class TestConventionalStrategy(HookCase, unittest.TestCase): + def test_the_cheapest_price_wins(self) -> None: + routes = [ + remote("pricey", model="claude-sonnet-4-5"), + remote("cheap", model="openai/gpt-4o-mini"), + ] + self.assertEqual(self.decide(routes), "cheap") + + def test_a_local_model_beats_every_paid_one(self) -> None: + self.assertEqual( + self.decide([remote("paid", model="openai/gpt-4o-mini"), local("home")]), + "home", + ) + + def test_a_local_model_wins_even_when_a_remote_one_is_cheaper(self) -> None: + routes = [ + remote("free-tier", traits={"input_cost": 0.0, "output_cost": 0.0}), + local("home", traits={"input_cost": 9.9, "output_cost": 9.9}), + ] + self.assertEqual( + self.decide(routes), + "home", + "a prompt can carry a credential, so locality outranks price", + ) + + def test_a_mock_counts_as_local(self) -> None: + self.assertEqual( + self.decide([remote("paid", model="openai/gpt-4o-mini"), mock("mock/x")]), + "mock/x", + ) + + def test_among_equal_cost_the_faster_declared_route_wins(self) -> None: + self.assertEqual( + self.decide( + [ + local("slow", traits={"speed": 12}), + local("fast", traits={"speed": 90}), + ] + ), + "fast", + ) + + def test_a_tie_is_broken_by_alias_so_a_rerun_repeats(self) -> None: + routes = [local("b"), local("a")] + first = self.decide(routes) + second = self.decide(list(reversed(routes))) + self.assertEqual((first, second), ("a", "a")) + + def test_a_caching_route_outranks_an_equal_one_without(self) -> None: + routes = [ + remote( + "plain", + model="claude-sonnet-4-5", + traits={"input_cost": 0.0, "output_cost": 0.0}, + ), + remote( + "cached", + model="openai/gpt-4o-mini", + traits={"input_cost": 0.0, "output_cost": 0.0}, + ), + ] + self.assertEqual(self.decide(routes), "cached") + + def test_the_prompt_does_not_change_the_weighted_order(self) -> None: + routes = [ + local("small", context=8192, traits={"speed": 5}), + local("big", context=99999, traits={"speed": 90}), + ] + small = self.decide(routes, {**self.need, "input_tokens": 10}) + large = self.decide(routes, {**self.need, "input_tokens": 4000}) + self.assertEqual((small, large), ("big", "big")) + + +class DeciderStub: + """Stands in for the System One server, recording what the router asked it. + + Args: + choice: the alias it answers with, or None to raise a transport error. + confidence: the confidence it reports alongside the choice. + """ + + def __init__(self, choice, confidence=0.9, question="route") -> None: + self.choice = choice + self.confidence = confidence + self.question = question + self.seen: dict = {} + + def urlopen(self, request, timeout=None): + if self.choice is None: + raise OSError("connection refused") + self.seen = { + "url": request.full_url, + "timeout": timeout, + "headers": {k.lower(): v for k, v in request.header_items()}, + "body": json.loads(request.data.decode()), + } + body = { + "model": "gliformer-large-v1", + "answers": { + self.question: { + "type": "choice", + "choice": self.choice, + "confidence": self.confidence, + "probabilities": {self.choice: self.confidence}, + } + }, + "usage": {"input_tokens": 12, "output_tokens": 1}, + } + return _Response(json.dumps(body).encode()) + + +class _Response: + def __init__(self, body: bytes) -> None: + self.body = body + + def __enter__(self): + return self + + def __exit__(self, *exc_info: object) -> bool: + return False + + def read(self) -> bytes: + return self.body + + +class TestSystemOneStrategy(HookCase, unittest.TestCase): + """The prompt goes to the decider, which answers one typed choice.""" + + ROUTES: ClassVar[list] = [ + local("small", context=4096), + local("large", context=131072), + ] + + def ask(self, stub, routes=None, need=None, text="pick one"): + need = need or self.need + with unittest.mock.patch("urllib.request.urlopen", stub.urlopen): + return self.hook.decide( + self.eligible(routes or self.ROUTES, need), + need, + text, + strategy="system_one", + ) + + def test_the_alias_the_decider_chooses_is_the_one_routed_to(self) -> None: + self.assertEqual(self.ask(DeciderStub("large")), "large") + self.assertEqual(self.ask(DeciderStub("small")), "small") + + def test_the_candidates_become_the_choice_options(self) -> None: + stub = DeciderStub("small") + self.ask(stub) + question = stub.seen["body"]["questions"]["route"] + self.assertEqual(question["type"], "choice") + self.assertEqual(sorted(question["criteria"]), ["large", "small"]) + + def test_the_prompt_is_the_state_it_classifies(self) -> None: + stub = DeciderStub("small") + self.ask(stub, text="summarise this invoice") + self.assertEqual(stub.seen["body"]["state"], "summarise this invoice") + + def test_the_request_carries_the_model_alias_and_the_bearer_key(self) -> None: + stub = DeciderStub("small") + self.ask(stub) + self.assertEqual(stub.seen["body"]["model"], "s1-latest") + self.assertEqual(stub.seen["headers"]["authorization"], "Bearer s1-test-key") + self.assertEqual(stub.seen["url"], f"{S1_URL}/v1/systemone") + + def test_a_long_prompt_is_truncated_to_what_the_server_accepts(self) -> None: + stub = DeciderStub("small") + self.ask(stub, text="x" * 50000) + self.assertEqual( + len(stub.seen["body"]["state"]), + self.hook.S1_MAX_STATE_CHARS, + "the server answers 422 above its state limit, which would read as " + "a routing failure rather than an oversized prompt", + ) + + def test_an_option_the_router_never_offered_is_refused(self) -> None: + with self.assertRaises(ValueError): + self.ask(DeciderStub("a-model-nobody-serves")) + + def test_the_prompt_changes_the_choice_where_weights_cannot(self) -> None: + need = {**self.need, "input_tokens": 10} + self.assertEqual( + self.hook.decide( + self.eligible(self.ROUTES, need), need, strategy="weighted" + ), + "large", + ) + self.assertEqual(self.ask(DeciderStub("small"), need=need), "small") + + def test_the_render_carries_the_strategy(self) -> None: + self.assertEqual(load(strategy="system_one").ROUTER_STRATEGY, "system_one") + + def test_an_unreachable_decider_rejects_rather_than_substituting_one(self) -> None: + hook = load(strategy="system_one") + stub = DeciderStub(None) + with unittest.mock.patch("urllib.request.urlopen", stub.urlopen): + verdict = TestLoudFailure.route( + types.SimpleNamespace(hook=hook), + [local("small", context=4096), local("large", context=131072)], + {"model": ALIAS, "messages": [{"content": "hi"}]}, + ) + self.assertIsInstance( + verdict, + str, + "falling back to another decider would make the alias mean " + "something the caller did not ask for", + ) + self.assertIn("could not reach a decision", verdict) + + +class TestShippedConfiguration(HookCase, unittest.TestCase): + SPEED_ONLY: ClassVar[dict] = {"locality": 0.0, "cost": 0.0, "speed": 1.0} + + def test_the_render_carries_the_shipped_weights(self) -> None: + self.assertEqual(self.hook.ROUTER_WEIGHTS, SHIPPED_WEIGHTS) + + def test_the_shipped_strategy_reads_the_decider_service_flag(self) -> None: + self.assertIn( + "services.s1.enabled", + SHIPPED_ROUTER["strategy"], + "the strategy follows whether the System One service is deployed, " + "and it reads that from the one flag that decides it", + ) + + def test_both_strategy_names_are_reachable_from_the_expression(self) -> None: + env = Environment(undefined=StrictUndefined, autoescape=False) # noqa: S701 - a config value, not markup + env.filters["bool"] = bool + template = env.from_string(SHIPPED_ROUTER["strategy"]) + self.assertEqual(template.render(lookup=lambda *_args: True), "system_one") + self.assertEqual( + template.render(lookup=lambda *_args: False), + "weighted", + "without the System One service there is nothing to ask, so the " + "gateway must fall to the strategy it can compute itself", + ) + + def test_the_shipped_weights_keep_locality_decisive(self) -> None: + others = sum(v for k, v in SHIPPED_WEIGHTS.items() if k != "locality") + self.assertGreater( + SHIPPED_WEIGHTS["locality"], + others, + "every other factor together must not reach locality, or the " + "shipped default stops being the safest one", + ) + + def test_a_zero_weight_removes_the_factor(self) -> None: + routes = [ + remote("pricey", model="claude-sonnet-4-5", traits={"speed": 99}), + remote("cheap", model="openai/gpt-4o-mini", traits={"speed": 1}), + ] + self.assertEqual(self.decide(routes, weights={"cost": 1.0}), "cheap") + self.assertEqual(self.decide(routes, weights=self.SPEED_ONLY), "pricey") + + def test_no_weighting_sends_a_secret_out_of_the_cluster(self) -> None: + hook = load(remote_fallback=True, weights=self.SPEED_ONLY) + routes = [ + local("home", context=99999, traits={"speed": 1}), + remote("quick", context=99999, traits={"speed": 999}), + ] + verdict = TestLocalOnly.route(self, hook, TestLocalOnly.SECRET, routes) + self.assertEqual( + verdict["model"], + "home", + "the locality filter runs before the score, so a weight cannot buy " + "a credential a trip to a provider", + ) + + +class TestRequestClass(unittest.TestCase): + """History only helps when comparable requests land in the same class.""" + + def setUp(self) -> None: + self.hook = load() + + def sign(self, tokens, **flags): + need = dict.fromkeys(("vision", "tools", "schema", "audio", "documents"), False) + need.update(flags) + need["input_tokens"] = tokens + need["max_output"] = 0 + return self.hook.signature(need) + + def test_two_sizes_in_one_band_share_a_class(self) -> None: + self.assertEqual(self.sign(300), self.sign(1000)) + + def test_crossing_a_band_splits_the_class(self) -> None: + self.assertNotEqual(self.sign(1000), self.sign(1100)) + + def test_a_capability_demand_splits_the_class(self) -> None: + self.assertNotEqual(self.sign(300), self.sign(300, vision=True)) + + def test_the_largest_band_still_names_a_class(self) -> None: + self.assertTrue(self.sign(10**9).endswith(":plain")) + + +class TestSampler(unittest.TestCase): + """Comparisons cost a multiple of a request, so the trigger must hold.""" + + def setUp(self) -> None: + self.hook = load() + + def test_seconds_fires_once_and_then_waits(self) -> None: + now = [1000.0] + sampler = self.hook.Sampler("seconds", 900, clock=lambda: now[0]) + self.assertTrue(sampler.due()) + now[0] += 899 + self.assertFalse(sampler.due()) + now[0] += 2 + self.assertTrue(sampler.due()) + + def test_requests_counts_out_the_interval(self) -> None: + sampler = self.hook.Sampler("requests", 3) + self.assertEqual( + [sampler.due() for _ in range(6)], + [False, False, True, False, False, True], + ) + + def test_probability_draws_one_in_the_interval(self) -> None: + draws = iter([0.0, 0.5]) + sampler = self.hook.Sampler("probability", 4, draw=lambda: next(draws)) + self.assertTrue(sampler.due()) + self.assertFalse(sampler.due()) + + +class TestDescribedMetrics(unittest.TestCase): + """What the decider is told about a route, and what it is not told.""" + + def setUp(self) -> None: + self.hook = load() + + def describe(self, route, tally=None): + need = self.hook.summarize({"messages": [{"role": "user", "content": "hi"}]}) + entry = self.hook.eligible([route], need, ALIAS, CATALOG, TODAY)[0] + return self.hook.describe(entry, need, tally) + + def test_a_measured_rate_is_offered(self) -> None: + line = self.describe(local("home", context=9999, traits={"speed": 42.5})) + self.assertIn("42.5 tokens per second measured", line) + + def test_an_unmeasured_rate_is_left_out_rather_than_zeroed(self) -> None: + line = self.describe(local("home", context=9999)) + self.assertNotIn("tokens per second", line) + + def test_the_price_of_this_request_is_offered(self) -> None: + self.assertIn("costs about", self.describe(local("home", context=9999))) + + def test_an_earlier_verdict_is_offered(self) -> None: + line = self.describe(local("home", context=9999), {"home": (7, 9)}) + self.assertIn("won 7 of 9 comparisons like this", line) + + def test_a_route_without_a_record_claims_none(self) -> None: + line = self.describe(local("home", context=9999), {"other": (3, 3)}) + self.assertNotIn("comparisons", line) + + +class TestAnswerVerdict(unittest.TestCase): + """Judging the replies, not the promises the catalog made about them.""" + + def setUp(self) -> None: + self.hook = load() + + def test_the_aliases_are_hidden_from_the_decider(self) -> None: + stub = DeciderStub("option-0", question="verdict") + with unittest.mock.patch("urllib.request.urlopen", stub.urlopen): + self.hook.judge_answers("q", {"cheap": "a", "pricey": "b"}) + criteria = stub.seen["body"]["questions"]["verdict"]["criteria"] + self.assertEqual(sorted(criteria), ["option-0", "option-1"]) + self.assertEqual(sorted(criteria.values()), ["a", "b"]) + + def test_the_label_maps_back_to_the_alias_that_wrote_it(self) -> None: + stub = DeciderStub("option-1", question="verdict") + with unittest.mock.patch("urllib.request.urlopen", stub.urlopen): + winner = self.hook.judge_answers("q", {"cheap": "a", "pricey": "b"}) + self.assertEqual(winner, "pricey") + + def test_an_answer_outside_the_options_is_not_read_as_a_verdict(self) -> None: + stub = DeciderStub("option-9", question="verdict") + with ( + unittest.mock.patch("urllib.request.urlopen", stub.urlopen), + self.assertRaises(ValueError), + ): + self.hook.judge_answers("q", {"cheap": "a", "pricey": "b"}) + + +class TestCollect(unittest.IsolatedAsyncioTestCase): + """A route that cannot answer is absent from the comparison, not a loser.""" + + def setUp(self) -> None: + self.hook = load() + + class _Router: + def __init__(self, replies): + self.replies = replies + + async def acompletion(self, **kwargs): + reply = self.replies[kwargs["model"]] + if isinstance(reply, Exception): + raise reply + return types.SimpleNamespace( + choices=[ + types.SimpleNamespace(message=types.SimpleNamespace(content=reply)) + ] + ) + + async def test_a_raising_route_is_left_out(self) -> None: + router = self._Router({"a": "alpha", "b": RuntimeError("upstream down")}) + replies = await self.hook.collect( + router, + {"messages": []}, + [{"model_name": "a"}, {"model_name": "b"}], + 2, + ) + self.assertEqual(replies, {"a": "alpha"}) + + async def test_the_width_bounds_the_fan_out(self) -> None: + router = self._Router({"a": "alpha", "b": "beta", "c": "gamma"}) + replies = await self.hook.collect( + router, + {"messages": []}, + [{"model_name": "a"}, {"model_name": "b"}, {"model_name": "c"}], + 2, + ) + self.assertEqual(sorted(replies), ["a", "b"]) + + async def test_an_empty_reply_is_not_offered_for_judging(self) -> None: + router = self._Router({"a": "", "b": "beta"}) + replies = await self.hook.collect( + router, + {"messages": []}, + [{"model_name": "a"}, {"model_name": "b"}], + 2, + ) + self.assertEqual(replies, {"b": "beta"}) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/unit/python/roles/svc-ai-litellm/vars/__init__.py b/tests/unit/python/roles/svc-ai-litellm/vars/__init__.py new file mode 100644 index 0000000000..9f69c70b47 --- /dev/null +++ b/tests/unit/python/roles/svc-ai-litellm/vars/__init__.py @@ -0,0 +1,3 @@ +from pathlib import Path + +PROJECT_ROOT: Path = Path(__file__).resolve().parents[6] diff --git a/tests/unit/python/roles/svc-ai-litellm/vars/test_keyed_providers.py b/tests/unit/python/roles/svc-ai-litellm/vars/test_keyed_providers.py new file mode 100644 index 0000000000..8fdcc1fadc --- /dev/null +++ b/tests/unit/python/roles/svc-ai-litellm/vars/test_keyed_providers.py @@ -0,0 +1,80 @@ +"""The provider list the gateway actually publishes from. + +A model whose provider is missing here is filtered out of the model list, and +the proxy then answers HTTP 400 for it while every name-level variable still +resolves to it. That combination cost a deploy: the chat model resolved to the +mock and the gateway served `model_list: []`. +""" + +from __future__ import annotations + +import ast +import unittest +from typing import ClassVar + +from jinja2 import Environment, StrictUndefined + +from utils.cache.yaml import load_yaml +from utils.roles.mapping import ROLE_FILE_VARS_MAIN + +from . import PROJECT_ROOT + +_ROLE_VARS = PROJECT_ROOT / "roles/svc-ai-litellm" / ROLE_FILE_VARS_MAIN +_MOCK = "mock" + + +class TestProviderLists(unittest.TestCase): + """Two lists that mean different things and are easy to confuse.""" + + source: ClassVar[dict] + + @classmethod + def setUpClass(cls): + cls.source = load_yaml(str(_ROLE_VARS)) + + def _render(self, name, provider_keys, keyed=None): + env = Environment(undefined=StrictUndefined, autoescape=False) # noqa: S701 - renders a Python list literal, not markup + env.filters["dict2items"] = lambda mapping: [ + {"key": key, "value": value} for key, value in mapping.items() + ] + return ast.literal_eval( + env.from_string(self.source[name]).render( + LITELLM_PROVIDER_KEYS=provider_keys, + LITELLM_KEYED_PROVIDERS=keyed if keyed is not None else [], + AI_MOCK_PROVIDER=_MOCK, + ) + ) + + def _keyed(self, provider_keys): + return self._render("LITELLM_KEYED_PROVIDERS", provider_keys) + + def test_a_provider_without_a_key_is_not_keyed(self) -> None: + self.assertNotIn("openai", self._keyed({"openai": "", "anthropic": "sk-x"})) + + def test_a_provider_with_a_key_is_keyed(self) -> None: + self.assertIn("anthropic", self._keyed({"openai": "", "anthropic": "sk-x"})) + + def test_the_mock_is_never_keyed(self) -> None: + self.assertNotIn( + _MOCK, + self._keyed({"openai": "sk-x"}), + "env.j2 emits _API_KEY for every keyed provider by indexing " + "LITELLM_PROVIDER_KEYS, so a keyless entry here fails the render", + ) + + def test_the_mock_is_served_without_any_key(self) -> None: + self.assertIn( + _MOCK, + self._render("LITELLM_SERVED_PROVIDERS", {}, keyed=[]), + "a mock answers from its own response, so the model list must carry it", + ) + + def test_a_keyed_provider_is_also_served(self) -> None: + self.assertIn( + "anthropic", + self._render("LITELLM_SERVED_PROVIDERS", {}, keyed=["anthropic"]), + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/unit/python/roles/svc-ai-s1/__init__.py b/tests/unit/python/roles/svc-ai-s1/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/unit/python/roles/svc-ai-s1/files/__init__.py b/tests/unit/python/roles/svc-ai-s1/files/__init__.py new file mode 100644 index 0000000000..9f69c70b47 --- /dev/null +++ b/tests/unit/python/roles/svc-ai-s1/files/__init__.py @@ -0,0 +1,3 @@ +from pathlib import Path + +PROJECT_ROOT: Path = Path(__file__).resolve().parents[6] diff --git a/tests/unit/python/roles/svc-ai-s1/files/test_warm_model.py b/tests/unit/python/roles/svc-ai-s1/files/test_warm_model.py new file mode 100644 index 0000000000..199afd7a97 --- /dev/null +++ b/tests/unit/python/roles/svc-ai-s1/files/test_warm_model.py @@ -0,0 +1,106 @@ +from __future__ import annotations + +import importlib.util +import io +import json +import unittest +from contextlib import redirect_stderr +from typing import ClassVar, Self +from unittest.mock import patch + +from . import PROJECT_ROOT + +SCRIPT_PATH = PROJECT_ROOT / "roles/svc-ai-s1/files/python/warm_model.py" + +ENVIRONMENT = { + "S1_PORT": "8080", + "S1_MODEL_ALIAS": "s1-latest", + "S1_API_KEY": "sk-first-key", +} + + +class FakeResponse: + def __init__(self, body: bytes) -> None: + self._body = body + + def __enter__(self) -> Self: + return self + + def __exit__(self, *exc_info: object) -> bool: + return False + + def read(self) -> bytes: + return self._body + + +def run_script(body: bytes) -> dict[str, object]: + seen: dict[str, object] = {} + + def fake_urlopen(request: object, timeout: float | None = None) -> FakeResponse: + seen["url"] = request.full_url + seen["method"] = request.get_method() + seen["headers"] = {k.lower(): v for k, v in request.header_items()} + seen["body"] = json.loads(request.data.decode()) + seen["timeout"] = timeout + return FakeResponse(body) + + spec = importlib.util.spec_from_file_location("s1_warm_model", SCRIPT_PATH) + module = importlib.util.module_from_spec(spec) + with ( + patch.dict("os.environ", ENVIRONMENT, clear=False), + patch("urllib.request.urlopen", fake_urlopen), + ): + spec.loader.exec_module(module) + return seen + + +class TestS1WarmModel(unittest.TestCase): + """The router refuses to route when the decider misses its deadline, so the + deploy pays the first inference instead of the first caller. That only holds + when the warm-up reaches the decider and recognises a real answer. + """ + + ANSWERED: ClassVar[bytes] = b'{"answers": {"warmup": {"choice": "weather"}}}' + + def test_asks_the_decider_on_its_own_loopback(self) -> None: + seen = run_script(self.ANSWERED) + self.assertEqual(seen["url"], "http://127.0.0.1:8080/v1/systemone") + self.assertEqual(seen["method"], "POST") + + def test_carries_the_flavor_neutral_alias_and_key(self) -> None: + seen = run_script(self.ANSWERED) + self.assertEqual(seen["headers"]["authorization"], "Bearer sk-first-key") + self.assertEqual(seen["body"]["model"], "s1-latest") + + def test_asks_a_choice_question_the_encoder_can_answer(self) -> None: + question = run_script(self.ANSWERED)["body"]["questions"]["warmup"] + self.assertEqual(question["type"], "choice") + self.assertEqual( + sorted(question["criteria"]), + ["cooking", "weather"], + "a choice needs labels to pick between, and the state names one of " + "them, so an encoder that loaded has something to answer", + ) + + def test_waits_longer_than_the_router_would(self) -> None: + self.assertGreater( + run_script(self.ANSWERED)["timeout"], + 20, + "the first inference is what overruns services.s1.request_timeout, so a " + "warm-up on that budget would time out exactly where the router did", + ) + + def test_an_answer_without_the_question_exits_non_zero(self) -> None: + with redirect_stderr(io.StringIO()), self.assertRaises(SystemExit) as caught: + run_script(b'{"answers": {}}') + self.assertEqual( + caught.exception.code, + 1, + "a 200 carrying no answer means the model did not decide, and " + "reporting success there would hand the cold load back to the " + "first caller", + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/unit/python/roles/svc-net-tor/test_test_env_sourcing.py b/tests/unit/python/roles/svc-net-tor/test_test_env_sourcing.py index e0a3f8ed8c..8bbd46ef02 100644 --- a/tests/unit/python/roles/svc-net-tor/test_test_env_sourcing.py +++ b/tests/unit/python/roles/svc-net-tor/test_test_env_sourcing.py @@ -68,7 +68,12 @@ class TestTestEnvSourcing(unittest.TestCase): self.assertEqual( result.stderr, "", "sourcing test.env must not run anything" ) - self.assertEqual(result.stdout, " ".join(str(port) for port in PORTS)) + self.assertEqual( + result.stdout, + ",".join(str(port) for port in PORTS), + "the value joins on a comma so it stays one shell word " + "without quoting, which swarm would deliver verbatim", + ) if __name__ == "__main__": diff --git a/tests/unit/python/roles/sys-stk-backend/test_handler_registration_order.py b/tests/unit/python/roles/sys-stk-backend/test_handler_registration_order.py index 2d3b278a69..45753356e6 100644 --- a/tests/unit/python/roles/sys-stk-backend/test_handler_registration_order.py +++ b/tests/unit/python/roles/sys-stk-backend/test_handler_registration_order.py @@ -16,7 +16,9 @@ BLUESKY_STAGING = tuple( PROJECT_ROOT / "roles/web-app-bluesky/tasks" / name for name in ("01_social_app.yml", "02_login_broker.yml", "03_pds.yml") ) +SEARXNG_CORE = PROJECT_ROOT / "roles/svc-ai-searxng/tasks/00_core.yml" COMPOSE_OWNER = "sys-svc-compose" +RENDER_REPLICATED = "render_replicated_templates.yml" _NOTIFY = re.compile(r"^\s*notify:\s*(?:\[\s*)?['\"]?(compose-[\w-]+)", re.MULTILINE) @@ -150,6 +152,32 @@ class TestHandlerRegistrationOrder(unittest.TestCase): "run_once_sys_svc_compose is not defined", str(tasks[bootstrap].get("when")) ) + def test_searxng_bootstraps_the_compose_host_before_it_renders(self) -> None: + tasks = load_yaml_str(read_text(str(SEARXNG_CORE))) + bootstrap = next( + index + for index, task in enumerate(tasks) + if _role_include(task).get("name") == COMPOSE_OWNER + and _role_include(task).get("tasks_from") == "00_core.yml" + ) + render = next( + index + for index, task in enumerate(tasks) + if RENDER_REPLICATED + in str((task.get("ansible.builtin.include_tasks") or {}).get("file", "")) + ) + self.assertLess( + bootstrap, + render, + "the render notifies compose-up, and the next task pulls in " + "sys-svc-container, whose unconditional flush would run the " + "handlers before sys-svc-compose installs the compose wrapper", + ) + self.assertIn( + "run_once_sys_svc_compose is not defined", + str(tasks[bootstrap].get("when")), + ) + def test_the_owner_still_holds_every_compose_topic_that_is_notified(self) -> None: owner = "" for path, content in iter_project_files_with_content( diff --git a/tests/unit/python/roles/test-e2e-playwright/test_service_gating_helper.py b/tests/unit/python/roles/test-e2e-playwright/test_service_gating_helper.py index b537c62c7b..2d18db137d 100644 --- a/tests/unit/python/roles/test-e2e-playwright/test_service_gating_helper.py +++ b/tests/unit/python/roles/test-e2e-playwright/test_service_gating_helper.py @@ -130,6 +130,16 @@ class TestServiceGatingHelper(unittest.TestCase): self.assertNotEqual(proc.returncode, 0) self.assertIn("SSO_SERVICE_ENABLED", proc.stdout) + def test_a_hyphenated_service_resolves_to_its_underscored_flag(self): + proc = self._run_in_node( + "return helper.isServiceEnabled('agent-broker');", + env={"AGENT_BROKER_SERVICE_ENABLED": "false"}, + ) + self.assertEqual( + proc.returncode, 0, msg=f"stderr={proc.stderr}\nstdout={proc.stdout}" + ) + self.assertIn("RESULT:false", proc.stdout) + def test_unknown_service_hard_fails(self): proc = self._run_in_node( "return helper.isServiceEnabled('oicd');", diff --git a/tests/unit/python/roles/web-app-openwebui/files/test_agent_model_grants.py b/tests/unit/python/roles/web-app-openwebui/files/test_agent_model_grants.py new file mode 100644 index 0000000000..655ea4e624 --- /dev/null +++ b/tests/unit/python/roles/web-app-openwebui/files/test_agent_model_grants.py @@ -0,0 +1,103 @@ +from __future__ import annotations + +import importlib.util +import json +import unittest +from unittest.mock import patch + +from . import PROJECT_ROOT + +SCRIPT_PATH = ( + PROJECT_ROOT / "roles/web-app-openwebui/files/python/agent_model_grants.py" +) +AGENT_MODELS = { + "hermes": "/roles/web-app-hermes/agent-user", + "openclaw": "/roles/web-app-openclaw/agent-user", +} + + +def load_script(gateway_url="http://litellm:4000"): + spec = importlib.util.spec_from_file_location("agent_model_grants", SCRIPT_PATH) + module = importlib.util.module_from_spec(spec) + with patch.dict( + "os.environ", + { + "OPENWEBUI_BASE": "http://localhost:8080", + "OPENWEBUI_ADMIN_EMAIL": "administrator@example.org", + "OPENWEBUI_ADMIN_NAME": "administrator", + "OPENWEBUI_ADMIN_PASSWORD": "x" * 32, + "OPENWEBUI_GATEWAY_URL": gateway_url, + "OPENWEBUI_GATEWAY_KEY": "sk-gateway", + "OPENWEBUI_AGENT_MODELS": json.dumps(AGENT_MODELS), + }, + ): + spec.loader.exec_module(module) + return module + + +class FakeOpenWebUI: + def __init__(self, gateway_models): + self.gateway_models = gateway_models + self.grants = {} + self.groups = {} + + def call(self, url, key, payload=None): + if url.endswith("/v1/models"): + return 200, {"data": [{"id": model} for model in self.gateway_models]} + if url.endswith("/api/v1/groups/"): + return 200, [{"id": gid, "name": name} for name, gid in self.groups.items()] + if url.endswith("/api/v1/groups/create"): + self.groups[payload["name"]] = f"g{len(self.groups)}" + return 200, {"id": self.groups[payload["name"]]} + if url.endswith("/api/v1/models/model/access/update"): + self.grants[payload["id"]] = payload["access_grants"] + return 200, {} + raise AssertionError(url) + + def stored(self, model_id): + return sorted( + (g["principal_type"], g["principal_id"], g["permission"]) + for g in self.grants.get(model_id, []) + ) + + +def patched(script, fake): + return patch.multiple(script, call=fake.call, current_grants=fake.stored) + + +class TestProvision(unittest.TestCase): + def test_gateway_models_become_public_and_agents_stay_in_their_group(self): + script = load_script() + fake = FakeOpenWebUI(["qwen2.5:0.5b", "hermes"]) + with patched(script, fake): + self.assertTrue(script.provision("key")) + self.assertEqual(fake.grants["qwen2.5:0.5b"], script.PUBLIC) + for model_id, group in AGENT_MODELS.items(): + self.assertEqual( + fake.grants[model_id], + [ + { + "principal_type": "group", + "principal_id": fake.groups[group], + "permission": "read", + } + ], + ) + + def test_a_second_run_changes_nothing(self): + script = load_script() + fake = FakeOpenWebUI(["qwen2.5:0.5b"]) + with patched(script, fake): + script.provision("key") + self.assertFalse(script.provision("key")) + + def test_without_a_gateway_only_the_agents_are_granted(self): + script = load_script(gateway_url="") + fake = FakeOpenWebUI(["qwen2.5:0.5b"]) + with patched(script, fake): + script.provision("key") + self.assertEqual(set(fake.grants), set(AGENT_MODELS)) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/unit/python/utils/cache/test_files.py b/tests/unit/python/utils/cache/test_files.py index 14b554017e..780c949467 100644 --- a/tests/unit/python/utils/cache/test_files.py +++ b/tests/unit/python/utils/cache/test_files.py @@ -82,6 +82,8 @@ class TestIterProjectFiles(_ProjectRootFixture, unittest.TestCase): _touch(self.root / "node_modules/pkg/index.js", "module.exports = {};") _touch(self.root / "vendor/nette/utils/readme.md", "# vendored") _touch(self.root / "__pycache__/dead.pyc", "") + _touch(self.root / "build/ruff-cache-0/0.16.7/entry.json", "{}") + _touch(self.root / "cli/build/docs/readme/overview.py", "x = 1") def test_skip_dirs_pruned(self) -> None: all_paths = set(iter_project_files()) @@ -93,6 +95,17 @@ class TestIterProjectFiles(_ProjectRootFixture, unittest.TestCase): self.assertFalse(any("/node_modules/" in p for p in all_paths)) self.assertFalse(any("/vendor/" in p for p in all_paths)) + def test_build_is_pruned_at_the_root_only(self) -> None: + all_paths = set(iter_project_files()) + self.assertFalse( + any(p.startswith(f"{self.root}/build/") for p in all_paths), + f"root build/ artefacts reached the walk: {all_paths}", + ) + self.assertTrue( + any(p.endswith("cli/build/docs/readme/overview.py") for p in all_paths), + f"cli/build is source and must survive the prune: {all_paths}", + ) + def test_extensions_filter(self) -> None: ymls = list(iter_project_files(extensions=(".yml",))) self.assertEqual(len(ymls), 1) @@ -104,12 +117,13 @@ class TestIterProjectFiles(_ProjectRootFixture, unittest.TestCase): def test_exclude_tests_skips_tests_subtree(self) -> None: with_tests = list(iter_project_files(extensions=(".py",))) - self.assertEqual(len(with_tests), 1) + self.assertEqual(len(with_tests), 2) without_tests = list( iter_project_files(extensions=(".py",), exclude_tests=True) ) - self.assertEqual(without_tests, []) + self.assertEqual(len(without_tests), 1) + self.assertTrue(without_tests[0].endswith("cli/build/docs/readme/overview.py")) def test_exclude_dirs_path_segment_match(self) -> None: kept = list(iter_project_files(exclude_dirs=("docs",))) diff --git a/tests/unit/python/utils/roles/test_deploy.py b/tests/unit/python/utils/roles/test_deploy.py new file mode 100644 index 0000000000..ef47824976 --- /dev/null +++ b/tests/unit/python/utils/roles/test_deploy.py @@ -0,0 +1,109 @@ +"""Unit tests for utils.roles.deploy.role_deploy_modes. + +Covers: + * a declared mode set wins over the role's container shape; + * an injector without a stack can still be offered compose and swarm; + * a role that declares nothing falls back on its shape; + * the ``enabled`` value of each offered mode is carried through. +""" + +from __future__ import annotations + +import textwrap +import unittest +from pathlib import Path +from tempfile import TemporaryDirectory + +from utils.roles.deploy import role_deploy_modes +from utils.roles.mapping import ROLE_FILE_META_SERVICES + + +class _RoleFixture: + def __init__(self, root: Path): + self.root = root + + def write(self, role_name: str, body: str, *, stack: bool) -> Path: + role_dir = self.root / role_name + (role_dir / "meta").mkdir(parents=True, exist_ok=True) + (role_dir / ROLE_FILE_META_SERVICES).write_text( + textwrap.dedent(body).lstrip("\n"), encoding="utf-8" + ) + if stack: + (role_dir / "templates").mkdir(parents=True, exist_ok=True) + (role_dir / "templates" / "docker-compose.yml.j2").write_text( + "services: {}\n", encoding="utf-8" + ) + return role_dir + + +class TestRoleDeployModes(unittest.TestCase): + def setUp(self) -> None: + self._tmp = TemporaryDirectory() + self.addCleanup(self._tmp.cleanup) + self.fx = _RoleFixture(Path(self._tmp.name)) + + def test_declaration_outranks_a_missing_stack(self) -> None: + role = self.fx.write( + "web-svc-probe", + """ + probe: + modes: + compose: + enabled: true + swarm: + enabled: true + host: + enabled: false + """, + stack=False, + ) + self.assertEqual( + role_deploy_modes(role, "web-svc-probe"), + {"compose": True, "swarm": True, "host": False}, + ) + + def test_declaration_outranks_a_present_stack(self) -> None: + role = self.fx.write( + "web-svc-probe", + """ + probe: + modes: + host: + enabled: true + """, + stack=True, + ) + self.assertEqual(role_deploy_modes(role, "web-svc-probe"), {"host": True}) + + def test_a_stack_role_without_a_declaration_falls_back_on_its_shape(self) -> None: + role = self.fx.write("web-svc-probe", "probe:\n enabled: true\n", stack=True) + self.assertEqual( + role_deploy_modes(role, "web-svc-probe"), + {"compose": True, "swarm": True}, + ) + + def test_a_stackless_role_without_a_declaration_is_host_only(self) -> None: + role = self.fx.write("web-svc-probe", "probe:\n enabled: true\n", stack=False) + self.assertEqual(role_deploy_modes(role, "web-svc-probe"), {"host": True}) + + def test_a_disabled_declared_mode_is_offered_as_false(self) -> None: + role = self.fx.write( + "web-svc-probe", + """ + probe: + modes: + compose: + enabled: true + swarm: + enabled: false + """, + stack=True, + ) + self.assertEqual( + role_deploy_modes(role, "web-svc-probe"), + {"compose": True, "swarm": False}, + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/unit/python/utils/roles/test_meta_lookup.py b/tests/unit/python/utils/roles/test_meta_lookup.py index 41280b795b..636d8e9a0d 100644 --- a/tests/unit/python/utils/roles/test_meta_lookup.py +++ b/tests/unit/python/utils/roles/test_meta_lookup.py @@ -120,13 +120,13 @@ class TestMetaLookup(unittest.TestCase): self.assertIsNone(get_role_lifecycle(role_dir, role_name="web-app-yourls")) def test_returns_empty_when_services_file_missing(self) -> None: - role_dir = self.fx.root / "desk-something" + role_dir = self.fx.root / "dsk-something" role_dir.mkdir(parents=True) self.assertEqual( - get_role_run_after(role_dir, role_name="desk-something"), + get_role_run_after(role_dir, role_name="dsk-something"), [], ) - self.assertIsNone(get_role_lifecycle(role_dir, role_name="desk-something")) + self.assertIsNone(get_role_lifecycle(role_dir, role_name="dsk-something")) def test_malformed_yaml_raises_clear_error(self) -> None: role_dir = self.fx.root / "web-app-broken" diff --git a/tests/unit/python/utils/roles/test_stage.py b/tests/unit/python/utils/roles/test_stage.py index 3b086c2ced..8f20fc17c0 100644 --- a/tests/unit/python/utils/roles/test_stage.py +++ b/tests/unit/python/utils/roles/test_stage.py @@ -17,7 +17,7 @@ class TestStage(unittest.TestCase): def test_role_stage_by_category(self) -> None: self.assertEqual(role_stage("web-app-nextcloud"), "server") - self.assertEqual(role_stage("desk-gnome"), "workstation") + self.assertEqual(role_stage("dsk-gnome"), "workstation") self.assertEqual(role_stage("update-pacman"), "constructor") def test_deepest_category_wins(self) -> None: @@ -30,16 +30,16 @@ class TestStage(unittest.TestCase): def test_sort_key_orders_by_stage_then_category_then_name(self) -> None: keys = [ role_sort_key(r) - for r in ["svc-opt-ssd-hdd", "web-app-a", "web-svc-html", "desk-gnome"] + for r in ["svc-opt-ssd-hdd", "web-app-a", "web-svc-html", "dsk-gnome"] ] self.assertEqual( [ r for _, r in sorted( - zip(keys, ["svc-opt", "web-app", "web-svc", "desk"], strict=True) + zip(keys, ["svc-opt", "web-app", "web-svc", "dsk"], strict=True) ) ], - ["desk", "web-svc", "web-app", "svc-opt"], + ["dsk", "web-svc", "web-app", "svc-opt"], ) def test_stage_groups_match_wired_stage_loops(self) -> None: @@ -61,7 +61,7 @@ class TestStage(unittest.TestCase): ], ) self.assertEqual(stage_groups("server"), ["web-svc", "web-app", "web-opt"]) - self.assertEqual(stage_groups("workstation"), ["desk"]) + self.assertEqual(stage_groups("workstation"), ["dsk"]) self.assertEqual(stage_groups("destructor"), ["svc-opt"]) def test_bootstrap_groups_excluded_from_stage_groups(self) -> None: diff --git a/tests/unit/python/utils/roles/validation/test_invokable.py b/tests/unit/python/utils/roles/validation/test_invokable.py index 718242590c..398a5932d0 100644 --- a/tests/unit/python/utils/roles/validation/test_invokable.py +++ b/tests/unit/python/utils/roles/validation/test_invokable.py @@ -23,7 +23,7 @@ class TestInvokable(TestCase): dump_yaml( self.categories_file, - {"categories": [{"invokable_paths": ["web-app", "update", "util-desk"]}]}, + {"categories": [{"invokable_paths": ["web-app", "update", "util-dsk"]}]}, ) def mk_role(name: str, application_id: str | None = None) -> None: @@ -38,7 +38,7 @@ class TestInvokable(TestCase): mk_role("web-app-matomo", "matomo-app") mk_role("web-svc-nginx", None) # not invokable by our patched paths mk_role("update", None) # exact match - mk_role("util-desk-custom", None) # prefix match + mk_role("util-dsk-custom", None) # prefix match def tearDown(self) -> None: shutil.rmtree(self.tmp) @@ -49,14 +49,14 @@ class TestInvokable(TestCase): patch.object( inv, "_get_invokable_paths", - return_value=["web-app", "update", "util-desk"], + return_value=["web-app", "update", "util-dsk"], ), ): got = inv.list_invokable_app_ids() self.assertEqual( got, - sorted(["web-app-nextcloud", "matomo-app", "update", "util-desk-custom"]), + sorted(["web-app-nextcloud", "matomo-app", "update", "util-dsk-custom"]), ) def test_list_invokables_by_type(self) -> None: @@ -65,7 +65,7 @@ class TestInvokable(TestCase): patch.object( inv, "_get_invokable_paths", - return_value=["web-app", "update", "util-desk"], + return_value=["web-app", "update", "util-dsk"], ), ): grouped = inv.list_invokables_by_type() @@ -75,7 +75,7 @@ class TestInvokable(TestCase): self.assertIn("universal", grouped) self.assertEqual(grouped["server"], sorted(["web-app-nextcloud", "matomo-app"])) - self.assertEqual(grouped["workstation"], sorted(["util-desk-custom"])) + self.assertEqual(grouped["workstation"], sorted(["util-dsk-custom"])) self.assertEqual(grouped["universal"], ["update"]) @@ -103,7 +103,7 @@ class TestInvokable(TestCase): patch.object( inv, "_get_invokable_paths", - return_value=["web-app", "update", "util-desk"], + return_value=["web-app", "update", "util-dsk"], ), ): grouped = inv.list_invokables_by_type(rules=rules) @@ -119,7 +119,7 @@ class TestInvokable(TestCase): patch.object( inv, "_get_invokable_paths", - return_value=["web-app", "update", "util-desk"], + return_value=["web-app", "update", "util-dsk"], ), patch.object(inv, "_role_skip_modes", side_effect=fake_skip), ): @@ -139,14 +139,14 @@ class TestInvokable(TestCase): patch.object( inv, "_get_invokable_paths", - return_value=["web-app", "update", "util-desk"], + return_value=["web-app", "update", "util-dsk"], ), ): got = inv.types_from_group_names( [ "all", "web-app-nextcloud", - "util-desk-custom", + "util-dsk-custom", "update", "foo", ] diff --git a/utils/cache/files.py b/utils/cache/files.py index 6efd2d95c4..599e654238 100644 --- a/utils/cache/files.py +++ b/utils/cache/files.py @@ -53,6 +53,8 @@ _DEFAULT_SKIP_DIRS: frozenset[str] = frozenset( } ) +_ROOT_SKIP_DIRS: frozenset[str] = frozenset({"build"}) + @lru_cache(maxsize=4) def _all_project_files(root_str: str) -> tuple[str, ...]: @@ -64,7 +66,10 @@ def _all_project_files(root_str: str) -> tuple[str, ...]: """ paths: list[str] = [] for dirpath, dirnames, filenames in os.walk(root_str, topdown=True): - dirnames[:] = [d for d in dirnames if d not in _DEFAULT_SKIP_DIRS] + skip = _DEFAULT_SKIP_DIRS + if dirpath == root_str: + skip = skip | _ROOT_SKIP_DIRS + dirnames[:] = [d for d in dirnames if d not in skip] paths.extend(str(Path(dirpath) / fn) for fn in filenames) return tuple(paths) diff --git a/utils/roles/applications/services/resources.py b/utils/roles/applications/services/resources.py index e6e61978bf..e848588808 100644 --- a/utils/roles/applications/services/resources.py +++ b/utils/roles/applications/services/resources.py @@ -76,13 +76,30 @@ def _parse_bond(value: Any) -> float | None: return None -def _is_enabled(service_conf: dict[str, Any], default_enabled: bool) -> bool: +def _is_enabled( + service_conf: dict[str, Any], + default_enabled: bool, + dynamic_enabled: bool = True, +) -> bool: + """Whether a service counts as running. + + Args: + service_conf: the service's merged configuration. + default_enabled: what an absent ``enabled`` key means. + dynamic_enabled: how to read an ``enabled`` that is a template rather + than a literal. True counts it, which is the safe reading wherever + an under-count would hide a service. A caller that scopes itself to + one variant passes False, because a variant that means to run such + a service pins it literally: the bond guard requires exactly that. + """ if "enabled" not in service_conf: return default_enabled raw = service_conf.get("enabled") if isinstance(raw, bool): return raw text = str(raw).strip().lower() + if "{{" in text: + return dynamic_enabled return text not in ("false", "0", "no", "off") @@ -138,6 +155,7 @@ def collect_role_resources( max_depth: int = 0, dedup: bool = True, loaded: set | None = None, + dynamic_enabled: bool = True, ) -> None: if loaded is None: loaded = set() @@ -176,7 +194,9 @@ def collect_role_resources( continue if not _is_enabled( - service_conf, default_enabled=_looks_like_container(service_conf) + service_conf, + default_enabled=_looks_like_container(service_conf), + dynamic_enabled=dynamic_enabled, ): continue @@ -220,6 +240,7 @@ def collect_role_resources( max_depth=max_depth, dedup=dedup, loaded=loaded, + dynamic_enabled=dynamic_enabled, ) diff --git a/utils/roles/deploy.py b/utils/roles/deploy.py index b9664137ea..8ff9f28b21 100644 --- a/utils/roles/deploy.py +++ b/utils/roles/deploy.py @@ -9,7 +9,7 @@ from __future__ import annotations from pathlib import Path -from utils.roles.meta_lookup import get_role_mode_enabled +from utils.roles.meta_lookup import get_role_declared_modes, get_role_mode_enabled def role_has_stack(role_dir: Path) -> bool: @@ -26,12 +26,18 @@ def role_deploy_modes(role_dir: Path, role_name: str | None = None) -> dict[str, """Return the deploy modes a role supports, keyed by mode with its ``modes..enabled`` SPOT value. - A stack role (ships a compose template) is offered ``compose`` and - ``swarm``; a host-only role is offered ``host``. The two are mutually - exclusive: swarm needs a stack, host configures the machine instead. + What the role declares wins. A role that names its modes gets exactly + those, which is how an injector without a stack of its own can still be + deployed into compose and swarm alongside the partner that carries one. + + Only a role that declares nothing falls back on its shape: a stack role + (ships a compose template) is offered ``compose`` and ``swarm``, anything + else is offered ``host``. """ role_dir = Path(role_dir) - modes = ("compose", "swarm") if role_has_stack(role_dir) else ("host",) + modes = get_role_declared_modes(role_dir, role_name=role_name) or ( + ("compose", "swarm") if role_has_stack(role_dir) else ("host",) + ) return { mode: get_role_mode_enabled(role_dir, mode=mode, role_name=role_name) for mode in modes diff --git a/utils/roles/meta_lookup.py b/utils/roles/meta_lookup.py index cecc6d3e0c..ecbdb7dbae 100644 --- a/utils/roles/meta_lookup.py +++ b/utils/roles/meta_lookup.py @@ -125,6 +125,42 @@ def get_role_run_after(role: PathLike, *, role_name: str | None = None) -> list[ return out +def get_role_guide_companions( + role: PathLike, *, role_name: str | None = None +) -> list[str]: + """Return the role ids the documented deploy has to name alongside this one. + + A role whose own asserts need something the dependency closure does not + supply (a served model, a second backend) carries them here, because + ``--include`` filters the generated inventory to the ids it is given + literally: a provider pulled in as a dependency runs, but never joins + ``group_names``, so a flag gated on that stays false and the deploy the + README documents stops on the assert. + """ + role_dir, name = _resolve_role(role, role_name) + services = _read_meta_services(role_dir) + primary = _primary_entry(name, services) + if primary is None: + return [] + raw = primary.get("guide_companions") + if raw is None: + return [] + if not isinstance(raw, list): + raise MetaServicesShapeError( + f"Invalid guide_companions type in meta/services.yml for role " + f"'{name}': expected list, got {type(raw).__name__}." + ) + out: list[str] = [] + for item in raw: + if not isinstance(item, str) or not item.strip(): + raise MetaServicesShapeError( + f"Invalid guide_companions entry in meta/services.yml for role " + f"'{name}': {item!r} (expected non-empty string)." + ) + out.append(item.strip()) + return out + + def get_role_lifecycle(role: PathLike, *, role_name: str | None = None) -> str | None: """Return the role's ``lifecycle`` string (or ``None`` when absent).""" role_dir, name = _resolve_role(role, role_name) @@ -150,6 +186,32 @@ DEPLOY_MODES: tuple[str, ...] = ("compose", "swarm") """The stack deploy modes the CI test-deploy matrix (get_role_skip) covers.""" +def get_role_declared_modes( + role: PathLike, *, role_name: str | None = None +) -> tuple[str, ...]: + """Return the deploy modes the role names under ``modes``, in ``MODES`` + order, regardless of their ``enabled`` value. + + Empty when the role declares no ``modes`` block, which leaves the caller + on its own default. A role that names a mode its stack shape would not + imply (an injector asking for compose) is stating an intent, so the + declaration outranks the shape. + """ + role_dir, name = _resolve_role(role, role_name) + primary = _primary_entry(name, _read_meta_services(role_dir)) + if primary is None: + return () + modes = primary.get("modes") + if modes is None: + return () + if not isinstance(modes, dict): + raise MetaServicesShapeError( + f"Invalid modes type in meta/services.yml for role '{name}': " + f"expected mapping, got {type(modes).__name__}." + ) + return tuple(mode for mode in MODES if mode in modes) + + def get_role_mode_enabled( role: PathLike, *, mode: str, role_name: str | None = None ) -> bool: diff --git a/utils/roles/validation/invokable.py b/utils/roles/validation/invokable.py index c89378d804..b9c4256bfc 100644 --- a/utils/roles/validation/invokable.py +++ b/utils/roles/validation/invokable.py @@ -30,7 +30,7 @@ DEFAULT_RULES: tuple[DeploymentTypeRule, ...] = ( ), DeploymentTypeRule( name="workstation", - include_re=re.compile(r"^(desk-|util-desk-)"), + include_re=re.compile(r"^(dsk-|util-dsk-)"), exclude_re=None, ), DeploymentTypeRule(