The contract check ran as an Ansible task in the middle of every provider deploy, and the two client checks ran the same way. A test that is part of the deploy cannot be run without deploying, cannot be re-run on its own, and reports its verdict as a deploy failure. Twenty-seven task files lose 491 lines to this.
It is now what it always was. Each provider ships files/test/test.sh, templates/test.env.j2 and meta/tests.yml, and test-e2e-cli stages one shared harness beside it: run.sh drives contract.sh in a throwaway container on the provider's own network, then isolation.sh for adapter-fronted providers. The read probe each provider offers moved from the calling task into meta/mcp.yml next to the allowlist it must satisfy, and the schema now rejects a read probe naming a tool outside that allowlist.
The bearer the test presents is the one a client resolves through mcp.credential, via the new mcp_client_bearer lookup over resolve_credential, which moved to utils.roles.applications.mcp so it is the same function mcp_servers renders every client connection with. The probe presented whatever the calling role passed, which for the fifteen adapter-fronted providers is a different secret than any client holds.
Sidecar isolation is read from the engine rather than measured inside the container, so it no longer needs a delegation to the node holding the task: attachment is compared by network name, which drops the gwbridge special case, and a swarm worker says it cannot measure instead of skipping silently.
Retrying is bounded by wall clock against MCP_DEADLINE_SECONDS, which scales through the same lookup('timeout') as the harness budget it has to stay under. A flag that is neither true nor false aborts rather than reading as off.
mcp-provider-probe becomes mcp-provider-cli-test: an enabled provider must ship the test rather than include the task. mcp-probe-bearer becomes mcp-adapter-bearer, tying ADAPTER_BEARER to the declared credential key now that no probe include carries it. A new integration test renders the shared env template for every provider through Ansible's own templar, so a later change to it breaks visibly instead of poisoning twenty deploys quietly.
Hermes and OpenClaw keep their client-side proof as CLI tests of their own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twelve callers load their scripts with the relative form
lookup('file', 'name.py'), which resolves against the role's files/
directory. dfe7a7fdd moved those scripts into files/python/ without
updating them, so the MCP provisioning and probes of baserow, checkmk,
flowise, gitlab, homeassistant, jellyfin, n8n, openwebui and pretix, plus
n8n's AI gateway task, all resolved a path that no longer exists.
This is the second time the same relocation left callers behind. After
2aec4326a fixed the php paths, the claim that no stale references remained
came from grepping the worktree, which already carried the corrected paths;
HEAD still held the old ones. The check has to run against the commit.
Verified against HEAD this time: the eleven remaining stale relative lookups
are exactly the files in this bundle, and no other loading form (copy src,
script) carries a stale path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role drove Jellyfin's first-run wizard as soon as the container existed, but
created is not serving: Jellyfin initialises its database and web host after
start, so the wizard POSTs raced that initialisation. Losing the race did not
merely fail the step, it could half-apply and consume the one-shot wizard on an
instance that was never configured.
The auth flow now polls the container's own healthcheck, up to five minutes,
before touching the wizard. Three details are deliberate: the Go template is
wrapped in {% raw %} so Jinja does not try to resolve .State.Health.Status and
fail at render rather than at run; the inspect is delegated to the hosting node,
because under swarm the container is not on the controller; and changed_when is
false, since a readiness probe reports state and never changes it.
Not verified: jellyfin was not in the last deploy's role set, so this has not
run. The gate is also only as good as the image's healthcheck -- if that reports
healthy on "HTTP responds" rather than "setup endpoints ready", this narrows the
race without closing it. The 60x5s bound is a choice, not a measurement against
Jellyfin's cold start on a loaded box.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The reconciler addresses a client with tasks_from: utils/mcp.yml, which
only works when every role offers its MCP work at the same path. The
files were named per role, and on the two agents the work sat inline in
the entry point, where nothing but a full role run could reach it.
The filter docstring pointed at the pre-rename path and now names the
file that exists.
Every MCP role keeps its task tree under tasks/utils/ without numeric
prefixes, so a caller can reach any of them through include_role
tasks_from. utils/roles/mapping.py holds that path as the single source.
Ansible resolves a relative include against the including file's own
directory when that file was reached by path, but against tasks/ when it
was reached through tasks_from. The same file therefore loaded from its
own role and failed from a caller using tasks_from. Every nested include
now names {{ role_path }}/tasks/utils/..., which holds under both bases,
and a lint walks each tasks_from entrypoint transitively to keep it so.
The exec-gating lint learns the same form, otherwise it loses the include
chain and reports inherited IS_STACK_HOST gates as missing.
The expression resolving a provider's stored token appeared eighteen
times. plugins/lookup/mcp_credential.py replaces it, and mcp_probe_owner
disappears from eight roles: it was only ever a step towards the bearer
and no probe read it.
The Hermes and OpenClaw probes stop aborting the play while it is still
running. Roles deploy in dependency order, so a client probing during its
own role can run before a provider exists at all, and failing there
prevents that provider from ever deploying. The verdict moves to the
reconciliation pass, gated on MCP_RECONCILE_STRICT, which the reconciler
sets and group_vars declares so the readers need no default.
The sys-svc-mcp-reconcile role is added here; wiring it into a stage is a
separate change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ten applications with no MCP server of their own get one without a line of code
inside them: qdrant, checkmk, fider, jellyfin, listmonk, pretix, prometheus,
snipe-it, zammad and libretranslate.
The sidecar is confined rather than trusted. It runs read-only, drops every
capability, forbids privilege escalation, publishes no port and joins only its
own provider's network, carrying neither that provider's alias nor the consumer
networks the provider itself joined. Its tool surface is a JSON contract, not
code: which tools exist, which upstream path each maps to and what shape the
arguments take. Adding a tool is a line of JSON, not a new image.
The image is built locally from the audited adapter source with the base image
as a build argument, so all ten share one reviewed build instead of ten upstream
dependencies. Each role provisions its own read-only identity in the language
its application speaks, and every adapter include is guarded by the role's MCP
flag, so switching it off leaves no container behind.
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.
Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Variant 1 (no auth stack) died because jellyfin_auth_env eagerly
resolved the LDAP bind and Keycloak client lookups even when neither
service is deployed; the env is now composed from a base dict plus
LDAP/SSO parts combined only when the respective service is enabled.
Jellyfin keeps all state in embedded SQLite under the shared /config
volume; concurrent replicas corrupt it via node-local NFS flocks
(SQLite disk I/O error in CI), and upstream has no multi-instance
support. One floating replica keeps reschedules safe: the NFS volume
follows the task to any node.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
configure-auth.sh ran `container exec/restart jellyfin` with the bare compose
name, which does not exist under docker stack deploy. Split the script into
install and configure phases (JELLYFIN_PHASE), target the runtime-resolved CID
(JELLYFIN_CT), and drive restarts from the task (container restart in compose,
container service update --force in swarm). The phase flow moves into
01a_auth.yml so the parent block stays within the size limit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
sys-svc-compose already flushes compose handlers (and runs post-deploy) when the including role passes compose_handlers_flush: true. Roles that set it false and then immediately hand-roll their own meta: flush_handlers just reimplement that flush. Add an adjacency-scoped lint that catches the back-to-back form (a deferred flush with real work between the include and the flush stays legitimate and keeps false), and collapse the five offenders (checkmk, chess, jellyfin, pihole, semaphore) to compose_handlers_flush: true.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Declare the LDAP and SSO auth plugins as proper meta/addons entries
(mechanism/source/version, github-releases update monitoring, url/tag/
md5/guid under config:) and resolve the install parameters in vars from
the addon config instead of hardcoding them.
Move biber's login journeys into the mandated per-addon Playwright specs:
ldap-authentication tests the LDAP manual login, sso-authentication the
OIDC round-trip. Because a Jellyfin user binds to exactly one auth
provider, the LDAP spec skips while SSO is active — the ldap-only matrix
variant owns that path — so a single test user suffices and the specs
stay parallel-safe. playwright.spec.js keeps guest + native admin;
biber's generic persona is opted out via PERSONA_BIBER_BLOCKED.
Also fill the optional meta/info.yml fields (homepage, video, display).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bring the full matrix (full-auth / no-auth / ldap-only) green:
- healthcheck: use curl (the image ships no wget)
- install the LDAP plugin from its GitHub release (repo.jellyfin.org is
unreachable from the deploy network); version/url/md5 live in
meta/addons/ldap-authentication.yml and reach the script via env
- configure the SSO/OIDC provider through the plugin config API as JSON
(order-agnostic) instead of hand-written XML, which silently produced
an empty provider map; set DisablePushedAuthorization + https
SchemeOverride so Keycloak accepts the redirect_uri
- inject the SSO login button via the branding login-disclaimer + CustomCss
- seed the break-glass admin unconditionally and retry the first-run
wizard until the admin token resolves (fixes the no-auth variant race)
- Playwright: role-based authenticated-chrome selector for JF 10.11,
localStorage-clearing logout, and LDAP-before-OIDC ordering (a Jellyfin
user binds to exactly one auth provider)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Implements requirement 030: web-app-jellyfin deploying jellyfin/jellyfin
10.11.11 behind sys-svc-proxy. No external DB (internal SQLite); /config,
/cache, /media named volumes; CPU transcoding. Jellyfin has no native SSO, so
auth is plugin-based (operator chose BOTH): jellyfin-plugin-ldapauth (central
OpenLDAP, all clients incl. native apps) + 9p4/jellyfin-plugin-sso (Keycloak
OIDC, web one-click). files/configure-auth.sh completes the first-run /Startup
wizard (seeds break-glass admin), installs both plugins via Jellyfin's own
/Packages installer, writes the plugin configs, and restarts.
Schemas source-verified against the pinned upstream: Jellyfin v10.11.11, LDAP
plugin v23 (GUID 958aad66-..., flat XmlSerializer config), SSO plugin v4.0.0.3
(OidConfig fields). Honest caveats: the SSO OidConfigs is a SerializableDictionary
(XML shape best-effort), plus plugin ABI compat + the SPA Playwright selectors
are live-deploy-validation items (env-blocked). Each config file is XML-well-
formedness checked before reload.
Validation: make autoformat clean; make test green (lint, test-lint, test-unit,
test-integration 190, test-external).
Refs: docs/requirements/030-web-app-jellyfin.md
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
