92c45e2471 moved Open WebUI off the model server onto the gateway and renamed
test-ollama-chat.js to test-litellm-chat.js in the same commit, git records it
as R073. `git ls-tree` on that commit shows only the successor in the
directory, and both OLLAMA_* variables left playwright.env.j2 with it.
fork/feature/svc-net-tor still carried the pre-refactor file. Merging it in
7162b7bf4c restored test-ollama-chat.js and OLLAMA_PRELOAD_MODEL without a
matching add: `git log --diff-filter=A` on that path knows only the original
creation from 06aa25c095. A scan of all 916 merges reachable from HEAD finds
exactly one other path resurrected the same way, .github/pull_request_template.md,
which is not a role file and is left alone here.
Deleting it removes no coverage. The file never executed: it does not match
the collector's `**/*.@(spec|test).js` and no aggregator required it, and the
JUnit report from openwebui#1 lists thirteen tests with no `ollama chat`
among them, not even as a skip. Four of its five assertions are byte-identical
in the registered successor, which runs green in openwebui#1 and #2.
The fifth is deliberately weaker there: the successor takes the first
non-embedding model from /api/models instead of pinning OLLAMA_PRELOAD_MODEL,
so a green run proves the gateway serves a chat model rather than that Ollama
specifically does. That was the refactor's stated intent, exercising the chain
rather than a fixture, and reviving the file would route a spec straight at a
model server that a lint exists to keep roles away from.
OLLAMA_PRELOAD_MODEL goes with it: the deleted spec was its only reader. The
plural OLLAMA_PRELOAD_MODELS in svc-ai-ollama is a different variable and
stays.
Each of these specs declared the service it exercises and then, as its first
action, signed in through the dashboard's OIDC button. In a round that ships
no identity provider that button does not exist, so the spec spent its whole
timeout waiting for it and reported the missing login rather than anything
about MCP. The junit already said as much, carrying the SSO_SERVICE_ENABLED
flag as the skip reason on the neighbouring tests.
Nine specs in the directory sign in that way; two already declared sso. The
other seven now do too, through the same gate they already imported.
Signing in as the deployed identity instead was not open to us: that helper
authenticates as the administrator whatever username it is handed, which
contradicts the non-member spec asserting the very endpoint the administrator
is asserted to reach. Nor can a native member stand in, because the account is
created by the OIDC login itself, so a round without one has no members to
grant anything to.
This buys the honest label at a price worth naming: a round that ships
seaweedfs without sso now reports green with no seaweedfs coverage, where it
used to report red for the wrong reason. Covering that needs a sign-in the
suite does not have yet.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
GET /api/v1/tools/ builds its answer for an MCP server out of stored config alone - routers/tools.py:125-126 reads info off the connection row and never touches the network. So the three specs that read it pass unchanged against a provider that is down, refuses the deployed bearer, or serves nothing. What they establish is registration and the OIDC group to access-grant chain, not that the bridge carries.
POST /api/v1/configs/tool_servers/verify is the one route in 0.11.0 that does a real MCP initialize plus tools/list without an LLM, and it authenticates with the key in the body it is handed. The spec reads the stored connection and echoes it back, so the handshake runs over the deployed URL with the deployed credential through Open WebUI's own client, and asserts three things per server: it answers, it serves at least one tool, and the tool set contains the read probe its meta/mcp.yml declares.
Only bearer connections are verified. A system_oauth one resolves the caller's own upstream token per request and the administrator holds none, so it would fail on the caller rather than on the bridge; no provider declares auth: oidc today, but the filter keeps the spec honest when one does. The per-test budget scales with the number of registered servers instead of a fixed guess.
MCP_EXPECTED_TOOLS carries the declared read probe per server. Its rendering is unit-tested by extracting the block from the template itself rather than restating it, and decoded the way a spec decodes it - dotenv_quote emits an escaped JSON string that decodeDotenvQuotedValue unwraps before the spec parses the payload, so comparing after one decode would test a value no spec ever sees.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twenty-five specs called the Playwright APIs with default or hardcoded
timeouts. Routed over Tor those budgets are too small, so a spec fails on
transport latency rather than on the behaviour it asserts.
Route the waits through resolveTimeout and reach pages via gotoOnion, the
helpers the tor branch introduced for exactly this. The remaining changes in
the diff are import wrapping around those calls.
The repository lint enforces this: a raw numeric timeout in a role spec is
rejected with a pointer to resolveTimeout, which is how a hardcoded 500ms
wait added during this session was caught.
Merge work rather than authored here; not deployed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The merged branch moved every role-local script out of files/ into a folder
named for its language (files/python/, files/php/, files/shell/), but the
moves lost their staging and never landed. HEAD still carried the old paths
while the worktree had the new ones.
Move the 24 remaining scripts and update the six places that resolve them:
the mcp-adapter instance/isolation/probe tasks, the litellm key task, the
adapter README and the endpoint-path lint. Every changed line in those
files is a path.
Also drop roles/sys-svc-compose/files/swarm/stack_ready.sh, a stale
duplicate of the copy already tracked under files/shell/swarm/.
The added comments are the branch's own `nocheck: mirrored-unit-test`
directives, which state why a script whose measurement only exists inside
its container cannot have a mirrored unit test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The matrix variant preloaded smollm2:135m, which is too small to emit a
stop token reliably. Ollama's own log for the prompt "hi" shows it running
away rather than running slowly:
n_gen = 22011, tg = 86.14 t/s
slot context shift, n_keep = 4, n_left = 4091, n_discard = 2045
aborting completion request due to client closing the connection
86 tokens per second is fast; 22000 tokens with repeated context shifts is
a model that never finishes. Every completion without a hard token cap ran
until the consumer's own timeout, which is why zammad reported
AI::Provider::ResponseError after exactly 300s while short warm-ups and
provider pings returned 200 immediately.
Preload qwen2.5:0.5b instead: a comparable footprint (397MB vs 270MB) that
is instruction-tuned and terminates.
web-app-openwebui asserted the served model with a hardcoded
id.startsWith("smollm2"), so it would have failed on any model change.
Render the deployed preload model into its Playwright env and compare
against that instead of a literal family name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ENABLE_OAUTH_ROLE_MANAGEMENT does two things and only one is obvious. Besides
mapping the administrator group onto the Open WebUI admin role, it turns
OAUTH_ALLOWED_ROLES into a sign-in gate: a user whose groups claim is non-empty
and matches neither the allowed list nor the admin list is refused outright.
Upstream defaults that list to the literals `user,admin`, and Keycloak emits
claims shaped /roles/<app>/<role>, which can never match them. A user with no
groups therefore signed in fine, and the moment they received their first `mcp`
grant their claim became non-empty, matched nothing, and they were refused. The
grant meant to give access was what removed it.
The env sets OAUTH_ALLOWED_ROLES=*. The grant itself stays the real gate, since
BYPASS_ADMIN_ACCESS_CONTROL=false keeps it binding on administrators too. Stated
plainly, this does widen a control: every authenticated platform user now passes
Open WebUI's role gate. A list matching the actual role paths would be stricter
and is not built here.
The LAM grant spec is rewritten to drive the real path rather than assert around
it. Three behaviours it now handles: LAM renders per-value add and clear controls
as anchors with an empty alt, so they cannot be found by accessible name and are
located structurally; a membership written in LAM crosses Keycloak federation and
then an Open WebUI login sync before it is observable, so the check polls instead
of reading once; and the sign-out step clears localStorage rather than only
cookies, because Open WebUI keeps its session token there and a cookie-only reset
left the old session live while the assertion passed.
Not verified: openwebui was not in the last deploy's role set, so neither the env
change nor the rewritten spec has run. The spec is also now genuinely
re-authenticating, which is more correct and will surface auth flakiness the
previous version masked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Measured against the running stack, not guessed:
- LAM sits behind the SSO proxy, so the first form is Keycloak's and rejects an LDAP bind DN as its username
- LAM offers its admin accounts as a select, so filling a text input never reached it and the bind was skipped silently
- hasText cannot see the value attribute of an input[type=submit], so the login and save buttons are addressed by role
- the vhost rewrites only "/" to "/lam/", and treeView.php base64-decodes ?dn
- the directory names people by uid=, so a cn= member DN pointed at nothing
- asserting the absence of "error" matches LAM's own hidden message templates, so the written value is checked instead
LDAP_ROLE_DN_BASE is wired through because the role groups live under ou=roles.
Unproven: the spec has not executed in any run - its title appears zero times in four deploy logs while sibling titles from the same file appear 24, 12 and 24 times. Everything from the tree navigation onward is therefore untested.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sixteen role specs decided something immediately after an action that had not settled, because locator.isVisible() discards the timeout it is handed. Three shapes: an assertion that could never fire, a login outcome read off the DOM before the app re-rendered, and a branch that selected a locator by whether an iframe had mounted yet.
The ones that could never fire are the worst of them. taiga test-seaweedfs polled for a rejection toast right after saveAction.click(), so a real upload rejection was swallowed and misreported as a SeaweedFS storage failure; friendica ldapauth decided whether to run its wrong-password test at the instant a fresh context finished goto, so a slow render skipped a security check silently. wordpress test-discourse-roundtrip picked titleBox by whether the Gutenberg iframe had mounted and, when it had not, bound the wrong one and then burned the full budget failing with a misleading message.
Every conversion keeps the control flow: the .catch(() => false) fallback stays, so a waitFor that throws still yields the boolean the caller expects. Twenty of the twenty-two waits use base 2000, one 3000, one 5000. resolveTimeout multiplies by five on an onion target, so the common case is ten seconds there. Across thirteen onion traces the largest wait of any kind that ever succeeded was 790 ms, so ten seconds is a twelvefold margin; the bases started higher and were pulled down because a swallowed deadline is a duration in disguise - where the optional element is normally absent, the full timeout is paid on every run, including green ones.
Not exercised: no spec was run. Three of these make a test stricter, so a job that turns red on them is surfacing a real failure that was previously hidden, not regressing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The provisioner fell back to the declared connections only while the
instance had none. Once it had one, the declared set was ignored
entirely, so a newly declared provider was dropped without a word and
the grant set never grew. Declared and existing entries are merged now,
and url, path, key and auth_type are refreshed on an entry already
present, so a provider that moves or rotates its secret is updated
rather than frozen.
The final check compared counts, which a swap would have passed. It
compares identities and names whatever the write lost.
Four defects in one chain kept the tool servers from ever being served.
The connections never arrived. The provisioner read them back from the config
API, but ENABLE_PERSISTENT_CONFIG=false means the environment feeds the running
process and is never written to the table that API serves, so a fresh
deployment answered with an empty list however long the deploy waited. It now
falls back to the declared list it was handed.
The groups stayed empty. Group resolution returned only the id and dropped the
group itself, so membership could not be written. It returns both, and the
member list is reconciled explicitly, because the OIDC claim cannot revoke on
its own: Open WebUI drops a stale membership only while the claim is non-empty,
so a user who loses their last group keeps it until some later login carries
another.
Success was not checkable. The script printed a count and exited zero even when
nothing was granted. It now refuses to report success when servers are declared
and none was granted, and emits the resolved group ids, which are persisted and
rendered back into the environment. Without that second render a bare container
restart drops the grants, since the environment is authoritative on every start
and the ids did not exist when it was first written.
The signing key is pinned because Open WebUI otherwise generates it into the
process environment only, where a container exec sees a different value than
the server.
The mcp role sat in the administrator's unscoped roles list. That list carries
no application, and the RBAC builders admit any user whose roles contain the
name, so the single entry granted mcp in every deployed application at once.
It is replaced by application_roles, a mapping of application to roles, which
the user cache now reads and passes through. Both directory builders resolve
membership through one shared function, so the Keycloak and LDAP paths cannot
drift.
The member list never reached its consumer. mcp_group_members yielded username
strings while the Open WebUI reconciler calls member.get("email"), so the first
real grant would have crashed the deploy. It went unnoticed because the
reconciler returns early on an empty list and nobody held the role. The filter
now yields username and email, which is what the reconciler needs to match
accounts Open WebUI created from an OIDC claim, whose display name need not be
the username. A test drives the real filter output through the real script so
the two sides cannot diverge again.
The Playwright scenario walks the boundary end to end: biber is served no MCP
tools, the administrator adds him to the provider's LDAP mcp group in LAM, and
after signing in again the tools are served. LAM is reached only when it is
present in the inventory, so no service edge or variant is added for it.
The administrator spec inverts its last assertion. With the role scoped, an
administrator holds no application's mcp group, and BYPASS_ADMIN_ACCESS_CONTROL
is false, so the tools must not be served to them either.
Eight server roles declare an `mcp` RBAC role, so the platform mints
roles/<app>/mcp and the administrator joins it. Open WebUI now honours those
groups instead of flattening them to user/admin, and each MCP tool server is
restricted to the group of the role that serves it.
The grant cannot be rendered: Open WebUI matches it against a group id that
insert_new_group mints as a fresh UUID, so tasks/01_mcp.yml resolves the group
after the container is up and writes the grant through the admin API, the way
every instance-minted credential in this repository is provisioned.
Entries render disabled. An empty access_grants is not 'nobody', Open WebUI
reads it as every administrator, and ENABLE_PERSISTENT_CONFIG=false makes the
env authoritative again on every start. Enabling only together with the grant
keeps the fallback at no access rather than too much.
The provisioning account is the role-internal openwebui-api-bot, never a
platform identity: OAUTH_MERGE_ACCOUNTS_BY_EMAIL ships disabled and an OIDC
login whose email a local account already holds is refused, so seeding a
person's address would lock them out.
Known limits are documented rather than claimed away: a bare restart leaves MCP
off until the next deploy, a user's roles list carries no application context so
roles: [mcp] grants every deployed mcp group at once, and Open WebUI drops a
stale membership only while the groups claim is non-empty.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Register every discovered server as an Open WebUI tool server, skipping
the auth schemes its ToolServerConnection cannot present so none is
stored with an unusable credential. Serialise that value with compact
separators: a space makes dotenv_quote wrap it, swarm keeps the quotes
inside the value, and Open WebUI then parses a string instead of a list
and crash-loops on startup.
Prepare the Flowise instance instead of registering servers in it.
Flowise 3.1.3 exposes no instance-level MCP registry; MCP lives in a
per-flow Custom MCP node. The role pins the transport to URL-based
servers so no flow can spawn a local command, and relaxes the host deny
list far enough to reach a container origin while loopback and the
cloud-metadata addresses stay denied. Selecting a server inside a flow
stays an operator step, and the README says so.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Open WebUI talked to Ollama directly. It now speaks the OpenAI-compatible surface of svc-ai-litellm with a per-consumer virtual key, so a second backend or an external provider becomes a gateway config change instead of a role change. A lint keeps future roles off the model servers.
The gateway advertised a hardcoded model list (ollama/llama3, ollama/nomic-embed-text) while the matrix variant preloads smollm2:135m, so it offered only models the backend did not have and hid the one it did. The list is derived from the ollama preload SPOT now.
The chat spec followed the rename: it gates on litellm instead of the removed ollama service flag, and picks the first non-embedding model from /api/models rather than pinning a variant-specific name, so it exercises the chain rather than a fixture.
Routing openwebui through the gateway pulls both model backends into the closure of every role that consumes it, which pushed nextcloud variant 2 over the memory budget. The budget stays at 32/64 GB; instead the matrix variants of svc-ai-ollama and svc-ai-lmstudio now declare the size their test model needs (1g/2g) while the base keeps the 6g/8g that llama3 and mistral require. The budget guard only saw half of that: it replaced the evaluated role with its variant but counted every dependency from meta/services.yml, a combination no matrix round deploys. It now evaluates every variant-carrying role at the same round index, falling back to variant 0, the way the deploy planner does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The applications lookup returns the raw domains map that never receives
onion injection, so on an onion node CANONICAL_DOMAIN stayed clearnet
while the app itself rode onion (SOCKS proxy set). isOnionTarget() then
returned false: onion skips never fired and timeout scaling stayed at
clearnet levels. lookup('domain') resolves through the injected map and
is byte-identical on clearnet (single-entry canonical).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When an app rides onion, Playwright routes through the node Tor SOCKS
proxy, which cannot resolve the clearnet-internal SeaweedFS filer UI
domain (net::ERR_SOCKS_CONNECTION_FAILED). Skip the filer storage check
on onion targets in all consumer specs — same pattern already proven in
the nextcloud spec. Clearnet runs are unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New lint test_no_raw_goto.py forbids raw `page.goto(...)` in role playwright specs + persona flows (the gotoOnion wrapper in personas/utils is exempt) so navigation retries transient Tor-transport errors over onion. Migrates every raw call across ~75 roles (343 sites) to `gotoOnion(page, ...)` and teaches gotoOnion to detect relative onion URLs (`isOnionCanonical`). gotoOnion is a behaviour-identical drop-in on clearnet (single attempt, same return). Verified: no-raw-goto lint green, eslint 0 errors, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the global TIMEOUT_FACTOR and a shared timeouts.js helper: resolveTimeout(base) scales by TIMEOUT_FACTOR and, over a .onion target, an extra multiplier, so Tor's per-request latency no longer trips hard-coded timeouts. Migrate every raw timeout in the specs (timeout:/timeout=/setTimeout/waitForTimeout) to resolveTimeout, stage the helper in all three staging paths, and add test_no_raw_timeouts to forbid raw numeric timeouts. Clearnet behaviour is unchanged (factor 1: resolveTimeout(N) === N).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- svc-ai-ollama variant-0: preload smollm2:135m (smallest instruct model for CI)
- integration_openai meta: Service URL -> {base}/api for the OpenWebUI backend
- web-app-openwebui: enable_api_keys config flag (services.yml + vars) rendered into
ENABLE_API_KEYS so sk- API keys authenticate against the OpenAI-compatible API
- integration_openai spec: drop the DOM-gated skip, assert the partner backend host
(openwebui chat.ai / flowise flow.ai), distinct from Nextcloud
- openwebui: OLLAMA_SERVICE_ENABLED playwright flag + test-ollama-chat.js asserting
the preloaded model answers a chat completion over OpenWebUI -> Ollama
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add files/playwright/test-seaweedfs.js (+ SEAWEEDFS_* playwright.env vars, spec require wiring, and removal of the playwright-service-flag nocheck above the seaweedfs service) for baserow, bookwyrm, mattermost, openwebui, peertube, penpot, pixelfed. Each test reuses the role's real login flow and triggers an upload that writes to the app's S3 bucket (file-field, avatar, chat attachment, composer photo, video), verified via the shared runSeaweedfsStorageCheck (bucket grew through the Filer UI).
openwebui: also export dismissAllOpenModals from _shared.js (used by the new test) and match the uploaded file by basename (not the full filename) to avoid the extension-split false-negative.
Exempt gitea and opentalk via meta/services.yml nocheck markers (seaweedfs-playwright + playwright-service-flag): gitea has no automatable UI login in this stack (oauth2-proxy + OIDC account-link / native login both fail; personas blocked); opentalk only writes S3 for recordings/whiteboard exports, which need the recorder service + a live multi-party meeting.
Builds on 9b6c6415f (shared runSeaweedfsStorageCheck helper + consumer lint).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`skipUnlessServiceEnabled("oidc")` tripped service-gating's unknown-service
guard after the oauth2+oidc → sso collapse. Rename the gate to `sso` so
both admin and biber persona tests skip/run via the rendered
`SSO_SERVICE_ENABLED` flag.
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mirror the web-app-wordpress/web-app-mattermost shape: extract env and
helpers into _shared.js and move each scenario into its own
test-*.js module that registers via the shared context.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Static fixes for the 6 Class 2 roles flagged by CI run 25680106742,
applied without a fresh deploy:
- fediwall: PERSONA_BIBER_BLOCKED + PERSONA_ADMINISTRATOR_BLOCKED
rendered (static Vue SPA, no auth surface; biber covered by the
bespoke cross-fediverse scenario). Confidence 30% → 75%.
- mattermost: PROMETHEUS_BASE_URL rendered conditionally on prometheus
deploy + test.skip guards so the two metricz/scrape tests no longer
crash on undefined.
- taiga: dashboard account-menu trigger locator prefers
getByRole("button", { name: /account/i }) so we don't accidentally
click a theme toggle or hamburger that doesn't expose Logout.
- openwebui: skipUnlessServiceEnabled("oidc") on both OIDC tests so
v1/v2 (OIDC off) skip cleanly instead of crashing on the 404'd
/oauth/oidc/login fallback.
- bigbluebutton: skipUnlessServiceEnabled("oidc") + Greenlight
?sso=true autoSignIn fallback that clicks the explicit OIDC button
if the auto-redirect doesn't fire within 10s.
- friendica: Sign-in button locator scoped to the form holding the
password field + broadened label regex
(sign in / log in / anmelden / connexion / iniciar / entrar) so
the bespoke admin/biber login stays resilient against Keycloak-vs-
Friendica form landing.
Matrix updated: fediwall jumps to 75%, the other five carry per-row
notes documenting what landed and what still needs deploy-time
verification.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Promote four cheap rules in `eslint.config.js`:
* `no-unused-vars`: warn → error (with `argsIgnorePattern: "^_"`)
* `no-var`: error
* `eqeqeq`: error (always)
* `playwright/no-wait-for-navigation`: off → error
* `linterOptions.reportUnusedDisableDirectives`: off → warn
Add a repo-root CommonJS block so `eslint.config.js` itself sees Node
globals (`require`, `module`).
Fix every flagged offender:
* 138 stale persona-helper imports across 70 spec files
(`isServiceEnabled`, `skipUnlessServiceEnabled`, `runAdminFlow`,
... — leftovers from the SPOT consolidation).
* 4 unused catch bindings → modern `catch {}` syntax.
* 2 unused `dashboardBaseUrl` declarations (minio, moodle).
* 1 unused `label` arg → renamed to `_label` to opt out per the
argsIgnorePattern.
* 1 intentionally-kept legacy helper in web-app-wordpress's spec
receives a per-line `eslint-disable-next-line` with reason; the
preceding docstring already explained why it stays.
* 1 `page.waitForNavigation` in web-app-fediwall's Mastodon OIDC flow
receives a per-line disable with reason — the navigation target
depends on the runtime OIDC issuer; `waitForURL` would need a
runtime-built pattern.
* 10 stale `// eslint-disable-next-line no-console` directives now
surfaced by `reportUnusedDisableDirectives`; removed.
ESLint after this pass: 0 errors, 154 warnings (148 pre-existing
`playwright/expect-expect` plus 6 `playwright/no-force-option` that
were already inherited from `playwright/flat/recommended`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`,
`requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from
non-Markdown source files. Comments anchored to requirement numbers
rot when requirements are renumbered or rewritten and conflate
present-state behaviour (which the comment should describe) with the
audit trail of how the code got there (which belongs in commit
messages, PR descriptions, and the requirement file itself).
* Document the rule in `docs/contributing/documentation.md` under
"Comments". Code MUST describe the *why* in present-state language;
contract enforcement lives in tests, not folklore.
* Clean 254 source files across `roles/`, `tests/`, `plugins/`,
`cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root
`eslint.config.js`. Pure comment / docstring / log-message edits;
no behavioural change.
* Add `tests/lint/repository/test_no_req_references_in_code.py` to
enforce the rule going forward across `.js`, `.py`, `.yml`,
`.yaml`, `.j2`, and `.sh` files. The legacy
`test_no_stale_requirement_refs.py` is exempt as the only file
whose purpose is to detect these very tokens.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Seven web-app-* specs failed the Rule 3 contract from
docs/contributing/artefact/files/role/playwright.specs.js.md:
- bigbluebutton, decidim, discourse, matrix, openwebui, peertube
each shipped `administrator:` and `biber:` test titles but no
`guest:` test.
- keycloak shipped four detailed admin / biber tests under non-
canonical names (`master-realm super administrator…`, `normal-
realm administrator logs in through account interface…`, etc.)
and no `<persona>: <flow>`-prefixed scenarios at all.
None of the seven declared a `PERSONA_<persona>_BLOCKED=true`
opt-out in their `templates/playwright.env.j2`, and none qualify
for the auth-less collapse exception (every role has an OIDC auth
surface; keycloak IS the auth surface).
Changes
-------
- Each of the six guest-only-missing specs gains a single appended
scenario:
test("guest: public-landing → auth chain → never authenticated", …)
→ runGuestFlow(page);
- Keycloak gains all three persona scenarios (guest, biber,
administrator) using the shared `runGuestFlow` / `runBiberFlow` /
`runAdminFlow` runners. Existing detailed tests stay untouched
alongside the persona contract.
- The `./personas` import in each spec is amended with whichever
runners are now needed.
Verification
------------
- 420 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors.
- 0 Rule-3 violators remain across all `web-app-*/files/playwright.spec.js`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fourteen role specs (bigbluebutton, discourse, fider, gitea, mailu,
matrix, mattermost, odoo, openwebui, peertube, postmarks, prometheus,
yourls, libretranslate) each carried an inline `performOidcLogin`
that drove the Keycloak login form. The bodies fell into four
clusters split between the role-based selector strategy
(`getByRole`) and the legacy input-name strategy
(`input[name='username']`); the existing shared
`performKeycloakLogin` already covered both via `.or()` chains but
also asserted post-login navigation, so it could not be reused
verbatim.
Changes
-------
- `personas/utils/keycloak.js` gains `performKeycloakLoginForm
(target, username, password)` — form-fill + submit only, accepts
`Page` OR `FrameLocator` so iframe-embedded Keycloak pages work
without branching at the call site. The selectors are the union
of all four prior cluster shapes (role-based getter chained with
`.or(input-name fallback)`, sign-in matched against
`/sign in|login|log in/i` plus `input#kc-login` family). The
existing `performKeycloakLogin` and
`performKeycloakLoginExpectingDenial` now delegate to the new
helper, so their public API is unchanged.
- The 14 specs strip their inline `performOidcLogin` and amend
`require("./personas")` with `performKeycloakLoginForm`. Every
call site and comment reference is renamed in lockstep.
Drift handling
--------------
- Cluster 1 (5 specs) expected the accessible name `"Password"`
exactly; the canonical regex `/^password$/i` is case-insensitive
and matches every prior name plus lowercase variants — strictly
more permissive.
- Cluster 1's sign-in button matched `/sign in/i`; the canonical
matches `/sign in|login|log in/i` plus the legacy `input#kc-login`
family — strictly broader.
- Cluster 1 awaited `usernameField.press("Tab")` without a `.catch`;
the canonical wraps the press in `.catch(() => {})` so an
unexpected Tab failure no longer aborts the flow.
- Cluster 2 / 3 used `expect(usernameField).toBeVisible({ timeout })`
for the visibility wait; the canonical uses
`usernameField.waitFor({ state: "visible", timeout })`, the
Locator-method counterpart of the same wait — functionally
equivalent.
Verification
------------
- 419 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors, 283 warnings (unchanged from
baseline).
- 0 inline `performOidcLogin` definitions remain in any role spec,
and 0 stale references to the legacy name in any context (call
sites and comments are renamed in the same pass).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Nine role specs (bigbluebutton, dashboard, discourse, keycloak,
matomo, matrix, moodle, openwebui, wordpress) each carried inline
copies of the same six CSP helpers and the `EXPECTED_CSP_DIRECTIVES`
constant. The bodies were semantically identical; only message
wording, brace style, and one captured-field-set differed.
Changes
-------
- Extends `roles/test-e2e-playwright/files/personas/utils/csp.js`
(already home to `assertCspInjections`) with
`installCspViolationObserver`, `readCspViolations`,
`parseCspHeader`, `assertCspResponseHeader`,
`assertCspMetaParity`, `expectNoCspViolations`, and the
`EXPECTED_CSP_DIRECTIVES` constant. The helpers are re-exported
through `personas/utils/index.js` so specs that
`require("./personas")` get them via the existing aggregator.
- Strips the inline definitions from the nine role specs and amends
each spec's `require("./personas")` destructuring with whichever
helpers the body still references.
Drift handling
--------------
- moodle's reduced violation-record shape (only `violatedDirective`
+ `blockedURI`) is replaced by the canonical 5-field record.
Moodle's downstream consumer asserts `.toEqual([])` on the array
only, so the extra fields are diagnostic-only — no behaviour
change.
- dashboard's 2-arg `expectNoCspViolations(page, label)` call site
is updated to the canonical 3-arg form
`expectNoCspViolations(page, null, label)`. The shared helper
guards `if (diagnostics && Array.isArray(diagnostics.cspRelated))`
before checking the diagnostics arm, so passing `null` keeps the
exact pre-migration behaviour for that call site.
- bigbluebutton and matrix had subtly different failure-message text
in `assertCspResponseHeader` and `expectNoCspViolations`. The
canonical wording is more descriptive; passing tests are
unaffected, failing tests get clearer diagnostics.
Doc
---
`docs/contributing/artefact/files/role/playwright.specs.js.md` is
extended in two places: the invariants section names the full CSP
test surface and pins the "import from `./personas`, no inline copy"
rule, and the per-service catalogue's CSP / injectors row lists the
helpers by name.
Verification
------------
- 419 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors, 283 warnings (unchanged from
baseline).
- `markdownlint-cli2`: 0 errors on the touched doc.
- 0 inline definitions of any extracted helper / constant remain in
any role spec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both helpers were inlined verbatim in 71 of the role-local
`playwright.spec.js` files (52 with the compact form, 19 with the
verbose form, plus one parameter-name variant). All bodies were
semantically identical.
Changes
-------
- New `roles/test-e2e-playwright/files/personas/utils/dotenv.js` holds
the canonical pair (`decodeDotenvQuotedValue` and `normalizeBaseUrl`),
re-exported through `personas/utils/index.js` so specs that
`require("./personas")` get them via the existing aggregator.
- Every role spec drops its inline copies and pulls the helpers from
`./personas` instead. Specs that only reference one of the two
carry only that one in the destructuring (no unused-import noise).
Doc
---
`docs/contributing/artefact/files/role/playwright.specs.js.md` no
longer carries the "once a third spec needs them, MUST be promoted"
directive: the helpers live at the shared path and inline copies are
forbidden.
Verification
------------
- 419 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors, 283 warnings (unchanged from
baseline; no new unused-imports introduced).
- 0 inline `function decodeDotenvQuotedValue|normalizeBaseUrl`
definitions remain in any role spec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires runGuestFlow / runBiberFlow / runAdminFlow from
roles/test-e2e-playwright/files/personas/ into the remaining 5 roles
(flowise, minio, opentalk, openwebui, wordpress). Each role gets the
spec.js persona scenarios plus matching meta/services.yml gate entries
and playwright.env.j2 env wiring; admin-flow callers pass a
role-specific adminInteraction that drives a real management surface.
Reason: continuation of d3e5ebe44 / 44d51744a — closes the long-tail
role list against the central library so per-role drift cannot
reintroduce the stub-shaped bodies the new lint forbids.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes Verification checkboxes in requirement 003. Each spec exercises
the `lookup('applications')` + `lookup('users')` pathway end-to-end:
login via lookup('users'), OIDC round-trip from dashboard (where the
app supports it), canonical-domain DOM assertion, and a full CSP
assertion covering every directive emitted by csp_filters.build_csp_header
(15 directives), meta-parity, enforced policy (not Report-Only), and
zero securitypolicyviolation events.
The dashboard spec is extended in place with the CSP + canonical-domain
assertions while keeping its existing login/logout coverage.
rerun-spec.sh is pointed at the Jinja template + the staged copy of
package.json (the repo has templates/package.json.j2, not files/package.json),
so the inner spec-only iteration loop works without re-deploying.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
