92c45e2471 moved Open WebUI off the model server onto the gateway and renamed
test-ollama-chat.js to test-litellm-chat.js in the same commit, git records it
as R073. `git ls-tree` on that commit shows only the successor in the
directory, and both OLLAMA_* variables left playwright.env.j2 with it.
fork/feature/svc-net-tor still carried the pre-refactor file. Merging it in
7162b7bf4c restored test-ollama-chat.js and OLLAMA_PRELOAD_MODEL without a
matching add: `git log --diff-filter=A` on that path knows only the original
creation from 06aa25c095. A scan of all 916 merges reachable from HEAD finds
exactly one other path resurrected the same way, .github/pull_request_template.md,
which is not a role file and is left alone here.
Deleting it removes no coverage. The file never executed: it does not match
the collector's `**/*.@(spec|test).js` and no aggregator required it, and the
JUnit report from openwebui#1 lists thirteen tests with no `ollama chat`
among them, not even as a skip. Four of its five assertions are byte-identical
in the registered successor, which runs green in openwebui#1 and #2.
The fifth is deliberately weaker there: the successor takes the first
non-embedding model from /api/models instead of pinning OLLAMA_PRELOAD_MODEL,
so a green run proves the gateway serves a chat model rather than that Ollama
specifically does. That was the refactor's stated intent, exercising the chain
rather than a fixture, and reviving the file would route a spec straight at a
model server that a lint exists to keep roles away from.
OLLAMA_PRELOAD_MODEL goes with it: the deleted spec was its only reader. The
plural OLLAMA_PRELOAD_MODELS in svc-ai-ollama is a different variable and
stays.
GET /api/v1/tools/ builds its answer for an MCP server out of stored config alone - routers/tools.py:125-126 reads info off the connection row and never touches the network. So the three specs that read it pass unchanged against a provider that is down, refuses the deployed bearer, or serves nothing. What they establish is registration and the OIDC group to access-grant chain, not that the bridge carries.
POST /api/v1/configs/tool_servers/verify is the one route in 0.11.0 that does a real MCP initialize plus tools/list without an LLM, and it authenticates with the key in the body it is handed. The spec reads the stored connection and echoes it back, so the handshake runs over the deployed URL with the deployed credential through Open WebUI's own client, and asserts three things per server: it answers, it serves at least one tool, and the tool set contains the read probe its meta/mcp.yml declares.
Only bearer connections are verified. A system_oauth one resolves the caller's own upstream token per request and the administrator holds none, so it would fail on the caller rather than on the bridge; no provider declares auth: oidc today, but the filter keeps the spec honest when one does. The per-test budget scales with the number of registered servers instead of a fixed guess.
MCP_EXPECTED_TOOLS carries the declared read probe per server. Its rendering is unit-tested by extracting the block from the template itself rather than restating it, and decoded the way a spec decodes it - dotenv_quote emits an escaped JSON string that decodeDotenvQuotedValue unwraps before the spec parses the payload, so comparing after one decode would test a value no spec ever sees.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The matrix variant preloaded smollm2:135m, which is too small to emit a
stop token reliably. Ollama's own log for the prompt "hi" shows it running
away rather than running slowly:
n_gen = 22011, tg = 86.14 t/s
slot context shift, n_keep = 4, n_left = 4091, n_discard = 2045
aborting completion request due to client closing the connection
86 tokens per second is fast; 22000 tokens with repeated context shifts is
a model that never finishes. Every completion without a hard token cap ran
until the consumer's own timeout, which is why zammad reported
AI::Provider::ResponseError after exactly 300s while short warm-ups and
provider pings returned 200 immediately.
Preload qwen2.5:0.5b instead: a comparable footprint (397MB vs 270MB) that
is instruction-tuned and terminates.
web-app-openwebui asserted the served model with a hardcoded
id.startsWith("smollm2"), so it would have failed on any model change.
Render the deployed preload model into its Playwright env and compare
against that instead of a literal family name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.
Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Measured against the running stack, not guessed:
- LAM sits behind the SSO proxy, so the first form is Keycloak's and rejects an LDAP bind DN as its username
- LAM offers its admin accounts as a select, so filling a text input never reached it and the bind was skipped silently
- hasText cannot see the value attribute of an input[type=submit], so the login and save buttons are addressed by role
- the vhost rewrites only "/" to "/lam/", and treeView.php base64-decodes ?dn
- the directory names people by uid=, so a cn= member DN pointed at nothing
- asserting the absence of "error" matches LAM's own hidden message templates, so the written value is checked instead
LDAP_ROLE_DN_BASE is wired through because the role groups live under ou=roles.
Unproven: the spec has not executed in any run - its title appears zero times in four deploy logs while sibling titles from the same file appear 24, 12 and 24 times. Everything from the tree navigation onward is therefore untested.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The block sat under services but never behaved like a service. A service has
exactly one providing role, declared with provides, so the registry can map the
key to that role and consumers bind their flag to its presence. No role declares
provides: mcp. There are seventeen providers and the pairing is done by
allowed_consumers inside the block itself.
Every rule that enforces the service model therefore needed an exemption for it:
a shared-provider suppression in fourteen roles, a playwright-service-flag
suppression, and two freshly added exemptions in the dynamic-flag and
variants-static-override checks. Four independent rules, four escape hatches,
one cause. They are all gone now, together with a bond key that only the
resource budgeter reads and only for entries under services, and a wordpress
addon bridge that pointed at a service block that no longer exists.
The block moves to meta/mcp.yml. The applications loader maps meta/<name>.yml to
applications.<app>.<name> generically, so the move needs no wiring; the file
takes the same shape as meta/rbac.yml. Every config path follows, the variant
overrides are lifted out of their services map, and the vocabulary module leaves
the services package because it no longer describes one. roles_with_service
gains an explicit topic parameter rather than an implicit fallback, so nothing
silently resolves against the wrong container.
Files that are otherwise unrelated to this move are included where the committed
tree would otherwise keep pointing at a path that no longer resolves: thirteen
vars files, eleven playwright env templates and one task file.
The mcp role sat in the administrator's unscoped roles list. That list carries
no application, and the RBAC builders admit any user whose roles contain the
name, so the single entry granted mcp in every deployed application at once.
It is replaced by application_roles, a mapping of application to roles, which
the user cache now reads and passes through. Both directory builders resolve
membership through one shared function, so the Keycloak and LDAP paths cannot
drift.
The member list never reached its consumer. mcp_group_members yielded username
strings while the Open WebUI reconciler calls member.get("email"), so the first
real grant would have crashed the deploy. It went unnoticed because the
reconciler returns early on an empty list and nobody held the role. The filter
now yields username and email, which is what the reconciler needs to match
accounts Open WebUI created from an OIDC claim, whose display name need not be
the username. A test drives the real filter output through the real script so
the two sides cannot diverge again.
The Playwright scenario walks the boundary end to end: biber is served no MCP
tools, the administrator adds him to the provider's LDAP mcp group in LAM, and
after signing in again the tools are served. LAM is reached only when it is
present in the inventory, so no service edge or variant is added for it.
The administrator spec inverts its last assertion. With the role scoped, an
administrator holds no application's mcp group, and BYPASS_ADMIN_ACCESS_CONTROL
is false, so the tools must not be served to them either.
Eight server roles declare an `mcp` RBAC role, so the platform mints
roles/<app>/mcp and the administrator joins it. Open WebUI now honours those
groups instead of flattening them to user/admin, and each MCP tool server is
restricted to the group of the role that serves it.
The grant cannot be rendered: Open WebUI matches it against a group id that
insert_new_group mints as a fresh UUID, so tasks/01_mcp.yml resolves the group
after the container is up and writes the grant through the admin API, the way
every instance-minted credential in this repository is provisioned.
Entries render disabled. An empty access_grants is not 'nobody', Open WebUI
reads it as every administrator, and ENABLE_PERSISTENT_CONFIG=false makes the
env authoritative again on every start. Enabling only together with the grant
keeps the fallback at no access rather than too much.
The provisioning account is the role-internal openwebui-api-bot, never a
platform identity: OAUTH_MERGE_ACCOUNTS_BY_EMAIL ships disabled and an OIDC
login whose email a local account already holds is refused, so seeding a
person's address would lock them out.
Known limits are documented rather than claimed away: a bare restart leaves MCP
off until the next deploy, a user's roles list carries no application context so
roles: [mcp] grants every deployed mcp group at once, and Open WebUI drops a
stale membership only while the groups claim is non-empty.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Register every discovered server as an Open WebUI tool server, skipping
the auth schemes its ToolServerConnection cannot present so none is
stored with an unusable credential. Serialise that value with compact
separators: a space makes dotenv_quote wrap it, swarm keeps the quotes
inside the value, and Open WebUI then parses a string instead of a list
and crash-loops on startup.
Prepare the Flowise instance instead of registering servers in it.
Flowise 3.1.3 exposes no instance-level MCP registry; MCP lives in a
per-flow Custom MCP node. The role pins the transport to URL-based
servers so no flow can spawn a local command, and relaxes the host deny
list far enough to reach a container origin while loopback and the
cloud-metadata addresses stay denied. Selecting a server inside a flow
stays an operator step, and the README says so.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Open WebUI talked to Ollama directly. It now speaks the OpenAI-compatible surface of svc-ai-litellm with a per-consumer virtual key, so a second backend or an external provider becomes a gateway config change instead of a role change. A lint keeps future roles off the model servers.
The gateway advertised a hardcoded model list (ollama/llama3, ollama/nomic-embed-text) while the matrix variant preloads smollm2:135m, so it offered only models the backend did not have and hid the one it did. The list is derived from the ollama preload SPOT now.
The chat spec followed the rename: it gates on litellm instead of the removed ollama service flag, and picks the first non-embedding model from /api/models rather than pinning a variant-specific name, so it exercises the chain rather than a fixture.
Routing openwebui through the gateway pulls both model backends into the closure of every role that consumes it, which pushed nextcloud variant 2 over the memory budget. The budget stays at 32/64 GB; instead the matrix variants of svc-ai-ollama and svc-ai-lmstudio now declare the size their test model needs (1g/2g) while the base keeps the 6g/8g that llama3 and mistral require. The budget guard only saw half of that: it replaced the evaluated role with its variant but counted every dependency from meta/services.yml, a combination no matrix round deploys. It now evaluates every variant-carrying role at the same round index, falling back to variant 0, the way the deploy planner does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The applications lookup returns the raw domains map that never receives
onion injection, so on an onion node CANONICAL_DOMAIN stayed clearnet
while the app itself rode onion (SOCKS proxy set). isOnionTarget() then
returned false: onion skips never fired and timeout scaling stayed at
clearnet levels. lookup('domain') resolves through the injected map and
is byte-identical on clearnet (single-entry canonical).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lookup('canonical_url', app_id[, key][, consumer=app]) resolves the web base
URL for an app or one of its named canonicals (element/synapse/filer),
following the onion-injected merged-domain map: .onion targets resolve to
http (Tor provides transport security), everything else follows TLS_ENABLED.
The consumer= kwarg returns '' unless the consuming app has the target's
service binding enabled, replacing the recurring
"(lookup(...) if lookup('config', app, 'services.X.enabled') else '')"
template branching.
- utils/cache/domains.py: inject the node onion into dict-form
(named-canonical) domain declarations, not just list/str forms
- keycloak redirect_uris: onion redirect URIs are http
- 13 playwright.env.j2: manual scheme branching -> canonical_url(consumer=)
- lint: forbid manual '://' concatenation and to_onion_url in role templates
- unit tests: 10 cases (primary/named x clearnet/onion, consumer gating)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extract the csp, domains, and networks blocks out of roles/*/meta/server.yml
into dedicated meta files, each mapping to its own top-level applications key
via the recursive meta-walk:
csp: -> meta/csp.yml -> applications.<app>.csp
domains: -> meta/domains.yml -> applications.<app>.domains
networks: -> meta/networks.yml -> applications.<app>.networks
Residual nginx/proxy keys (status_codes, client_max_body_size, locations,
ip6_enabled, config_upstream_url) stay in server.yml; an emptied server.yml is
deleted. 98 server.yml -> 249 new files, 74 deleted.
- Consumers repointed applications.<app>.server.{csp,domains,networks} ->
applications.<app>.{csp,domains,networks} across templates, vars, tasks,
lookups and filters.
- Infra readers migrated: canonical_domains_map, application_domain_index,
build_ldap_role_entries, service-registry overlay discovery,
roles_with_service, and the file-based readers utils/domains/list.py +
utils/meta/scan.py (read the now-deleted server.yml directly).
- Detection + mapping: _META_TOPICS gains csp/domains/networks;
ROLE_FILE_META_{CSP,DOMAINS,NETWORKS} constants + ROLE_FILES entries.
- New self-expiring lint guard (test_server_meta_no_split_fields) fails if a
server.yml still carries a split field; auto-fails from 2026-12-28 as a
delete reminder.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- discourse-activity-pub, integration_matrix specs: drop unused catch
binding and useless json init (eslint no-unused-vars/no-useless-assignment)
- openwebui playwright.env.j2: order OLLAMA_SERVICE_ENABLED after
OIDC_ISSUER_URL (env keys must be ASC, test_alphabetical_sort)
- test_addon_env_flags.py: autoformat
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- svc-ai-ollama variant-0: preload smollm2:135m (smallest instruct model for CI)
- integration_openai meta: Service URL -> {base}/api for the OpenWebUI backend
- web-app-openwebui: enable_api_keys config flag (services.yml + vars) rendered into
ENABLE_API_KEYS so sk- API keys authenticate against the OpenAI-compatible API
- integration_openai spec: drop the DOM-gated skip, assert the partner backend host
(openwebui chat.ai / flowise flow.ai), distinct from Nextcloud
- openwebui: OLLAMA_SERVICE_ENABLED playwright flag + test-ollama-chat.js asserting
the preloaded model answers a chat completion over OpenWebUI -> Ollama
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add files/playwright/test-seaweedfs.js (+ SEAWEEDFS_* playwright.env vars, spec require wiring, and removal of the playwright-service-flag nocheck above the seaweedfs service) for baserow, bookwyrm, mattermost, openwebui, peertube, penpot, pixelfed. Each test reuses the role's real login flow and triggers an upload that writes to the app's S3 bucket (file-field, avatar, chat attachment, composer photo, video), verified via the shared runSeaweedfsStorageCheck (bucket grew through the Filer UI).
openwebui: also export dismissAllOpenModals from _shared.js (used by the new test) and match the uploaded file by basename (not the full filename) to avoid the extension-split false-negative.
Exempt gitea and opentalk via meta/services.yml nocheck markers (seaweedfs-playwright + playwright-service-flag): gitea has no automatable UI login in this stack (oauth2-proxy + OIDC account-link / native login both fail; personas blocked); opentalk only writes S3 for recordings/whiteboard exports, which need the recorder service + a live multi-party meeting.
Builds on 9b6c6415f (shared runSeaweedfsStorageCheck helper + consumer lint).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
scripts/install/venv.sh
-----------------------
INFINITO_VENV_BASE is a static-default convention; the active VENV
may end up outside that base (e.g. \$HOME/.venvs/<name> when
/opt/venvs is not writable, or a wholly custom override). Probing
writability on the legacy base triggered a spurious sudo branch
whenever VENV lived elsewhere -- fatal in no_new_privs sandboxes
where sudo cannot escalate. Switch the writability + mkdir + chown
chain to \$(dirname "\${VENV}") so the script prepares the directory
it is actually about to create the venv in.
tests/lint/ansible/roles/web-app/playwright/test_app_base_url_uses_role.py
-------------------------------------------------------------------------
New lint: every role's templates/playwright.env.j2 must declare
APP_BASE_URL as a reference to the role's own surface. APP_BASE_URL
feeds the shared guest-persona helper and Playwright's global
baseURL; a role that hardcodes a sibling role name (most commonly
'web-app-dashboard') sends the guest test to the wrong site and
breaks any matrix variant where the sibling is not deployed
(ERR_SSL_UNRECOGNIZED_NAME_ALERT).
The 16 web-app role templates that violated the contract switch
from sibling lookups (`lookup('tls', 'web-app-dashboard', ...)`)
to the canonical `lookup('tls', application_id, ...)` form.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rename env/static.env → env/default.env across the entire repo
(string label, doc links, Python file-path constants) — the
`parse_static_env*` function names stay as a stable API surface.
Sort all `*.env` / `*.env.j2` entries alphabetically (52 in
default.env, 4 in ci.env, 46 of 49 j2 templates; 3 templates
with Jinja2 `{% %}` control flow are skipped because conditional
blocks would tear apart under a naive sort). Sort all 83
Makefile target blocks alphabetically; .PHONY group headers
stay in place.
Collapse the two remaining multi-line Makefile recipe comments
(dotenv-force, test-signed) to single-line WHY comments.
Add four lint tests under tests/lint/filesystem/ to keep the
invariants enforced going forward:
- env/test_alphabetical_sort.py
- env/test_single_line_comments.py (# nocheck: markers exempt)
- makefile/test_alphabetical_targets.py
- makefile/test_single_line_comments.py (# nocheck: markers exempt)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`,
`requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from
non-Markdown source files. Comments anchored to requirement numbers
rot when requirements are renumbered or rewritten and conflate
present-state behaviour (which the comment should describe) with the
audit trail of how the code got there (which belongs in commit
messages, PR descriptions, and the requirement file itself).
* Document the rule in `docs/contributing/documentation.md` under
"Comments". Code MUST describe the *why* in present-state language;
contract enforcement lives in tests, not folklore.
* Clean 254 source files across `roles/`, `tests/`, `plugins/`,
`cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root
`eslint.config.js`. Pure comment / docstring / log-message edits;
no behavioural change.
* Add `tests/lint/repository/test_no_req_references_in_code.py` to
enforce the rule going forward across `.js`, `.py`, `.yml`,
`.yaml`, `.j2`, and `.sh` files. The legacy
`test_no_stale_requirement_refs.py` is exempt as the only file
whose purpose is to detect these very tokens.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cross-service surface assertions for dashboard, prometheus and matomo
no longer run in every consumer role's persona. They are owned by the
provider's own spec, parameterised once over a `<NAME>_TARGET_ROLES_JSON`
manifest rendered into the provider's `templates/playwright.env.j2` via
the new `lookup('roles_with_service', '<svc>')` Ansible lookup.
Changes
-------
- New `plugins/lookup/roles_with_service.py` enumerates roles that
declare `services.<svc>.{enabled, shared}` truthy in the merged
applications config, returning `[{id, canonical_domain}, …]`.
- `roles/web-app-{dashboard,prometheus,matomo}/files/playwright.spec.js`
parse `<NAME>_TARGET_ROLES_JSON` and parameterise the per-consumer
assertion (dashboard tile reachability, prometheus `up=1`, matomo
SitesManager registration). The matomo spec gains an explicit
biber-denial test (request body validated against admin DOM markers
per the body-marker rule); the prometheus admin-reach test drops
the dashboard-iframe path in favour of direct goto.
- Persona helpers `roles/test-e2e-playwright/files/personas/{admin,
biber,guest}.js` enter at `${APP_BASE_URL}/` directly. The
`clickRoleTileFromDashboard` step and the `assertPrometheus*` /
`assertMatomo*` cross-service probes are gone; `utils/{dashboard,
prometheus,matomo}.js` are deleted. `utils/csp.js` switches to
literal `process.env.<KEY>` reads so the env-keys-used parity guard
recognises matomo's CSP injector consumption.
- 42 consumer-role `templates/playwright.env.j2` templates drop
`DASHBOARD_BASE_URL`, `DASHBOARD_SERVICE_ENABLED`, `PROMETHEUS_BASE_URL`,
`PROMETHEUS_SERVICE_ENABLED`. `MATOMO_BASE_URL` and
`MATOMO_SERVICE_ENABLED` stay because the per-role CSP injection
helper still gates on matomo's tracker host.
- `tests/integration/roles/test_playwright_env_services_match.py`
globally exempts `dashboard` and `prometheus` from the
services.yml-to-env parity rule.
- `tests/unit/plugins/lookup/test_roles_with_service.py` adds 14
unit tests for the lookup, including an explicit-contract test
that pins the over-inclusive behaviour on raw Jinja strings (the
lookup expects a templar-rendered payload).
- Docs: `docs/requirements/019-playwright-meta-services-parity.md`
rewords Rules 1, 3, 12 and adds Rule 13 describing the SPOT shift.
`docs/contributing/artefact/files/role/playwright.specs.js.md`
reflects the new persona shape and per-service catalogue, and is
cleaned per `docs/contributing/documentation.md` (no SPOT self-
labels, no req-references, em dashes replaced with complete
sentences, multi-clause lines split onto separate lines, broken
anchor `#two-personas-fixed-flow-` corrected).
Verification
------------
- 418 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors.
- `markdownlint-cli2`: 0 errors on all touched docs.
- `make autoformat`: clean (FURB171 single-item-set inequality fixed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a lint that fails when a role's templates/playwright.env.j2 declares
DASHBOARD_SERVICE_ENABLED= but does NOT render DASHBOARD_BASE_URL,
PROMETHEUS_BASE_URL, and MATOMO_BASE_URL. Renders those three URLs
across 42 role env templates so the lint passes.
Also reverts the matomo "role-closed" strikethrough in req 019 — the
URL wiring remains in scope for the persona rollout.
Reason: the persona helpers (roles/test-e2e-playwright/files/personas/
{biber,admin,guest}.js) collapse the persona scenario when their
required URLs are empty (req 019 persona-collapse exception). Without
this lint, a role that declares dashboard: enabled but forgets the
URL silently SKIPs the persona — the deploy goes green but no real
flow ran. Per-line opt-out is `# nocheck: persona-required-envs` for
auth-less svc-* roles.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires runGuestFlow / runBiberFlow / runAdminFlow from
roles/test-e2e-playwright/files/personas/ into the remaining 5 roles
(flowise, minio, opentalk, openwebui, wordpress). Each role gets the
spec.js persona scenarios plus matching meta/services.yml gate entries
and playwright.env.j2 env wiring; admin-flow callers pass a
role-specific adminInteraction that drives a real management surface.
Reason: continuation of d3e5ebe44 / 44d51744a — closes the long-tail
role list against the central library so per-role drift cannot
reintroduce the stub-shaped bodies the new lint forbids.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes Verification checkboxes in requirement 003. Each spec exercises
the `lookup('applications')` + `lookup('users')` pathway end-to-end:
login via lookup('users'), OIDC round-trip from dashboard (where the
app supports it), canonical-domain DOM assertion, and a full CSP
assertion covering every directive emitted by csp_filters.build_csp_header
(15 directives), meta-parity, enforced policy (not Report-Only), and
zero securitypolicyviolation events.
The dashboard spec is extended in place with the CSP + canonical-domain
assertions while keeping its existing login/logout coverage.
rerun-spec.sh is pointed at the Jinja template + the staged copy of
package.json (the repo has templates/package.json.j2, not files/package.json),
so the inner spec-only iteration loop works without re-deploying.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
