Commit Graph
35 Commits
Author SHA1 Message Date
kevinveenbirkenbach 44a4e05f7c Merge branch 'main' into fix/web-app-n8n/sso-hook-logger/852 2026-10-14 13:35:46 +02:00
kevinveenbirkenbachandClaude Opus 5.5 8c0a670627 fix(web-app-n8n): start the gateway probe from its trigger under n8n 2
Since the role moved to n8n 2 (2.42.3 on 2026-10-05, now 2.44.0),
POST /rest/workflows/:id/run accepts a manual run only with a trigger to
start from or a destination node; any other key, `workflowData` among them,
is dropped. The LiteLLM gateway test still sent only `workflowData`, so every
attempt failed with HTTP 400 "To run the workflow manually, specify either a
trigger to start from or a destination node." (PR #912, run 38455633497,
job 115523214958). On main the test sat behind the MCP provisioning failure of
n8n#0 that e4078c05fd fixed, so the next main run would have hit it.

The test now runs the stored probe workflow from its manual trigger.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 12:01:49 +02:00
kevinveenbirkenbachandClaude Opus 5.5 a0c7023eab fix(web-app-n8n): log the SSO hook through n8n's own logger
The SSO hook reported its start, every sign-in, every created account and
every error through this.logger (#852). n8n 2 calls an external hook with
a context that holds only dbCollections, so this.logger is undefined and
the optional chaining turned every one of those calls into a no-op: after
an SSO sign-in that created an account, the container log held no line
about it.

The hook now resolves the Logger service from n8n's dependency container
(@n8n/di and @n8n/backend-common, loaded through n8n's own require like
the other modules the hook uses) and writes every message through it. The
requires moved to the top of the ready hook so the first message can use
the logger.

Verified on the pinned image docker.n8n.io/n8nio/n8n:2.44.0, started
with EXTERNAL_HOOK_FILES pointing at the hook and N8N_FORWARD_AUTH_HEADER
set: with this hook the log carries "SSO middleware initializing with
header: X-Forwarded-Email" and "SSO middleware initialized successfully"
before "Editor is now accessible via:"; with the hook of main the same
start reaches the editor without a single SSO line.

Closes #852

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 21:32:10 +02:00
kevinveenbirkenbachandClaude Opus 5.5 ad99ded588 fix(web-app-n8n): stop paging when n8n hands back a page it already sent
Both provisioners follow nextCursor for workflows and step skip for API
keys until a short page arrives. A cursor n8n repeats, or a key listing
that ignores skip, would keep that loop running forever and hang the
deploy instead of failing it.

workflows_named now remembers every cursor it followed and exits with
"FAILED listing workflows: nextCursor repeats" when one comes back.
api_keys_labelled exits when a non-empty page brings no key id it has not
already seen. Normal paging is unchanged.

Four regression tests drive both scripts against a fake API that repeats
its cursor or ignores skip; without the guards exactly those four fail.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 10:06:29 +02:00
kevinveenbirkenbachandClaude Opus 5.5 75f024e71a fix(web-app-n8n): page through API keys and workflows when provisioning
n8n 2 returns ten API keys and a hundred workflows per page by default. A
stale managed key behind the first page survived, and a managed workflow
behind it was created a second time. Both scripts now follow every page,
narrowed by label or name, and keep the exact match in the script.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 22:13:28 +02:00
kevinveenbirkenbachandClaude Opus 5.5 e4078c05fd fix(web-app-n8n): provision the MCP surfaces against the n8n 2 API
n8n 2 pages GET /rest/api-keys as {data: {items, counts, totals, owners}},
ten keys at a time, and matches `label` as a substring, so walking the body
raised "'str' object has no attribute 'get'". It also enforces API key
scopes, so activating the MCP server workflow needs `workflow:activate`.
Both scripts now read every list and create response through one
normalisation that unwraps the page, look the managed key up by label, and
name an unexpected shape by its keys only. The normalisation is taken over
from a392e2dd49 on feature/store-i18n.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 19:41:06 +02:00
kevinveenbirkenbachandClaude Opus 5.5 1d5e7f76e1 fix(web-app-n8n): resolve the SSO hook's modules and roles as n8n 2 does
n8n 2 installs its dependencies as a strict pnpm tree, so the hook's
require('router/lib/layer') and require('bcryptjs') no longer resolve
from the hook file. The n8n.ready hook failed with "Cannot find module
'router/lib/layer'", n8n exited, and the deploy ran into the timeout of
the readiness check. The hook now resolves both through n8n's own
module, the router by way of express.

A user's role is a relation in n8n 2. The hook loads it together with
the user, since the role check refuses an account without it, and
creates a new account with the role as an object, the shape n8n's own
SSO code passes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 13:24:44 +02:00
kevinveenbirkenbachandClaude Opus 5 4ce588fea1 test(lint): forbid a JavaScript default for an unset environment variable
`process.env.X || <value>` is a second source for something the deployment
already renders, and the two drift without a sound. playwright.config.js read
`process.env.PLAYWRIGHT_ACTION_TIMEOUT || 30_000` while nothing ever set that
variable, so every run used the unscaled 30 s and no failure said so.

Normalising an absent variable stays allowed, because it substitutes nothing:
`|| ""`, `|| '[]'`, `|| false`, `|| 0` and falling through to another
`process.env` entry only let the caller test absence itself.

The six sites the rule found:

  n8n and flowise built a Postgres port from `|| '5432'` while env.j2 renders it
  from `lookup('database', application_id, 'port')`. Node throws
  ERR_SOCKET_BAD_PORT on the resulting NaN, so an unfed port now names itself
  instead of connecting somewhere plausible.

  The bluesky broker did the same with BROKER_PORT, which compose.yml.j2 renders
  unconditionally. Its INSECURE_TLS default was `"false"`, which is what `""`
  already evaluates to.

  matomo defaulted MATOMO_TRACKING_SCOPE to "sub"; the only comparison is against
  "root", so `""` takes the identical branch.

  moodle and simpleicons defaulted a name their playwright.env.j2 always renders.
  simpleicons already asserts the value is truthy; moodle now fails on the scope
  it cannot find rather than on a name nobody configured.

Four values the code itself owns keep a default under `// nocheck: env-default`:
PDS_INTERNAL_URL (which falls through to requireEnv, not to a literal),
HANDOFF_COOKIE_NAME, HANDOFF_COOKIE_MAX_AGE and PASSWORD_SALT_HASH_ROUNDS. No
role renders any of them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-12 12:07:57 +02:00
kevinveenbirkenbachandClaude Opus 5 43e09ebf0d fix(n8n): carry a rotated owner password into the account n8n already has
n8n accepts the owner password only at /rest/owner/setup, which answers
400 "Instance owner already setup" once an owner exists. After the CI
rotated owner_password between the passes, the probe got 401, setup got
400, the assert accepted "already setup", and the owner kept the old
password: the Playwright owner login failed in n8n#1 and the AI gateway
login in n8n#0 got 401 and 429 (run 34615516009, jobs 103327034581 and
103327034619).

Only in exactly that state, probe 401/403 plus "already setup", a node
script now runs inside the n8n container with n8n's own modules and
DB_POSTGRESDB_* settings: it compares the stored bcrypt hash with the
deployed password, writes a fresh bcryptjs hash when they differ, and
prints CHANGED or OK. A single confirming login then requires 200, so a
failed write stops the deploy with a clear status. Fresh hosts and deploys
whose password already matches never run the script. The pattern follows
web-app-flowise's identity_password.js.

The module path was measured in n8nio/n8n:1.100.1: bcryptjs and pg
resolve from /usr/local/lib/node_modules/n8n/node_modules, pg's
transitive modules load, and bcryptjs hashes as $2a$ like n8n.

no_log moves from the bootstrap include's apply to the four tasks that
carry the password, so the setup assert and the host and container
resolution stay readable when they fail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 21:30:14 +02:00
kevinveenbirkenbachandClaude Opus 5 3db5820b6e fix(n8n): serve the MCP trigger over Streamable HTTP on 1.100.1 so Open WebUI can connect
Run 34421944328 n8n#1: Open WebUI 0.11.0 ships only a Streamable HTTP
MCP client. n8n 1.95.3's MCP Server Trigger speaks SSE only and answered
the client's POST to /webhook/infinito-mcp/sse with 404, so "every
registered MCP server answers Open WebUI's client" failed.

n8n moves to 1.100.1 and the managed trigger to typeVersion 2, which
serves Streamable HTTP at /mcp/<path>. meta/mcp.yml declares
transport: streamable_http at /mcp/infinito-mcp (the trigger path stays
infinito-mcp) and points source_url at the 1.100.1 tree. Open WebUI's
supported_transports drops sse, which its client never spoke; no
provider declares sse any more, so no pairing is lost.

n8n#0: the guest spec probed ${APP_BASE_URL}${MCP_ENDPOINT_PATH} and
expected a refusal of >= 300 without an event stream. The trigger is
exposure: internal, so the public vhost must answer 404; the spec now
asserts exactly that through mcpEndpointUrl(..., N8N_BASE_URL), and the
bearer guard itself is proven in-cluster by the CLI MCP contract.

Requirement 035 and the MCP audit table follow the version and
transport.

Upgrade path, read from upstream at both tags: the postgres migrations
are identical (45/45) and the only new common migration is
AddProjectDescriptionColumn, an additive nullable column; BREAKING-
CHANGES.md lists only 1.98.0 in range (route metric format, Form node
tag attributes, Node >= 20), none of which this deployment uses. A fresh
1.100.1 install ran all 89 migrations and passed the MCP bridge, server,
non-member and guest specs locally.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 12:20:46 +02:00
kevinveenbirkenbachandClaude Opus 5 57f90fb731 test(lint): an upper bound must scale even when the literal hides behind a name
The rule read the character after `timeout:` and objected to a digit, so a
duration bypassed the helper the moment it was given a name. That was already
reachable before, and it became easy to reach now that a poll interval is
allowed to stay unscaled: the two look identical at the call site and only one
of them is correct.

They are opposites, and the rule now says so. An upper bound must scale, because
the answer over a Tor circuit arrives later and a fixed cap expires before it
does. A poll interval must not, because scaling one makes a slow target notice
its element later rather than sooner, which is the reverse of the tolerance the
scaling exists to buy. So `timeout:` rejects a CONSTANT_CASE name as well as a
digit, while `waitForTimeout` and `setTimeout` keep matching on a leading digit
only and accept a named interval.

Across the 1307 `timeout:` sites in scope the sharpened rule finds exactly one,
and it is a true positive that wanted saying out loud. The n8n LiteLLM spec
passes a completion timeout into an `n8n-nodes-base.httpRequest` node: not a
Playwright wait at all, but a parameter for a request n8n makes to LiteLLM
inside the deployment, which never crosses a circuit. Leaving it unscaled is
right and was invisible; it now carries the sanctioned marker and its reason.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 00:32:36 +02:00
kevinveenbirkenbach 121fd3f485 test(playwright): register three specs the collector never reached
playwright.config.js collects `**/*.@(spec|test).js`, so a module named
test-<something>.js runs only when its role's playwright.spec.js requires it.
Three roles wrote theirs as standalone specs with a top-level test() instead:
the MCP guest guards of n8n and wordpress, and the OIDC login of hermes. All
three have been dead files since they were added, so every MCP surface was
believed to be probed by a hostile guest and hermes' oauth2-proxy round trip
to be covered while none of those assertions had ever executed.

Each moves to the exports.register shape its siblings use and is appended to
its aggregator. Nothing else changes: `git diff -w` on this commit is the
test.use line becoming the register wrapper plus one require per aggregator,
and no assertion is touched. The module-level
test.use({ignoreHTTPSErrors: true}) goes away because all three aggregators
already set it for the tests they register, and a test.use inside register()
would not apply anyway. registerMcpDisabledState moves inside register() so
the switched-off case comes with it, which the addon specs do not cover since
they skip when the addon is off.

Registering them also subjects them to test.beforeEach(shared.beforeEach).
That is safe here: wordpress sets a viewport and asserts env presence, n8n
does the same and clears cookies, hermes is a no-op. None authenticates, so
the guest guards stay guests.

wordpress is the only one of the three with a green job to lose, and its
status assertion is already proven there: addons/mcp-adapter.spec.js posts to
the same endpoint with maxRedirects 0 and asserts >= 400, it matches the
collector glob, and WordPress#1 passes. The permission callback runs at the
transport before the JSON-RPC method is parsed, so initialize is refused
exactly like tools/list.

A fourth dead module, openwebui's test-ollama-chat.js, gates on a service no
role renders and follows separately.
2026-09-08 17:30:16 +02:00
kevinveenbirkenbachandClaude Opus 5 cb7773fe5c test(n8n): assert the gate that is deployed, not the one that was
4e46858202 closed the auth gate: meta/services.yml admits only the administrator and mcp groups. biber is provisioned with no roles, so the oauth2-proxy rejects its OIDC callback and serves a 403. The spec still asserted the opposite, and its comment still described the era when the hook auto-provisioned any Keycloak identity. Both n8n jobs of run 33445831564 burned three Playwright attempts against that contradiction.

The journey is unchanged: same env assertions, same signInViaN8nOidc pass through Keycloak. Only the expectation is inverted, and the load-bearing half is the absence of n8n's surface, carrying all five markers the old positive assertion used. The wording check is secondary and broad, so a reworded proxy page does not decide the verdict.

n8nLogout leaves the spec without losing coverage; it is exercised twice in test-login-administrator.js and once in test-litellm-gateway.js, and a persona that never signs in has nothing to sign out of.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 23:00:04 +02:00
kevinveenbirkenbachandClaude Opus 5 791eee7c9f refactor(mcp)!: prove the endpoint contract with a CLI test, not a deploy probe
The contract check ran as an Ansible task in the middle of every provider deploy, and the two client checks ran the same way. A test that is part of the deploy cannot be run without deploying, cannot be re-run on its own, and reports its verdict as a deploy failure. Twenty-seven task files lose 491 lines to this.

It is now what it always was. Each provider ships files/test/test.sh, templates/test.env.j2 and meta/tests.yml, and test-e2e-cli stages one shared harness beside it: run.sh drives contract.sh in a throwaway container on the provider's own network, then isolation.sh for adapter-fronted providers. The read probe each provider offers moved from the calling task into meta/mcp.yml next to the allowlist it must satisfy, and the schema now rejects a read probe naming a tool outside that allowlist.

The bearer the test presents is the one a client resolves through mcp.credential, via the new mcp_client_bearer lookup over resolve_credential, which moved to utils.roles.applications.mcp so it is the same function mcp_servers renders every client connection with. The probe presented whatever the calling role passed, which for the fifteen adapter-fronted providers is a different secret than any client holds.

Sidecar isolation is read from the engine rather than measured inside the container, so it no longer needs a delegation to the node holding the task: attachment is compared by network name, which drops the gwbridge special case, and a swarm worker says it cannot measure instead of skipping silently.

Retrying is bounded by wall clock against MCP_DEADLINE_SECONDS, which scales through the same lookup('timeout') as the harness budget it has to stay under. A flag that is neither true nor false aborts rather than reading as off.

mcp-provider-probe becomes mcp-provider-cli-test: an enabled provider must ship the test rather than include the task. mcp-probe-bearer becomes mcp-adapter-bearer, tying ADAPTER_BEARER to the declared credential key now that no probe include carries it. A new integration test renders the shared env template for every provider through Ansible's own templar, so a later change to it breaks visibly instead of poisoning twenty deploys quietly.

Hermes and OpenClaw keep their client-side proof as CLI tests of their own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 20:46:26 +02:00
kevinveenbirkenbachandClaude Opus 5 4e46858202 feat(n8n): provision the MCP server and client, and close the auth gate
The oauth2 gate declared no allowed_groups at all while the role carried an mcp RBAC role nothing consulted, so it admitted anyone the proxy authenticated. It now admits the role's own administrator and mcp groups, and a lint derives the client set from meta/mcp.yml rather than a list, which is how the gap surfaced.

Two idempotence defects went with it. The run minted a public-API key, found it again next time and counted that as a change, so the task never converged; and it left that workflow-scoped key standing after every deploy although nothing read it. It is now minted per run and revoked before the run ends. The server workflow also connected nothing, and the trigger serves exactly the tools that reach it over an ai_tool connection, so it served none; it now carries one checked-in read tool with an explicit input schema.

The client side did not exist. It renders an agent workflow with one MCP Client Tool per discovered provider, each pinned to that provider's allowlist, because the node defaults to including every tool an upstream happens to serve. Credentials go through the internal API since the public one of this tag can only create and delete them, never converge an existing one. Every node type, typeVersion and connection literal was read from the pinned source.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:46:57 +02:00
kevinveenbirkenbachandClaude Opus 5 7d8e7e87ff test(mcp): prove a switched-off surface is actually gone
Every role-local MCP spec gates itself on the service flag and skips when the surface is off, so nothing ever checked that switching it off removes it. A surface still serving after the flag went false was indistinguishable from one that was never deployed: both read as a green skip. The gating helper had no inverse at all, so there was no way to assert what a disabled service must not do.

A shared assertion now probes the endpoint with both a GET and a POST while the flag is false and fails on any answer that shows something listening. It counts 401 and 403 as listening, which is the point: a refused credential still means the surface is there. The check is transport-agnostic because the roles differ in their enabled-state probes, over SSE, Streamable HTTP or plain JSON-RPC, while an endpoint that is gone answers none of them. Each role passes its endpoint URL as a thunk, since some resolve it only once a test is running.

The coverage lint requires the registration, and derives the role set from meta/mcp.yml rather than from a filename convention, which is how the tenth role surfaced: its spec is not named like the other nine.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:40:15 +02:00
kevinveenbirkenbachandClaude Opus 5 27dcd975e5 test(e2e): scale the spec timeouts for onion transport
Twenty-five specs called the Playwright APIs with default or hardcoded
timeouts. Routed over Tor those budgets are too small, so a spec fails on
transport latency rather than on the behaviour it asserts.

Route the waits through resolveTimeout and reach pages via gotoOnion, the
helpers the tor branch introduced for exactly this. The remaining changes in
the diff are import wrapping around those calls.

The repository lint enforces this: a raw numeric timeout in a role spec is
rejected with a pointer to resolveTimeout, which is how a hardcoded 500ms
wait added during this session was caught.

Merge work rather than authored here; not deployed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:52:19 +02:00
kevinveenbirkenbachandClaude Opus 5 dfe7a7fdd4 refactor(roles): sort role scripts into per-language folders
The merged branch moved every role-local script out of files/ into a folder
named for its language (files/python/, files/php/, files/shell/), but the
moves lost their staging and never landed. HEAD still carried the old paths
while the worktree had the new ones.

Move the 24 remaining scripts and update the six places that resolve them:
the mcp-adapter instance/isolation/probe tasks, the litellm key task, the
adapter README and the endpoint-path lint. Every changed line in those
files is a path.

Also drop roles/sys-svc-compose/files/swarm/stack_ready.sh, a stale
duplicate of the copy already tracked under files/shell/swarm/.

The added comments are the branch's own `nocheck: mirrored-unit-test`
directives, which state why a script whose measurement only exists inside
its container cannot have a mirrored unit test.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:00:59 +02:00
kevinveenbirkenbachandClaude Opus 5 7162b7bf4c Merge fork/feature/svc-net-tor into feature/ai-agent-employees
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.

The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.

Resolutions that were not unions:

Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.

plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.

compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.

tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.

Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.

01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.

Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.

roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.

config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.

READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.

Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 16:32:30 +02:00
kevinveenbirkenbachandClaude Opus 5 8e8dbbd08c refactor(layout): fold the prefix-shared AI files into their own folders
test_prefix_groups_belong_in_subfolders flagged three groups added by this change
set: a shared delimiter-bounded prefix is a folder in disguise, so the group moves
into a folder named after the prefix and the prefix leaves the file names.

  sql/select/ai-{bot-user,service}.sql.j2  -> sql/select/ai/{bot-user,service}.sql.j2
  files/provision_{ai,mcp}.py              -> files/provision/{ai,mcp}.py
  files/ai_engine_{configure,endpoint}.php -> files/ai_engine/{configure,endpoint}.php

All six are recorded as renames, so history follows them. Every reference moves
with them: four templated query paths in mattermost's ai_gateway.yml, the two
lookup('file', ...) calls in n8n, the two in wordpress, and SCRIPT_PATH plus the
module docstring in the n8n unit test. lookup('file', ...) resolves relative to
the role's files/ directory, so the new argument is provision/mcp.py rather than
a repo-relative path.

Worth recording why this survived every gate: make quality was green on each run
this session, including immediately before the commits that introduced these
files. The lint enumerates through git ls-files, and the files were untracked, so
it could not see them. Committing them is what made them visible. A green gate
taken before new files are tracked says nothing about the gate after.

Verified: repository lints go from 88 passed with 1 failure to 89 passed, the
unit suite passes at 4448, and a repo-wide grep for every old path returns
nothing.
Not verified: no deploy has run. The lookup('file', ...) arguments and the
templated SQL paths resolve at play time, so a wrong path would surface as a task
failure rather than a test failure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 15:19:26 +02:00
kevinveenbirkenbachandClaude Opus 5 8c3d6fcde1 fix(n8n): give the managed MCP trigger the bearer credential it enforces
The mcpTrigger node forces authentication to bearerAuth, because the upstream
`none` option would publish an unauthenticated SSE endpoint on the role's own
vhost. Setting that parameter is only half the contract: the node must also
reference an httpBearerAuth credential holding the token. Without one the
trigger answers every request to <path>/sse with

    500 No authentication data defined on node!

before the SSE transport is ever opened, while the workflow itself looks
correctly configured and reports active.

The provisioning chain gains a credential leg between the API key and the
workflow: POST /rest/login for the owner cookie, /rest/api-keys for the
deterministic managed key, /rest/credentials for the bearer, then the public
/api/v1 routes for the workflow. The credential is upserted under the
deterministic name in N8N_MCP_CREDENTIAL, which is why that constant landed with
the AI bundle: the variable has to exist before this references it.

The workflow body is now pushed on every run, including while the workflow is
active. Previously a node that had drifted from the definition stayed broken
behind its own active flag, because the reconciler saw an existing active
workflow and left it alone. Pushing unconditionally lets a wrong node heal.

Verified: the /rest credential route with the owner cookie is exercised by the
sibling provision_ai.py, which ran in a variant-2 round, so the mechanism this
depends on works even though this call has not been observed.
Not verified: no MCP request against n8n has been seen answering 200 since the
change. Two consequences are deliberate but real -- an operator's intentional
edit to the managed workflow is now overwritten on every deploy, not just a
drifted one, and because n8n never returns encrypted credential data, a token
that drifted from N8N_MCP_TOKEN cannot be detected, only overwritten.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 14:16:56 +02:00
kevinveenbirkenbachandClaude Opus 5 43ca7f73ae feat(n8n): provision the managed openAiApi credential against the gateway
AI Agent nodes read their endpoint from a stored credential, not from the
environment, so the gateway base URL and this role's virtual key have to land in
n8n's credential store. files/provision_ai.py logs in as the owner and upserts an
openAiApi credential under the deterministic name infinito:litellm.

The write goes through the internal /rest routes because that is the only place
the pinned 1.95.3 exposes credential creation: the public /api/v1 key this role
already mints for MCP carries workflow scopes and cannot create a credential.
That is why this is a second script rather than another call inside
provision_mcp.py.

n8n encrypts credential data at rest and never returns it, so there is nothing
to compare against and the managed credential is overwritten rather than
reconciled. The script prints CHANGED or OK so the task can report change state
without pretending to have diffed a value it cannot read.

env.j2 sets only N8N_AI_ENABLED. An earlier revision of this work templated
N8N_INSTANCE_AI_MODEL_URL, which does not exist in 1.95.3 -- it was invented
rather than measured, and has been removed along with the requirement table row
that repeated it. The one real AI environment variable in that release,
N8N_AI_ASSISTANT_BASE_URL, expects n8n's own assistant service rather than an
OpenAI-compatible endpoint, so the credential store is the only usable surface.

vars/main.yml also carries N8N_MCP_CREDENTIAL_NAME. It belongs to the MCP work
rather than to this change, but a file cannot be split across commits and this
order is the safe one: the MCP tasks land later and find the constant already
present, where the reverse would reference a variable that does not exist yet.

Verified: the spec runs rather than skipping -- it appears in the variant-2
JUnit as "administrator: the managed openAiApi credential answers a prompt
through the in-cluster gateway".
Not verified: I confirmed it executed, not its verdict. The owner-cookie login
is also unexercised on an sso-only variant, and because the credential is
rewritten every run, an operator's manual edit is replaced without notice.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 13:44:34 +02:00
kevinveenbirkenbachandClaude Opus 5 575a5a87ac refactor(layout): sort shipped scripts by language and split the unit suite
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).

This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.

Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:

  * 7 `src:` values built from a role variable holding a bare filename
    (svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
    The variable doubles as the destination name inside the container, so the
    language segment belongs at the call site, not in the variable.
  * `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
    through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
    18 roles ship exactly that and have no template fallback: the play would
    have died on each.
  * `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
    interface now reads `files/javascript/javascript.js`.
  * `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
    path through `path_absolute(... 'files', ...)`.
  * 4 `ansible.builtin.script:` invocations naming a bare command
    (pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
  * `dev-python/files/shell/install.sh` pointed at a sibling role's script.

Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 01:31:09 +02:00
kevinveenbirkenbachandClaude Opus 5 f842402632 fix(n8n): activate the managed MCP trigger when mcp is enabled
The workflow was created deactivated and never activated, so the declared
endpoint answered 404 and every client configured against it failed its
reachability check. The intent was that an operator opts in before an MCP
server answers, but mcp.enabled already is that opt-in: a second one hidden
in the provisioning script made the declared state unenforceable, and the
mismatch surfaced three layers away as a failing consumer.

Activate after create and after update. Keep the protection that matters:
an already-active workflow is left untouched rather than rewritten under
callers bound to its endpoint, and that case now returns instead of ending
the deploy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 15:34:16 +02:00
kevinveenbirkenbach 3b22cd4073 feat(web-app-n8n): provision the bidirectional MCP surface
n8n is the only role that speaks MCP in both directions, consuming other
servers and serving one of its own.

The provisioner carries the auth cookie by hand rather than through a
CookieJar. n8n sets it with Secure, which is right for the browser-facing
origin, so a jar drops it over the plain-http internal address and every call
after the login answers 401. Setting N8N_SECURE_COOKIE=false would have fixed
the provisioning at the cost of the cookie the browser actually relies on, so
the session is carried explicitly instead.

The mcp role is declared in the role's rbac metadata, without which the group
path lookup hard-fails, and a Playwright spec proves the surface serves a guest
nothing. That check needs no credential, which is why it belongs in Playwright
while the contract probe with the real bearer stays at deploy time and out of
the trace.
2026-08-06 21:17:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 07c6ad8e7c test(e2e): scale every nav click + timeout literal for onion targets
Over Tor the redirect chain behind a login/logout/OIDC click routinely exceeds Playwright's fixed 30s action timeout, so the click throws before the flow's scaled waits run. Sweep all role Playwright suites and the shared persona helpers: give every navigation/auth-transition click an onion-scaled timeout (.click({ timeout: resolveTimeout(30_000) })) and route the remaining raw millisecond literals (timeout options, waitForTimeout, sleep-promise setTimeout, positional wait args) through resolveTimeout. Local UI clicks (menus, toggles, modal dismissals, DOM clicks inside evaluate) stay untouched.

Two new lints keep it that way: test_no_raw_timeout_literals forbids raw numeric timeout literals in Playwright specs (opt-out: nocheck: raw-timeout), test_persona_clicks_scaled requires every click in the shared persona auth flows to carry a resolveTimeout timeout (opt-out: nocheck: unscaled-click).

Verified: node --check on all 133 changed JS files, all 13 playwright lints green, make autoformat clean, make test green (2745+196+163+4 tests OK).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 11:35:53 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4c91df5de7 test(e2e): forbid raw page.goto, migrate all specs to Tor-resilient gotoOnion
New lint test_no_raw_goto.py forbids raw `page.goto(...)` in role playwright specs + persona flows (the gotoOnion wrapper in personas/utils is exempt) so navigation retries transient Tor-transport errors over onion. Migrates every raw call across ~75 roles (343 sites) to `gotoOnion(page, ...)` and teaches gotoOnion to detect relative onion URLs (`isOnionCanonical`). gotoOnion is a behaviour-identical drop-in on clearnet (single attempt, same return). Verified: no-raw-goto lint green, eslint 0 errors, make test green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:59:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8988fdf158 test(web-app-n8n): route playwright timeouts through resolveTimeout
n8n's playwright specs arrived un-migrated via a branch update; wrap their raw numeric timeouts in resolveTimeout so they satisfy the no-raw-timeouts lint and scale over onion. Clearnet-identical (resolveTimeout(N) === N).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 13:00:16 +02:00
Prageeth Panicker 2b831f56d1 fix(web-app-n8n): address Kevin's review comments
- tasks/02_bootstrap.yml: set retries=4 delay=30 on owner probe to
  stay within n8n's 5-attempt/5-minute rate limit window
- files/playwright: remove test-oidc-login.js - redundant with
  test-login-administrator.js which already covers the full OIDC flow
- rebase onto latest main
2026-07-07 12:53:39 +00:00
Prageeth Panicker 167bbe9e24 fix(web-app-n8n): fix hooks.js for n8n 1.95.3 API
Remove relations: ['role'] from UserRepo.findOne() - in 1.95.3
user.role is a plain string column, not a relation. The relations
array caused EntityPropertyNotFoundError which produced 500 on
every SSO login attempt. Also removed dead RoleRepo/dbCollections.Role
references dropped in 1.95.3's DropRoleTable migration.

Verified: biber@infinito.example now auto-provisioned as global:member
after Keycloak SSO. All 5 Playwright tests pass locally.
2026-07-07 12:51:06 +00:00
Prageeth Panicker 31f3434d66 feat(web-app-n8n): implement community SSO via trusted-header hook
- Add files/hooks.js (adapted from PavelSozonov/n8n-community-sso)
  Auto-provisions users and issues n8n session from Remote-Email header
- Add templates/proxy.conf.j2 - openresty vhost forwarding Remote-Email
  trusted header in SSO-gated branch
- Update env.j2: EXTERNAL_HOOK_FILES + N8N_FORWARD_AUTH_HEADER when SSO enabled
- Update compose.yml.j2: bind-mount hooks.js read-only into container
- Update tasks/01_core.yml: stage hooks.js before container starts
- Fix Playwright: V1 SSO tests now assert direct workflow editor landing
- Fix Playwright: administrator test works in both V1 (SSO) and V2 (no SSO)
- Update README: document auto-provisioning SSO behavior
2026-07-07 12:51:06 +00:00
Prageeth Panicker 58ce233f97 fix(web-app-n8n): fix Playwright tests for CE edge-SSO model
- Remove LDAP variant (V3) - unsupported on n8n Community Edition
- Fix test-login-administrator: Keycloak SSO + n8n native login form
- Fix test-login-biber: verify oauth2-proxy gate only (no n8n account)
- Add performN8nLoginForm() helper to _shared.js
- Add ADMIN_EMAIL and N8N_OWNER_PASSWORD to playwright.env.j2
- Update README to clarify edge-SSO vs native n8n session
2026-07-07 12:51:06 +00:00
Prageeth Panicker f2388849e9 fix(web-app-n8n): fix LDAP sign-in button selector and related fixes
- Fix button selector in _shared.js: button[type=submit] → getByRole('button', {name: /sign in/i})
- Update test-login-via-ldap.js accordingly
- Fix 03_ldap.yml LDAP configuration task
- Update playwright.env.j2
2026-07-07 12:51:06 +00:00
Prageeth Panicker faa8b6b3ee fix(ci): resolve 4 CI failures across n8n, pihole, and mastodon roles
- Add bond:1 to all 9 enabled+shared services in web-app-n8n/meta/services.yml
- Fix web-app-n8n/meta/variants.yml V3 postgres shared:false → shared:true (only V2 is standalone)
- Remove dead PIHOLE_SSO_PROXY_CA_CONTAINER_PATH var from web-app-pihole/vars/main.yml
- Add nocheck:url marker for dead link in web-app-mastodon/README.md
- Fix LDAP sign-in button selector in _shared.js (button[type=submit] → getByRole)
2026-07-07 12:51:06 +00:00
Prageeth Panicker 8332c7edf0 feat(web-app-n8n): add n8n workflow automation role
- Community edition with edge oauth2-proxy SSO (Decision #1b)
- PostgreSQL backend via svc-db-postgres
- Three variants: V1 sso+ldap, V2 no-auth, V3 ldap-only
- Bootstrap owner account via /api/v1/owner/setup
- LDAP config via direct PostgreSQL upsert (CE API limitation)
- Playwright: baseline, CSP, OIDC, administrator and biber personas
- All 5 active tests pass, 2 skipped correctly for this variant
2026-07-07 12:51:06 +00:00