Moodle 5.3 marks `external_generate_token()` as a final deprecation
(`#[deprecated('\core_external\util::generate_token', since: '4.4',
final: true)]` in public/lib/externallib.php), and
`core\deprecation::emit_deprecation_notice()` throws a coding_exception
for every final deprecation, whatever the debug level. The MCP
provisioning script called it whenever no permanent token existed yet,
so every fresh install of the variant that enables MCP failed at
"Enable web services and mint the MCP bearer" (main run 39494396920,
Moodle#1: `Error code: codingerror` from deprecation.php:273 via
externallib.php:30).
The variant had not run in CI since the Moodle 5.3 bump, which is why
this surfaced only now.
Mint the token with `\core_external\util::generate_token()`, which
takes the service record instead of its id, and create the service
account with `\core\user::create_user()` in place of
`user_create_user()`, deprecated since 5.3 (MDL-82650). Neither needs
externallib.php or user/lib.php any more.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
config.php set `$CFG->debug = 32767` whenever MODE_DEBUG was true, as
the development and CI inventories set it. Moodle 5.3's
`core_component::is_developer()` treats any debug bit defined in
config.php as developer mode, so `init()` skipped
`$CFG->cachedir/core_component.php` and ran `fill_all_caches()` on
every PHP request: a walk over every plugin directory and every
`classes/` tree. The autoload in setup.php triggers it before the
ABORT_AFTER_CONFIG return, so even font.php and requirejs.php paid it.
On local disk the walk is cheap. In swarm the code tree is an NFS4
mount, and the walk turns into GETATTR round trips: main run
39494396920 served `/theme/font.php` in no less than 1.6 s and
`/lib/requirejs.php` in no less than 2.0 s, the five FPM workers per
replica queued, and the CSP check timed out loading the academy page
after 20 s on both Moodle swarm jobs.
Write `debug` and `debugdisplay` into the site config with
`admin/cli/cfg.php` instead, on every deploy and as integers (32767/1
in debug deployments, 0/0 otherwise). `is_developer()` reads only
values defined in config.php, so the component cache is honoured,
while `setup.php` still applies the stored debug level once the
database is up. The assignments leave config.php entirely, because
cfg.php refuses to change a name config.php defines, and
MOODLE_DEBUG leaves env.j2 with them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
provision_ai_gateway.php and provision_mcp.php shared a prefix inside one
directory, which test_prefix_grouped_files rejects. Both move into
files/php/provision/ as ai_gateway.php and mcp.php, the layout
web-app-n8n already uses for its Python provisioning scripts.
The two lookups that feed the scripts to the container follow, and the
unit test moves to its mirrored path as provision/AiGatewayTest.php.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Moodle 5 keeps AI provider settings in provider instances, not in the
plugin configuration. The deploy kept writing aiprovider_openai/* through
cfg.php and read the same value back, so its own check stayed green while
no provider existed. The gateway spec then found no settings page and
web-app-moodle#2 failed in run 37646050826.
The deploy now pipes files/php/provision_ai_gateway.php into the
container. It finds the instance by class and name or creates it through
core_ai\manager, points the three text actions at the chat endpoint,
switches image generation off, stores the virtual key and prints what
Moodle stored. The task asserts on that output. An instance an
administrator added is left alone.
The spec finds the instance by MOODLE_AI_PROVIDER_NAME on the providers
page and reads the endpoint from its generate_text action page.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The entrypoint removed the old tree before it copied the new one. A
copy that failed left the volume without a working Moodle, and every
restart removed again instead of recovering. It now copies into a
staging directory inside the volume and swaps the trees once the copy
is complete, so a failed copy leaves the old tree in place.
An image that ships an older release than the volume records no longer
replaces it and exits with a message instead. Two instances on
different releases would otherwise overwrite each other's tree.
upgrade.php reports "No upgrade needed" on stderr, so the task counted
every run on a current site as a change. It reads stderr now.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The pin moved from 4.5.11 to 5.3.0, and the role fitted neither a new
install nor an existing one.
Moodle 5 serves from public/ and routes every other path through
r.php. nginx now uses public/ as its root, hands unknown paths to the
router and passes PATH_INFO for scripts with slash arguments, and
config.php declares the router as configured. The paths the role checks
and writes (version.php, auth, webservice, the OIDC plugin) moved below
public/, the URL of the OIDC plugin stays auth/oidc. The source archive
no longer contains vendor/, so the image installs the Composer
dependencies it declares. The OIDC plugin is pinned to v5.2.1, the
newest release.
An existing site kept its old code: the entrypoint filled the code
volume once and never again. It now records the release it copied and
replaces the tree when the image carries another one, and a new task
runs admin/cli/upgrade.php. Plugins that were installed through the web
interface are not carried over, the README says how to ship them.
A site installed with 5.3 forces login by default, which 4.5 did not,
so its start page answers 303 to the login page and the webserver
health check refused it. The role declares 200 and 303 as the codes
that check accepts: 200 is what an upgraded site keeps answering.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Memcached, Joomla, Moodle, Odoo, Shopware, Snipe-IT and SuiteCRM set
their cache size, PHP memory_limit, OPcache, worker limits and search
heap as literals next to a mem_limit they never read. They now ask
lookup('service_memory_mb') for a share of that limit. The shares are
chosen so every rendered value stays what it was.
Where the parameter lives in a file that cannot hold a template, the
value is handed in: Shopware's init script reads INIT_PHP_MEMORY_LIMIT
and refuses to start without it, SuiteCRM's image takes it as a build
argument. Shopware's worker gets --memory-limit as a command appended
to its entrypoint, and Moodle's OPcache size moves from the static ini
into the rendered one.
The performance.php.memory_limit keys of Joomla and Moodle are gone
from services.yml; nothing else read them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`process.env.X || <value>` is a second source for something the deployment
already renders, and the two drift without a sound. playwright.config.js read
`process.env.PLAYWRIGHT_ACTION_TIMEOUT || 30_000` while nothing ever set that
variable, so every run used the unscaled 30 s and no failure said so.
Normalising an absent variable stays allowed, because it substitutes nothing:
`|| ""`, `|| '[]'`, `|| false`, `|| 0` and falling through to another
`process.env` entry only let the caller test absence itself.
The six sites the rule found:
n8n and flowise built a Postgres port from `|| '5432'` while env.j2 renders it
from `lookup('database', application_id, 'port')`. Node throws
ERR_SOCKET_BAD_PORT on the resulting NaN, so an unfed port now names itself
instead of connecting somewhere plausible.
The bluesky broker did the same with BROKER_PORT, which compose.yml.j2 renders
unconditionally. Its INSECURE_TLS default was `"false"`, which is what `""`
already evaluates to.
matomo defaulted MATOMO_TRACKING_SCOPE to "sub"; the only comparison is against
"root", so `""` takes the identical branch.
moodle and simpleicons defaulted a name their playwright.env.j2 always renders.
simpleicons already asserts the value is truthy; moodle now fails on the scope
it cannot find rather than on a name nobody configured.
Four values the code itself owns keep a default under `// nocheck: env-default`:
PDS_INTERNAL_URL (which falls through to requireEnv, not to a literal),
HANDOFF_COOKIE_NAME, HANDOFF_COOKIE_MAX_AGE and PASSWORD_SALT_HASH_ROUNDS. No
role renders any of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
install_database.php was the only place that ever wrote the Moodle site
administrator password, and its failed_when tolerates "Database tables already
present". That is not laxity: upstream aborts in install_database.php before
install_cli_database() writes the password once tables exist, so the role
swallowed exactly the path in which the write is unreachable. After the
credential rotation gate Moodle therefore kept the install-time secret while
templates/playwright.env.j2 rendered the rotated one, and the login answered
"Invalid login, please try again". The same job proves it: variant-2/sync
reports failures="0" with the spec passing in 5.6s, variant-2/async
failures="1", and only the rotation gate lies between them.
The deeper problem was the credential itself. secrets.credentials.user_password
described itself as "Initial password for the Moodle admin user" - a role-local
secret for an account that is not role-local. Moodle hosts the platform
administrator, so read that user instead, the way web-app-baserow,
web-app-gitlab, web-app-penpot, web-app-matrix, web-app-bluesky and
web-app-pihole already do. lookup(users, administrator).password is stable
because roles/user-administrator/meta/users.yml pins it to
ansible_become_password; only a user with no declared password re-renders
{{ 42 | strong_password }} on every evaluation.
vars/main.yml now holds username, password and email as one constant each, so
install, realign and the Playwright env cannot drift apart again, and the
now-unread credential leaves meta/secrets.yml.
Realigning uses admin/cli/reset_password.php. Verified against the pinned
release v4.5.11: --username and --password are non-interactive (lines 35-37,
prompts only when empty at 76 and 88), --ignore-password-policy is required
because the platform password is not chosen to satisfy Moodle policy (line 96),
and the "when" gate is required because line 84 resolves only auth=manual rows
while tasks/04_oidc.yml rewrites every row to auth=oidc.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Commit 1cd9578b8 moved the OIDC plugin version out of a hardcoded string
into meta/addons/auth_oidc.yml so the update pipeline tracks it. That
declaration is service-coupled (enabled reads services.sso.enabled), which
obliges the role to ship an integration hook and a per-addon Playwright
spec. Both were missing, so test_addons_integration_tasks and
test_addons_playwright_spec went red.
The hook has no work to do: files/build/install/auth-oidc.sh bakes the
plugin into the image and tasks/04_oidc.yml configures it, included by
tasks/01_manager_ops.yml on the same services.sso.enabled flag. It records
where the wiring actually lives so the coupling stays greppable.
The spec does not follow the pretix shape of hunting a login-page link,
because tasks/04_oidc.yml sets alternateloginurl and /login/index.php
bounces straight to the plugin. It asserts the bounce instead: the login
page must land on the Keycloak openid-connect authorization endpoint
carrying a client_id and a redirect_uri pointing back at auth/oidc.
Rendering the addon flags into playwright.env.j2 is what makes that spec
real. Without lookup('addon_env_flags'), AUTH_OIDC_ADDON_ENABLED is
undefined, skipUnlessAddonEnabled skips forever and the spec is a check
that can never fail.
download.moodle.org returns 403 to GitHub-hosted runners while serving the identical artefact elsewhere: 12 failures across three runners over twelve hours, each 40-66ms after TLS, with curl's four retries and Ansible's three exhausted. The GitHub tag archive is content-equivalent - both tarballs extracted and diffed over 39312 entries, the only delta being githash.php, which nothing in the 4.5.11 tree includes - and needs --strip-components=1 because it roots at moodle-4.5.11/ rather than moodle/.
auth-oidc.sh was the next landmine, two Dockerfile lines behind the fixed one: an anonymous api.github.com tag query, 60 requests per hour per runner IP, no retry, -s swallowing the rate-limit body under set -e, and a floating plugin version as the build input. It now reads a new meta/addons/auth_oidc.yml, which declares bridges: [sso] and update.monitored so the version is watched by the update pipeline instead of frozen.
MOODLE_BRANCH and MOODLE_TARBALL_FILENAME die with the old URL; the latter was already dead, demanded only by its own :?required guard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The contract check ran as an Ansible task in the middle of every provider deploy, and the two client checks ran the same way. A test that is part of the deploy cannot be run without deploying, cannot be re-run on its own, and reports its verdict as a deploy failure. Twenty-seven task files lose 491 lines to this.
It is now what it always was. Each provider ships files/test/test.sh, templates/test.env.j2 and meta/tests.yml, and test-e2e-cli stages one shared harness beside it: run.sh drives contract.sh in a throwaway container on the provider's own network, then isolation.sh for adapter-fronted providers. The read probe each provider offers moved from the calling task into meta/mcp.yml next to the allowlist it must satisfy, and the schema now rejects a read probe naming a tool outside that allowlist.
The bearer the test presents is the one a client resolves through mcp.credential, via the new mcp_client_bearer lookup over resolve_credential, which moved to utils.roles.applications.mcp so it is the same function mcp_servers renders every client connection with. The probe presented whatever the calling role passed, which for the fifteen adapter-fronted providers is a different secret than any client holds.
Sidecar isolation is read from the engine rather than measured inside the container, so it no longer needs a delegation to the node holding the task: attachment is compared by network name, which drops the gwbridge special case, and a swarm worker says it cannot measure instead of skipping silently.
Retrying is bounded by wall clock against MCP_DEADLINE_SECONDS, which scales through the same lookup('timeout') as the harness budget it has to stay under. A flag that is neither true nor false aborts rather than reading as off.
mcp-provider-probe becomes mcp-provider-cli-test: an enabled provider must ship the test rather than include the task. mcp-probe-bearer becomes mcp-adapter-bearer, tying ADAPTER_BEARER to the declared credential key now that no probe include carries it. A new integration test renders the shared env template for every provider through Ansible's own templar, so a later change to it breaks visibly instead of poisoning twenty deploys quietly.
Hermes and OpenClaw keep their client-side proof as CLI tests of their own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every role-local MCP spec gates itself on the service flag and skips when the surface is off, so nothing ever checked that switching it off removes it. A surface still serving after the flag went false was indistinguishable from one that was never deployed: both read as a green skip. The gating helper had no inverse at all, so there was no way to assert what a disabled service must not do.
A shared assertion now probes the endpoint with both a GET and a POST while the flag is false and fails on any answer that shows something listening. It counts 401 and 403 as listening, which is the point: a refused credential still means the surface is there. The check is transport-agnostic because the roles differ in their enabled-state probes, over SSE, Streamable HTTP or plain JSON-RPC, while an endpoint that is gone answers none of them. Each role passes its endpoint URL as a thunk, since some resolve it only once a test is running.
The coverage lint requires the registration, and derives the role set from meta/mcp.yml rather than from a filename convention, which is how the tenth role surfaced: its spec is not named like the other nine.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four unmarked `|| true` remained in this file after c505ce7466 grandfathered
the corpus; they entered later and left the file in violation of shell-or-true.
Handling them beats marking them.
cp -f writing config.php is the one that mattered: swallowing its failure let
Moodle boot against a missing or stale configuration instead of stopping. It
now fails.
The chown and chmod calls can only fail for lack of privilege, so the root
check states that condition instead of discarding every error. The image
declares no USER over php, and the compose template sets no user:, so this
runs as root today; the check guards a future USER rather than an observed
failure.
moodle_refresh_config inverts its guard to an early return so the copy sits one
level shallower.
The five grandfathered `|| true` keep their nocheck and their TODO, including
the chown three lines above the one hardened here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twenty-five specs called the Playwright APIs with default or hardcoded
timeouts. Routed over Tor those budgets are too small, so a spec fails on
transport latency rather than on the behaviour it asserts.
Route the waits through resolveTimeout and reach pages via gotoOnion, the
helpers the tor branch introduced for exactly this. The remaining changes in
the diff are import wrapping around those calls.
The repository lint enforces this: a raw numeric timeout in a role spec is
rejected with a pointer to resolveTimeout, which is how a hardcoded 500ms
wait added during this session was caught.
Merge work rather than authored here; not deployed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The merged branch moved every role-local script out of files/ into a folder
named for its language (files/python/, files/php/, files/shell/), but the
moves lost their staging and never landed. HEAD still carried the old paths
while the worktree had the new ones.
Move the 24 remaining scripts and update the six places that resolve them:
the mcp-adapter instance/isolation/probe tasks, the litellm key task, the
adapter README and the endpoint-path lint. Every changed line in those
files is a path.
Also drop roles/sys-svc-compose/files/swarm/stack_ready.sh, a stale
duplicate of the copy already tracked under files/shell/swarm/.
The added comments are the branch's own `nocheck: mirrored-unit-test`
directives, which state why a script whose measurement only exists inside
its container cannot have a mirrored unit test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Moodle 303-redirects every request whose Host is not $CFG->wwwroot, and an MCP
client dialling the internal service name cannot follow that redirect from the
container network. The previous accommodation was on the wrong side: the probe
sent -H 'Host: <public domain>' and -H 'X-Forwarded-Proto: https' and pinned
mcp_probe_host to MOODLE_DOMAIN, so it asserted the endpoint works when spoofed
and said nothing about how a real client reaches it.
Both headers and the probe host are gone. Acceptance moves into config.php,
which reads MOODLE_INTERNAL_HOST -- rendered from MOODLE_MCP_SERVICE_KEY -- and,
when the incoming HTTP_HOST matches it, rewrites HTTP_HOST and SERVER_NAME to
the wwwroot host before Moodle's redirect logic runs. The comparison splits on
':' so a port cannot defeat the match, and a request from any other host is
untouched, leaving the canonical-host protection intact for real traffic.
None of that could have taken effect on a deployed instance. The bootstrap seeds
the code directory with `cp -an`, which never clobbers, so an existing volume
keeps its first-run config.php forever and every later config change is silently
discarded. moodle_refresh_config now copies the deployment-managed config.php on
every start, which is what makes the rewrite above reachable at all.
Not verified: no MCP call has been observed succeeding against the internal name
since the change. This is also the same failure class that ended the last
deploy on another role, where the client already shared the provider's network
and had the right alias registered, yet the DNS search suffix still resolved the
single-label name to the public ingress. If that applies here too, the Host fix
is correct and the request still never arrives -- and removing the spoofing
headers will surface it as a failure where the probe previously passed.
Two further consequences worth stating: config.php is now overwritten on every
start, so an operator edit inside the container is discarded; and the HTTP_HOST
rewrite runs in the hot path of every page load, narrow though its condition is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Moodle 4.5 ships aiprovider_openai in core, where every action carries its own
endpoint rather than one shared base URL, so the provider is configured per
action. aiprovider_openaicompatible would be the natural fit but requires
Moodle 5.0, which the role does not pin, so the per-action endpoints are the
only surface available on this branch. MOODLE_LITELLM_SETTINGS carries that as
a list of component/name/value triples and also enables the courseassist and
editor placements: a configured provider with no enabled placement exposes no
AI surface to any user, so both halves are required for the feature to exist.
An earlier revision set a global `enableai`, which does not exist in 4.5.
tasks/utils/ai_gateway.yml verifies the shared gateway contract, writes each
setting through admin/cli/cfg.php, then purges caches. Without the purge Moodle
keeps answering from its config cache and the new endpoint appears to have been
ignored.
The egress rule is the half that is easy to miss. Moodle's cURL security helper
blocks private address ranges outright, so a correctly configured in-cluster
endpoint is still refused, and it surfaces as a generic provider error rather
than as a blocked request. 01_manager_ops.yml therefore writes
curlsecurityblockedhosts and curlsecurityallowedport, appending the gateway's
port only while litellm is enabled. This deliberately narrows a security
control: 172.16/12 and 192.168/16 become reachable so the container network
resolves, while 10.0.0.0/8, link-local and loopback stay blocked. The README
records the trade-off.
MOODLE_EGRESS_NEWLINE is double-quoted on purpose. Moodle stores both settings
one entry per line; a single-quoted YAML scalar hands Jinja a literal
backslash-n, the whole list collapses into one unparseable token, and the net
effect is that every port is blocked rather than the intended two. Verified in
the container with negative controls: keycloak and litellm reachable,
10.208.0.5 and 127.0.0.1 still refused.
The spec opens a site-administrator session -- through Keycloak when sso is on,
because 04_oidc.yml rewrites every account to auth: oidc and no password
reaches it then, and through the native form otherwise -- asserts the endpoint
is http on an undotted host that is not Moodle's own domain, then calls
aiplacement_editor_generate_text and requires a non-empty completion. That last
step is a prompt actually answered through the gateway, not a config read-back.
Verified: the deploy-time read-back assertion ran ok in a variant-2 round and
the spec executed rather than skipping. The egress newline defect and its fix
were both measured in-container.
Not verified: I confirmed the spec ran but not that its final assertion passed,
and the cfg.php loop is changed_when: false, so the deploy log cannot show
whether a setting actually moved.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A role-files-layout exemption frees a tree from files/<lang>/ because
its import paths are fixed from outside. The mirrored-test lint derived
its scope from that same directory, so every layout exemption silently
became a test exemption nobody ever argued for.
The language now comes from the file suffix. Test code is excluded by
scope rather than exempted: a test-* role and a files/playwright/ or
files/test/ tree are the tests, so demanding a test of them is circular.
The eleven files this newly reaches are framework-instantiated and carry
a stated reason each.
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Playwright declares the timeout option on locator.isVisible() and isHidden() as deprecated and ignored: both return immediately and never wait. Passing resolveTimeout(10_000) to one therefore changes nothing at runtime and only tells the reader a wait happened that never did. That reading is what let 7538b8a0b conclude two genuine call sites were fine, and what hid the false negative b97cf8888 had to fix.
Sixty call sites across fifty-eight role-local specs lose the argument and nothing else. Every added line in this commit is a bare isVisible(); a grep for added lines that are not one returns zero, so a behaviour change cannot have entered here. Files whose diff also alters waiting semantics are deliberately not part of this commit.
A repository lint added in b97cf8888 keeps the construct out. It matches isVisible and isHidden only, because isChecked, isDisabled, isEditable and isEnabled take a real timeout.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Moodle kept everything under dataroot, which is the shared NFS export in swarm
mode - including localcachedir. Upstream is explicit that this is the one
directory that must not be shared: config-dist.php marks tempdir and cachedir
as 'MUST BE SHARED by all cluster nodes' and localcachedir as 'Intended for
local node caching'.
The path is declared once, as the mount target in meta/volumes.yml, and every
other place derives from it: vars/main.yml reads it back through
lookup('volume', ...), compose passes it as a build argument, the Dockerfile
exports it, the entrypoint requires it with :? and creates it with the runtime
owner, env.j2 hands it to config.php as MOODLE_LOCALCACHEDIR. The two mounts of
the same volume share one YAML anchor rather than repeating the literal.
The cron service mounts it too, because it runs the same code and would
otherwise write its cache somewhere else than the web service reads it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nine roles reached their MCP surface as the deployment administrator or with
an account password kept beside the account in credentials. A Moodle site
admin bypasses every capability check, a Nextcloud app password is not
scopable, a Gitea token reads every private repository, and a GitLab token on
root reads every private project. Each role now declares its own principal in
meta/users.yml and provisions it, so the exposed surface is what the service
role permits. jellyfin, zammad and snipe-it kept their account but took its
password from a credential named token; they read the registry too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Install the protocol plugin during the image build, because a Moodle
plugin must match the pinned release and installing it at runtime would
leave the two free to drift.
Provision the webservice and its token through PHP, and attach only
read-only functions, so the read-only tool policy is enforced by the
service definition rather than declared in metadata.
Address the endpoint as its canonical origin. Moodle compares every
request against wwwroot, so a probe over the internal HTTP origin was
answered with an HTML redirect instead of JSON and every retry hit the
same wall; the verify step now sends the canonical Host and the
forwarded-proto header.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three waits in the login path were raw literals while their neighbours
already went through resolveTimeout, so they kept a flat 30s where a tor
circuit needs the onion multiplier.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
core/togglesensitive fetches its template async and replaces the
password input via outerHTML, serializing the empty value attribute
and discarding anything typed before the swap. Swarm timing lost that
race 3/3 (empty password= in every POST body of run 30161503176),
compose won it. Both specs now wait for the post-replacement
.toggle-sensitive-wrapper input before filling. profile-readonly also
asserts login success; it previously passed vacuously on the login
page where none of the 19 profile fields exist.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
core/togglesensitive fetches its template async and replaces the
password input via outerHTML, serializing the empty value attribute
and discarding anything typed before the swap. Swarm timing lost that
race 3/3 (empty password= in every POST body of run 30161503176),
compose won it. Both specs now wait for the post-replacement
.toggle-sensitive-wrapper input before filling. profile-readonly also
asserts login success; it previously passed vacuously on the login
page where none of the 19 profile fields exist.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Transient CDN/mirror TLS resets are a recurring hard-fail class (docker.com, PyPI). Harden all three egress paths at their SPOT: the curl filter now emits --retry-all-errors (retries=0 opts a call out), so every ansible curl retries; a new lint requires --retry-all-errors on file-downloading curl and --tries on wget in verbatim files/*.sh (which cannot use the filter), fixing the moodle/alias downloads it flagged; and the update role configures native retries for every package manager - apt Acquire::Retries, dnf retries=, pacman XferCommand over retrying curl - covering all five CI distros (ubuntu shares apt, centos shares dnf).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolves the runner Dockerfile to the lint-standard Acquire::Retries=3
and keeps the collabora TODO wording; hardens the merged tree for the
stricter lints (apt retries in the tor Dockerfile, shell-or-true
markers, comment cleanup).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A transient Debian/Ubuntu mirror fetch aborts a docker build layer with exit 100 (it killed the web-app-nextcloud swarm openldap build while the same image built fine elsewhere). New lint under tests/lint/docker/dockerfile/apt/ flags any RUN with apt-get update/install that omits Acquire::Retries; retrofit all 15 apt Dockerfiles with -o Acquire::Retries=3. Both apt lints move into the apt/ subpackage. dnf retries by default and apk has no equivalent flag, so both stay out of scope.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Six replicas race the bootstrap lock across three NFS clients; the truncating open invalidated the holder's lock state, so flock died with EIO and the task crash-looped with exit 65 until the winner finished the ~6min code copy. The lock file is now opened append-only and flock failures fall back to a sentinel recheck loop instead of killing PID 1. The converge budget gets the same 600-retry override the other slow-bootstrap roles already carry; the default 150 missed this run's recovery by six seconds.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.
Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Over Tor the redirect chain behind a login/logout/OIDC click routinely exceeds Playwright's fixed 30s action timeout, so the click throws before the flow's scaled waits run. Sweep all role Playwright suites and the shared persona helpers: give every navigation/auth-transition click an onion-scaled timeout (.click({ timeout: resolveTimeout(30_000) })) and route the remaining raw millisecond literals (timeout options, waitForTimeout, sleep-promise setTimeout, positional wait args) through resolveTimeout. Local UI clicks (menus, toggles, modal dismissals, DOM clicks inside evaluate) stay untouched.
Two new lints keep it that way: test_no_raw_timeout_literals forbids raw numeric timeout literals in Playwright specs (opt-out: nocheck: raw-timeout), test_persona_clicks_scaled requires every click in the shared persona auth flows to carry a resolveTimeout timeout (opt-out: nocheck: unscaled-click).
Verified: node --check on all 133 changed JS files, all 13 playwright lints green, make autoformat clean, make test green (2745+196+163+4 tests OK).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New lint test_no_raw_goto.py forbids raw `page.goto(...)` in role playwright specs + persona flows (the gotoOnion wrapper in personas/utils is exempt) so navigation retries transient Tor-transport errors over onion. Migrates every raw call across ~75 roles (343 sites) to `gotoOnion(page, ...)` and teaches gotoOnion to detect relative onion URLs (`isOnionCanonical`). gotoOnion is a behaviour-identical drop-in on clearnet (single attempt, same return). Verified: no-raw-goto lint green, eslint 0 errors, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the global TIMEOUT_FACTOR and a shared timeouts.js helper: resolveTimeout(base) scales by TIMEOUT_FACTOR and, over a .onion target, an extra multiplier, so Tor's per-request latency no longer trips hard-coded timeouts. Migrate every raw timeout in the specs (timeout:/timeout=/setTimeout/waitForTimeout) to resolveTimeout, stage the helper in all three staging paths, and add test_no_raw_timeouts to forbid raw numeric timeouts. Clearnet behaviour is unchanged (factor 1: resolveTimeout(N) === N).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A curl against an accepted-but-silent peer hangs forever: task retries
never fire because the first attempt never returns, and the CI job dies
at the 6h runner cut (proven by the mailu admin-API hang that took the
suitecrm/peertube swarm legs down). The new `curl` filter renders the
hardened base command ("{{ 30 | curl }}" -> curl -s --connect-timeout 5
--max-time 30); scaling composes with lookup('timeout').
test_curl_via_filter enforces it: bare curl invocations in role tasks,
templates and shipped scripts fail the lint with a pointer to the
filter. Plain .sh files, which cannot render Jinja, pass by pinning
--max-time inline; Docker healthchecks are exempt (their timeout field
caps them); `# nocheck: curl-timeout` opts out (catalogued in
suppression.md).
Convert all 34 existing invocations: mailu (replacing the short-lived
MAILU_CURL var), elasticsearch, checkmk, yourls (300s for the
schema-creating installer POST), the telegram alert script, and the
plain scripts of jellyfin, svc-runner, apt_repo, moodle (600s tarball)
and the mattermost provisioner. Mailu's user/token tasks additionally
carry task-level timeouts so a wedged container exec cannot stall the
play.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Moodle's login page re-renders the form during theme JS init, so the
first fill could land in a discarded DOM node and submit an empty
password. Both specs now retry fill until the field holds the value.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
moodle runs replicas:3 sharing one NFS code volume. The entrypoint guarded its cp -a with a version.php TOCTOU check, so several replicas copied concurrently; losers hit 'File exists' and, under set -euo pipefail, aborted before chown + php-fpm, leaving an incomplete/unreadable tree -> install_database.php failed Permission denied (rc 255). Serialize the bootstrap under flock, gate on an atomic .bootstrap.done sentinel, cp -an || true so a benign collision can't abort, make the tree world-readable before the install (root_squash neuters the in-container chown), and make losing replicas wait for the sentinel.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Five interacting bugs kept `test-lam-writethrough.js` red across
all retries on the live stack:
- URLs hit `/templates/...` but LAM is mounted at the Apache alias
`/lam/`, so the post-Keycloak goto landed on a 404.
- Password fill used `LAM_PASSWORD` (LAM master), but LAM with
`loginMethod=list` expects the selected admin's LDAP bind cred.
Adds `LDAP_ADMIN_PASSWORD` to `playwright.env.j2` (gated behind the
existing LAM service-enabled `{% if %}`) and prefers it over the
master password.
- Login-form input selector also accepts `name='passwd'` (LAM's
actual HTML), not only `name='password'` / `id='passwd'`.
- User-list filter selector restricted to `input[type='text']` so it
no longer matches the `apply_filter` submit button.
- biber-row locator targets the LAM Edit anchor (the username cell
is a plain `<td>`, not a link), then the assertion checks the
`input[name='initials']` value via `toHaveValue` instead of the
body's textContent (the probe lives in the input value).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the LAM_OAUTH2_FRONTED skip predicate with a branched login
flow: when LAM is fronted by oauth2-proxy/Keycloak, click through the
Keycloak login form with the administrator credentials (already in
LAM's allowed_groups via meta/services.yml), then fall back to LAM's
own admin form only if it still appears post-SSO. Native path
(LAM_OAUTH2_FRONTED=false, lam-public profile with LDAP-bind auth) is
unchanged.
Move adminUsername/adminPassword from _shared.js module scope into
shared.env so the LAM verifier can read them (previously single-
consumer to beforeEach, now multi-consumer).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirror the nextcloud playwright layout: orchestrator + _shared.js for
multi-consumer logic, one test-*.js per scenario (baseline, ldap-variant,
profile-readonly, keycloak-scope, lam-writethrough, administrator-persona,
guest-persona). Extract the duplicated Account REST middleName-write
into setMiddleNameViaAccountRest in _shared.js, parametrized by
withRestore so keycloak-scope drives the round-trip+restore path and
lam-writethrough drives the write-only path.
Add LAM_OAUTH2_FRONTED to playwright.env.j2 and gate the LAM UI verifier
on it. With web-app-keycloak deployed, LAM is fronted by oauth2-proxy,
its native /templates/login.php is replaced by the Keycloak login page,
and the LDAP-bind admin password is unusable. The Keycloak-side write
is still verified end-to-end by keycloak-scope's main test.
Net diff -20 lines vs. the original monolith despite the new helper.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
