config.php set `$CFG->debug = 32767` whenever MODE_DEBUG was true, as
the development and CI inventories set it. Moodle 5.3's
`core_component::is_developer()` treats any debug bit defined in
config.php as developer mode, so `init()` skipped
`$CFG->cachedir/core_component.php` and ran `fill_all_caches()` on
every PHP request: a walk over every plugin directory and every
`classes/` tree. The autoload in setup.php triggers it before the
ABORT_AFTER_CONFIG return, so even font.php and requirejs.php paid it.
On local disk the walk is cheap. In swarm the code tree is an NFS4
mount, and the walk turns into GETATTR round trips: main run
39494396920 served `/theme/font.php` in no less than 1.6 s and
`/lib/requirejs.php` in no less than 2.0 s, the five FPM workers per
replica queued, and the CSP check timed out loading the academy page
after 20 s on both Moodle swarm jobs.
Write `debug` and `debugdisplay` into the site config with
`admin/cli/cfg.php` instead, on every deploy and as integers (32767/1
in debug deployments, 0/0 otherwise). `is_developer()` reads only
values defined in config.php, so the component cache is honoured,
while `setup.php` still applies the stored debug level once the
database is up. The assignments leave config.php entirely, because
cfg.php refuses to change a name config.php defines, and
MOODLE_DEBUG leaves env.j2 with them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The entrypoint removed the old tree before it copied the new one. A
copy that failed left the volume without a working Moodle, and every
restart removed again instead of recovering. It now copies into a
staging directory inside the volume and swaps the trees once the copy
is complete, so a failed copy leaves the old tree in place.
An image that ships an older release than the volume records no longer
replaces it and exits with a message instead. Two instances on
different releases would otherwise overwrite each other's tree.
upgrade.php reports "No upgrade needed" on stderr, so the task counted
every run on a current site as a change. It reads stderr now.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The pin moved from 4.5.11 to 5.3.0, and the role fitted neither a new
install nor an existing one.
Moodle 5 serves from public/ and routes every other path through
r.php. nginx now uses public/ as its root, hands unknown paths to the
router and passes PATH_INFO for scripts with slash arguments, and
config.php declares the router as configured. The paths the role checks
and writes (version.php, auth, webservice, the OIDC plugin) moved below
public/, the URL of the OIDC plugin stays auth/oidc. The source archive
no longer contains vendor/, so the image installs the Composer
dependencies it declares. The OIDC plugin is pinned to v5.2.1, the
newest release.
An existing site kept its old code: the entrypoint filled the code
volume once and never again. It now records the release it copied and
replaces the tree when the image carries another one, and a new task
runs admin/cli/upgrade.php. Plugins that were installed through the web
interface are not carried over, the README says how to ship them.
A site installed with 5.3 forces login by default, which 4.5 did not,
so its start page answers 303 to the login page and the webserver
health check refused it. The role declares 200 and 303 as the codes
that check accepts: 200 is what an upgraded site keeps answering.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
PASSWORD_TOKEN_RE matches the token anywhere inside a Jinja expression, so a
path or a filename carrying it demands "| quote" as loudly as a credential
does. It fired on Moodle upstream CLI script:
roles/web-app-moodle/tasks/01_manager_ops.yml:55: In shell tasks, password
expressions must include quote: {{ [MOODLE_CLI_DIR, reset_password.php] | path_join }}
The credential on the next line already carries the filter. Listmonk hit the
same shape earlier and was resolved by renaming the file, which is not open
here: the name belongs to Moodle.
The rule had no suppression at all - it never called is_suppressed_at - so a
marker in the role would have been silently ignored. Teach it the marker, then
use it.
Suppression is honoured at the finding line OR at the shell: key that opens the
block, and the second placement is the one that matters. A marker beside the
finding necessarily sits inside the scalar, and this block is folded: lines
join with spaces, so a hash would comment out the rest of the command and
swallow --username, --password and --ignore-password-policy. On the shell: key
it is a YAML comment instead. The failure message now says so, so the next
false positive explains itself rather than repeating the discovery.
Verified the command is untouched by loading the task and reading the string
Ansible would hand the shell: no hash, no marker, byte-identical.
Note the cost: the marker grants the whole block, not one expression, because
a per-expression marker cannot be placed. This block loses the guard against a
future unquoted addition.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
install_database.php was the only place that ever wrote the Moodle site
administrator password, and its failed_when tolerates "Database tables already
present". That is not laxity: upstream aborts in install_database.php before
install_cli_database() writes the password once tables exist, so the role
swallowed exactly the path in which the write is unreachable. After the
credential rotation gate Moodle therefore kept the install-time secret while
templates/playwright.env.j2 rendered the rotated one, and the login answered
"Invalid login, please try again". The same job proves it: variant-2/sync
reports failures="0" with the spec passing in 5.6s, variant-2/async
failures="1", and only the rotation gate lies between them.
The deeper problem was the credential itself. secrets.credentials.user_password
described itself as "Initial password for the Moodle admin user" - a role-local
secret for an account that is not role-local. Moodle hosts the platform
administrator, so read that user instead, the way web-app-baserow,
web-app-gitlab, web-app-penpot, web-app-matrix, web-app-bluesky and
web-app-pihole already do. lookup(users, administrator).password is stable
because roles/user-administrator/meta/users.yml pins it to
ansible_become_password; only a user with no declared password re-renders
{{ 42 | strong_password }} on every evaluation.
vars/main.yml now holds username, password and email as one constant each, so
install, realign and the Playwright env cannot drift apart again, and the
now-unread credential leaves meta/secrets.yml.
Realigning uses admin/cli/reset_password.php. Verified against the pinned
release v4.5.11: --username and --password are non-interactive (lines 35-37,
prompts only when empty at 76 and 88), --ignore-password-policy is required
because the platform password is not chosen to satisfy Moodle policy (line 96),
and the "when" gate is required because line 84 resolves only auth=manual rows
while tasks/04_oidc.yml rewrites every row to auth=oidc.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Moodle#1 of run 33414875521 failed in PASS 2 with rc 137, no output at all:
rc: 137, delta '0:00:01.530480', start 20:05:15.897321, stdout '', stderr ''
The rescue bundle names the killer to the millisecond:
moodle_moodle.3.mwdxfc7z6m3q877dr3ow1f5qr finished = 20:05:17.418099896Z
install task end = 20:05:17.427801
Ten milliseconds. OOMKilled is false on all six containers, so the surviving
alternative -- a per-cgroup OOM against mem_limit: 2g -- is out.
PASS 2 re-rendered the env file and rotated the file-backed secret hashes, so the
stack deploy changed the moodle and cron specs and swarm rolled every replica.
The rescue tree shows two generations for each moodle_moodle slot and for
moodle_cron, and exactly one for each moodle_nginx slot; moodle and cron are the
two services carrying env_file. Slots went at 20:05:17, 20:05:36 and 20:05:55 --
19s apart, which is parallelism: 1 plus delay: 10s. Whichever slot the exec drew,
it was going to be killed: the install needs ~176s and the roll takes ~39s.
The convergence gate did not stop this and structurally cannot. It is a handler,
it returns before the task starts, and a barrier at time T cannot protect an exec
that begins at T+1.6s. The same-pass control settles what does work:
svc-db-mariadb went through the identical gate (0.517s, zero retries) and its next
in-container exec survived, because it has retries. This task was the only
in-container command in the role without them -- the health probe above it has 120
and the mail-marker probe below it has 30.
MOODLE_CONTAINER_ADDRESS expands to a literal $(resolve-container-id ...) that is
re-executed per attempt, so a retry re-resolves onto the surviving container by
itself. until gates only on the SIGKILL signature; a genuine install failure
satisfies rc != 137 immediately and the existing failed_when still decides.
Not covered: this exec drew the first slot replaced and died at 1.5s, before any
DDL. A kill at second ~100 of ~176 could leave a half-written schema that matches
neither 'Moodle is already installed' nor 'Database tables already present', and
the retry would then fail differently. Unmeasured.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The container_address lookup does not return a container name in swarm. It
returns a shell fragment, "$(/usr/bin/resolve-container-id <stack> <service>)",
which is only a name once an interpreter evaluates the substitution. The shell
module provides one. The command module does not -- it execs the binary directly,
so the fragment arrives at container exec as literal text and the call dies with
"No such container: $(...)".
Compose hides the entire class: there the same lookup yields a bare name and
command: works. The failure exists only in swarm, and no swarm run had touched
these paths -- both were added by feature commits (b23653adf8, 0421ebeb16) that
wrote the same argv shape independently, which is the argument for a rule rather
than six edits.
Six call sites moved to the shell module: the moodle egress task, the three
moodle AI-gateway tasks, and the two wordpress AI-Engine tasks. The wordpress
pair keeps its stdin, so it uses the dict form with cmd:. Moving off argv exposes
the surrounding values to word splitting, so they now go through quote -- the
address itself must NOT, because single quotes would suppress exactly the
substitution this depends on.
test_argv_exec_needs_shell.py closes the gap that let this through.
container-exec-resolver checks that the resolver is used; it has nothing to say
about whether anything can evaluate the result. The new rule checks that other
half. It reuses the sibling's argv parser, which recognises only container/docker
in first position, so an argv list that does start an interpreter
(argv: [bash, -lc, ...], as joomla does) is correctly not matched.
Run against the tree the rule reported exactly the five sites found by hand plus
bigbluebutton, and passed over joomla and the discourse task, whose address is a
config lookup rather than the resolver. Bigbluebutton is pinned to compose via
compose_mode_force and can only ever see a bare name, so it takes a nocheck.
Five container-exec-resolver markers disappear with the argv form: under shell
the resolver is the ordinary path, not an exception.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Moodle 4.5 ships aiprovider_openai in core, where every action carries its own
endpoint rather than one shared base URL, so the provider is configured per
action. aiprovider_openaicompatible would be the natural fit but requires
Moodle 5.0, which the role does not pin, so the per-action endpoints are the
only surface available on this branch. MOODLE_LITELLM_SETTINGS carries that as
a list of component/name/value triples and also enables the courseassist and
editor placements: a configured provider with no enabled placement exposes no
AI surface to any user, so both halves are required for the feature to exist.
An earlier revision set a global `enableai`, which does not exist in 4.5.
tasks/utils/ai_gateway.yml verifies the shared gateway contract, writes each
setting through admin/cli/cfg.php, then purges caches. Without the purge Moodle
keeps answering from its config cache and the new endpoint appears to have been
ignored.
The egress rule is the half that is easy to miss. Moodle's cURL security helper
blocks private address ranges outright, so a correctly configured in-cluster
endpoint is still refused, and it surfaces as a generic provider error rather
than as a blocked request. 01_manager_ops.yml therefore writes
curlsecurityblockedhosts and curlsecurityallowedport, appending the gateway's
port only while litellm is enabled. This deliberately narrows a security
control: 172.16/12 and 192.168/16 become reachable so the container network
resolves, while 10.0.0.0/8, link-local and loopback stay blocked. The README
records the trade-off.
MOODLE_EGRESS_NEWLINE is double-quoted on purpose. Moodle stores both settings
one entry per line; a single-quoted YAML scalar hands Jinja a literal
backslash-n, the whole list collapses into one unparseable token, and the net
effect is that every port is blocked rather than the intended two. Verified in
the container with negative controls: keycloak and litellm reachable,
10.208.0.5 and 127.0.0.1 still refused.
The spec opens a site-administrator session -- through Keycloak when sso is on,
because 04_oidc.yml rewrites every account to auth: oidc and no password
reaches it then, and through the native form otherwise -- asserts the endpoint
is http on an undotted host that is not Moodle's own domain, then calls
aiplacement_editor_generate_text and requires a non-empty completion. That last
step is a prompt actually answered through the gateway, not a config read-back.
Verified: the deploy-time read-back assertion ran ok in a variant-2 round and
the spec executed rather than skipping. The egress newline defect and its fix
were both measured in-container.
Not verified: I confirmed the spec ran but not that its final assertion passed,
and the cfg.php loop is changed_when: false, so the deploy log cannot show
whether a setting actually moved.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A handler name was doing two jobs at once: the line an operator reads in the
play output, and the identifier a `notify:` binds to. The two pull in opposite
directions -- the log wants "restart the SSH service, with diagnostics", the
binding wants a token that survives quoting, YAML flow style and a shell
round-trip -- and a name that changes for readability silently changes which
handler runs.
Every handler that a notify reached by name now carries a `listen:` with that
exact wording, so the binding is explicit before anything is renamed. The
topics themselves become identifiers: `compose up` reads `compose-up`,
`Restart NFS ganesha` reads `restart-nfs-ganesha`, 22 of them across both
sides of every notify. Where the topic is assembled at runtime, both branches
moved together: the conditional in sys-service carries two literals, and the
LDAP renderer interpolates `import-{{ folder }}-ldif-files`, which is what
makes those handlers reachable at all.
Freed from the binding, the names take the emoji prefix every other task line
already has.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Moodle renders the same msmtprc as WordPress and mounts it into two services,
but nothing ever proved a message left the container: its healthcheck opens the
FPM port and nothing else. Moodle 0 of run 31658241728 passed green while mail
could not have worked -- the defect 2da285067 fixed for WordPress was invisible
here, and would have stayed invisible until a user reset a password.
WordPress proves delivery with the msmtp_curl flavor: the probe sends one mail
and writes a marker, and a task after convergence waits for that marker. Moodle
cannot use it, because curl cannot judge a port that answers FastCGI. Rather
than add a second msmtp_* class per liveness shape, the declaration now composes:
healthcheck:
flavor:
- msmtp
- connect
port_key: fpm
msmtp stops being a flavor and becomes a prefix -- shell that runs on the way
past and never decides liveness. connect joins the probes as the one that only
asks whether the port accepts a TCP connection. A declaration names exactly one
probe and any number of prefixes, as a string or a list; msmtp_curl stays as an
alias for [msmtp, curl], and the lint that validates declarations now shares
resolve_flavors with the lookup instead of keeping its own idea of a flavor.
The module became five responsibilities in one file, so it is a package:
probes.py answers which probes exist, prefixes.py what runs before them,
compose.py how a declaration becomes a rendered block. The curl helpers sit with
the probes and mail_branch with the prefix -- each next to its only caller,
rather than together in a module header.
Two defects in this work were found by measuring, not reading:
- the timing merge took the maximum across prefixes only, which cut Moodle's
start period from 15m to 120s and would have marked a heavy PHP app dead
while it was still installing. A prefix may now raise a timing and never
lower one, pinned by test.
- composing moved msmtp_curl's interval from 30s to 1m, because the composed
class derives from Curl where MsmtpCurl derived from Probe. That is the
interval every other curl probe uses, but it was not intended; it now has
its own test rather than riding along unseen.
Regression measured across the whole repo rather than argued: every declared
healthcheck was rendered with the pre-refactor code and with this one. 58 render
byte-identically, 1 is Moodle's intentionally changed declaration, and 2 differ
-- friendica and wordpress, in that interval alone, with the probe command
itself unchanged.
Coverage was checked in both directions. No test was removed: the four deleted
lines are an import, the registry literal, one class name in a url test, and the
helper, which now goes through compose() instead of reaching into PROBES -- a
tightening. New code had two gaps, both closed: connect was the only probe whose
emitted command no test pinned, and compose() raised KeyError for an unknown
name unobserved.
Not verified: no deploy has run any of these probes. The composed class is built
dynamically, so what it emits is proven for all 59 services while docker
accepting it is not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
meta/schema.yml became meta/secrets.yml, but its content still landed at
applications.<app>.credentials — the one meta file whose payload did not sit
under a root named after the file. Every other topic obeys that rule:
meta/volumes.yml goes to volumes, meta/rbac.yml to rbac. Credentials now go to
secrets.credentials.
The path exists exactly once, in utils/manager/credential_key.py, and
OVERRIDE_SECTION is derived from its two segments rather than spelled beside
them, so the dotted form used by --set and the nested form used by the
inventory cannot drift. Six consumers read it: the loader, the generator, the
CLI override parser, the two emit helpers, and the four lookup validators.
--set now accepts exactly one key shape,
applications.<app>.secrets.credentials.<key>. The three short forms are
rejected instead of guessed at: the same short key names different credentials
on different apps. That also retired primary_app_id, which only existed to
disambiguate them.
Three defects surfaced while wiring this up. _apply_one_role_schema still wrote
to the old root, so an app's own credentials would have moved while its
providers' stayed behind. web-app-pgadmin reads through
lookup('applications', ...).credentials.<key>, an access path no regex over
lookup('config', ...) could see, and would have deployed with an empty postgres
password. And _META_NON_GENERIC_TOPICS still listed 'schema' — a bare topic
name, invisible to any path search — so the generic file-root loop assigned the
raw schema under secrets and the follow-up write landed inside the cached
document, truncating keycloak's four credentials to one for every later
reader.
tests/integration/ansible/lookups/config/test_credentials_roundtrip.py pins both
directions of the seam and the cache invariant that last defect broke.
Existing inventories still carry the pre-rename root; migrating them is
deliberately out of scope here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three roles reach their MCP work from a manager-ops file rather than
from the role entry point, and the rename missed them, leaving them
including task files that no longer exist.
Install the protocol plugin during the image build, because a Moodle
plugin must match the pinned release and installing it at runtime would
leave the two free to drift.
Provision the webservice and its token through PHP, and attach only
read-only functions, so the read-only tool policy is enforced by the
service definition rather than declared in metadata.
Address the endpoint as its canonical origin. Moodle compares every
request against wwwroot, so a probe over the internal HTTP origin was
answered with an HTML redirect instead of JSON and every retry hit the
same wall; the verify step now sends the canonical Host and the
forwarded-proto header.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An untyped inspect searches every docker object kind, and a role's overlay network carries the same entity name as its main container, so the call can resolve the network and report a missing State key instead of naming the container. That is how the openresty assert died on every swarm job of run 30709070050.
Extend container-exec-resolver to cover inspect, stripping raw markers and quoted spans first so the flag walker survives Go template format strings. Add container-inspect-type, scoped to roles/ because the collision follows the entity naming convention and the outer act namespace carries no role networks. Sweep 40 call sites; suppress the delegate_to-resolved CIDs and the compose co-location probes with a stated reason.
Move the five container rules into tests/lint/ansible/shell/container/ and drop the container_ prefix from their filenames. Register both markers in the suppression docs, where the resolver marker was missing entirely.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.
Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
