diagnostics.sh runs under a 900-second deadline that the sweep deadline
can shorten further. The package manager section came last, after the NFS,
ganesha backtrace and controller probes, so a cut-off run would have lost
exactly the log this collection exists for. It now runs first.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Proxy:
- server_tokens off, X-Powered-By stripped from upstream responses,
HSTS max-age raised to one year.
- No global X-Content-Type-Options: under nosniff, ORB (step 3.4)
blocks no-cors cross-origin responses typed text/html, e.g. tracking
beacons answered with an empty 204.
PHP (CVE-2026-109618, CVE-2026-109619):
- New role sys-svc-compose-php installs a shared ini (expose_php Off,
session.upload_progress.enabled 0, fatal_error_backtraces 0 for
PHP 8.5, interim memory_limit -1) and a PHP-FPM [www] drop-in
(pm.max_requests = PHP_FPM_MAX_REQUESTS) once per stack host,
before 04_files renders any compose file.
- The host file names carry a sha1 prefix of the controller-side
source, so a content change alters every PHP app's compose.yml and
recreates it, also on remote deploys where compose_volumes cannot
read the stack host file and hashes the path instead.
- 20 PHP roles mount the ini as a compose/swarm config into their
image's scan dir; moodle, nextcloud and shopware also mount the FPM
drop-in. Magento keeps its own pool: its php-fpm.conf has no
include= and is rendered without notify.
- Joomla's php-upload.ini now loads as zzz-joomla-uploads.ini so its
own memory_limit wins: the media API compares Content-Length with
toBytes(memory_limit) and rejects every upload at -1.
TODOs record the memory_limit revert condition and accepted
trade-offs, Akaunting on PHP 8.1, FusionDirectory on PHP 7.4 and the
public Keycloak admin console.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
In swarm mode `docker stack deploy` ignores the host IP of a short port
mapping, so every `127.0.0.1:<port>:<port>` that container_ports renders for
a `ports.local` entry is published by the ingress routing mesh on every
node, via docker-proxy on 0.0.0.0 and [::] plus DNAT in nat PREROUTING.
Databases, LDAP and application backends answered from outside the cluster.
svc-swarm-node now loads an nftables table through svc-net-firewall. Its
prerouting filter chain (priority mangle, ahead of Docker's dstnat) drops
packets addressed to the node for any ingress-published port that no
application declares under `ports.public`. Loopback, docker0,
docker_gwbridge, the user bridges and the swarm nodes' own addresses stay
admitted. Host-side provisioning over 127.0.0.1, containers that reach a
host port through host-gateway, and node-to-node consumers such as the
registry-cache mirror keep working.
Both sets are read live on the manager (`docker service inspect` for
`.Endpoint.Ports`, `docker node inspect` for `.Status.Addr`), so ephemeral
ingress ports and nodes outside `--limit` are covered. The table is loaded
on a node before it joins, reloaded on every node after the join, and
reloaded after each stack deploy by the sys-svc-compose swarm handler.
Refs #1089
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Six Fedora jobs of main runs 39494396920 and 39590304501 (Postmarks#0,
OpenBao#1, Jenkins#1, Magento#0, Matrix#2, OpenResty#0) died in dnf with
`Failed to download packages`. Ansible's dnf5 module reports nothing
more: `failures: []`. Its file logger writes the download errors, with
mirror, URL and status, at debug level to /var/log/dnf5.log, but no
rescue bundle collects that file, so the cause of five of them stays
unknown.
The swarm stack diagnostics now append the last 3000 lines of
/var/log/dnf5.log and, for dnf4 hosts, /var/log/dnf.librepo.log of every
node. A node without them records the `tail` error and the capture goes
on.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`tarball_name()` picked the archive from `platform.machine()` alone, so
an end-of-life Node.js on Intel macOS would have unpacked a Linux build
into /usr/local, Homebrew's prefix there. The upstream install now
supports Linux only and stops with a message to install a supported
Node.js through the package manager everywhere else.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The welcome message linked to https://www.bigbluebutton.org/html5 for
"tutorial videos". That page is gone: it answers HTTP 404 with and
without www and with a trailing slash, while bigbluebutton.org itself
answers 200. tests/external/repository/test_urls_reachable.py reports
any other 4xx as a failure, so test-external went red for every branch.
Link the BigBlueButton documentation instead, which the welcome footer
already references, and name it as such.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Debian trixie ships nodejs 20.19, whose release line reached its end of
life on 2026-04-30. `@mermaid-js/mermaid-cli` 12 and its puppeteer
declare `node >=22.13.0`, so `make install-lint` failed in every fresh
Debian stack container with `All providers failed for
chrome-headless-shell` (#1088).
`ensure_node_present()` still installs the distro package first, then
reads the release line of `node` on PATH and looks it up in the
nodejs/Release schedule. A line past its `end` date is replaced by the
newest LTS from nodejs.org/dist/index.json: SHASUMS256.txt is verified
with gpgv against the active Node.js release keys from
nodejs/release-keys, the tarball against its SHA-256 entry, and the
release is unpacked into /usr/local, after which `node` on PATH must be
that release. Nothing pins a major: the next LTS line is picked up the
same way. Ubuntu 26.04's Node.js 22, supported until 2027-04-30, stays.
`ensure_npm_present()` calls it first, so a container that already holds
the end-of-life node is upgraded too rather than skipped because npm
exists. The call runs once per process.
Closes#1088
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`https://public.api.bsky.app` in meta/csp.yml is a CSP origin, not a
page. Its root answered HTTP 404 to CI runners at least twice on
2026-10-20 (runs 39417613648 and 39469069313) while answering 200 from
other hosts, and each time test-external failed the whole workspace or
PR test job.
Mark the line `# nocheck: url`, as the neighbouring bsky origins already
are, and place it directly above an already-marked entry: the
suppression also covers the following line, so this order keeps
`https://events.bsky.app` probed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The nginx sidecar's static location (`location ~* \.(css|js|...)$`) ended
in `try_files $uri =404`. Moodle 5.3 serves its ES modules from
`/core/esm/<rev>/...` through r.php, and those paths do not exist on
disk. Regex locations win over `location /`, so every module URL ending
in `.js` (`bootstrap/bootstrap.js`,
`@moodlehq/design-system/theme-boost/index.js`, ...) got nginx's own 404
and never reached PHP; both are preloaded on every page.
Fall back to `/r.php` like `location /` does. Files that exist on disk
are still served directly with their cache headers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Moodle 5.3 marks `external_generate_token()` as a final deprecation
(`#[deprecated('\core_external\util::generate_token', since: '4.4',
final: true)]` in public/lib/externallib.php), and
`core\deprecation::emit_deprecation_notice()` throws a coding_exception
for every final deprecation, whatever the debug level. The MCP
provisioning script called it whenever no permanent token existed yet,
so every fresh install of the variant that enables MCP failed at
"Enable web services and mint the MCP bearer" (main run 39494396920,
Moodle#1: `Error code: codingerror` from deprecation.php:273 via
externallib.php:30).
The variant had not run in CI since the Moodle 5.3 bump, which is why
this surfaced only now.
Mint the token with `\core_external\util::generate_token()`, which
takes the service record instead of its id, and create the service
account with `\core\user::create_user()` in place of
`user_create_user()`, deprecated since 5.3 (MDL-82650). Neither needs
externallib.php or user/lib.php any more.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
config.php set `$CFG->debug = 32767` whenever MODE_DEBUG was true, as
the development and CI inventories set it. Moodle 5.3's
`core_component::is_developer()` treats any debug bit defined in
config.php as developer mode, so `init()` skipped
`$CFG->cachedir/core_component.php` and ran `fill_all_caches()` on
every PHP request: a walk over every plugin directory and every
`classes/` tree. The autoload in setup.php triggers it before the
ABORT_AFTER_CONFIG return, so even font.php and requirejs.php paid it.
On local disk the walk is cheap. In swarm the code tree is an NFS4
mount, and the walk turns into GETATTR round trips: main run
39494396920 served `/theme/font.php` in no less than 1.6 s and
`/lib/requirejs.php` in no less than 2.0 s, the five FPM workers per
replica queued, and the CSP check timed out loading the academy page
after 20 s on both Moodle swarm jobs.
Write `debug` and `debugdisplay` into the site config with
`admin/cli/cfg.php` instead, on every deploy and as integers (32767/1
in debug deployments, 0/0 otherwise). `is_developer()` reads only
values defined in config.php, so the component cache is honoured,
while `setup.php` still applies the stored debug level once the
database is up. The assignments leave config.php entirely, because
cfg.php refuses to change a name config.php defines, and
MOODLE_DEBUG leaves env.j2 with them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The shared `container pull` of the Playwright image had no timeout. A
wedged registry-proxy tunnel kept a local deploy on that task for 18
minutes with no traffic until the pull was killed by hand; the task's
`retries: 3` never got a chance because the first attempt never
returned.
Run the pull under `timeout -k 60 <lookup('timeout', 1800)>`, the same
shape the compose build handlers use. A stuck pull now exits with 124
and the existing retry loop starts a fresh connection.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The device backup runs `rsync -aHAXP` since #1090, which needs the
mounted filesystem to store owners, POSIX ACLs and extended attributes.
The role never creates that filesystem, so name the requirement where
the operator reads how the device is used.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
