A local deploy stopped four times in two days in "Ensure Docker apt key
and sources are consistent" (#849). apt_repo.sh downloads the Docker GPG
key on every run, and when download.docker.com refused the connection,
curl's four attempts were over within about six seconds and the script
returned 1, although the key from an earlier run was installed and is
all the following apt operations need.
install_key now fails only when no key is installed. With a key in
place, a refused download prints a warning and keeps it. A downloaded
key that differs still replaces the installed one, and the 0644 mode is
enforced in every case. The keyring path became an argument:
ensure-key-and-sanitize passes the keyring it already hands to
sanitize_sources, so the role's call still resolves to
/etc/apt/keyrings/docker.asc, and a test can point the script at a
temporary keyring.
Verified: tests/unit/python/roles/sys-svc-container/files/test_apt_repo.py
stubs curl and sudo. A refused download keeps the installed key and ends
with changed=0, a refused download without a key exits 1, and a new key
replaces the installed one. A copy of the script whose keep branch always
errors, the old behaviour, fails the first test. Two local deploys ran
the new script with rc 0 and an empty stderr and left the installed key
untouched. shellcheck is clean.
Not verified live: the refused download itself, which needs
download.docker.com to refuse a connection during a deploy. A host
without a key still needs the download and still fails when it is
refused.
Closes#849
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Both provisioners follow nextCursor for workflows and step skip for API
keys until a short page arrives. A cursor n8n repeats, or a key listing
that ignores skip, would keep that loop running forever and hang the
deploy instead of failing it.
workflows_named now remembers every cursor it followed and exits with
"FAILED listing workflows: nextCursor repeats" when one comes back.
api_keys_labelled exits when a non-empty page brings no key id it has not
already seen. Normal paging is unchanged.
Four regression tests drive both scripts against a fake API that repeats
its cursor or ignores skip; without the guards exactly those four fail.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
n8n 2 returns ten API keys and a hundred workflows per page by default. A
stale managed key behind the first page survived, and a managed workflow
behind it was created a second time. Both scripts now follow every page,
narrowed by label or name, and keep the exact match in the script.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
n8n 2 pages GET /rest/api-keys as {data: {items, counts, totals, owners}},
ten keys at a time, and matches `label` as a substring, so walking the body
raised "'str' object has no attribute 'get'". It also enforces API key
scopes, so activating the MCP server workflow needs `workflow:activate`.
Both scripts now read every list and create response through one
normalisation that unwraps the page, look the managed key up by label, and
name an unexpected shape by its keys only. The normalisation is taken over
from a392e2dd49 on feature/store-i18n.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The entrypoint removed the old tree before it copied the new one. A
copy that failed left the volume without a working Moodle, and every
restart removed again instead of recovering. It now copies into a
staging directory inside the volume and swaps the trees once the copy
is complete, so a failed copy leaves the old tree in place.
An image that ships an older release than the volume records no longer
replaces it and exits with a message instead. Two instances on
different releases would otherwise overwrite each other's tree.
upgrade.php reports "No upgrade needed" on stderr, so the task counted
every run on a current site as a change. It reads stderr now.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
MCP Adapter 0.7.0 for WordPress refuses a request that follows
initialize without the MCP-Protocol-Version header: 'Invalid Request:
MCP-Protocol-Version header is required for a 2025-06-18 session'. The
contract probe never sent it, so the CLI test of web-app-wordpress
failed at tools/list.
The probe now keeps the version the server answers initialize with and
sends it on every later request of the streamable HTTP transport, which
the specification asks of a client since 2025-06-18.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The pin moved from 4.5.11 to 5.3.0, and the role fitted neither a new
install nor an existing one.
Moodle 5 serves from public/ and routes every other path through
r.php. nginx now uses public/ as its root, hands unknown paths to the
router and passes PATH_INFO for scripts with slash arguments, and
config.php declares the router as configured. The paths the role checks
and writes (version.php, auth, webservice, the OIDC plugin) moved below
public/, the URL of the OIDC plugin stays auth/oidc. The source archive
no longer contains vendor/, so the image installs the Composer
dependencies it declares. The OIDC plugin is pinned to v5.2.1, the
newest release.
An existing site kept its old code: the entrypoint filled the code
volume once and never again. It now records the release it copied and
replaces the tree when the image carries another one, and a new task
runs admin/cli/upgrade.php. Plugins that were installed through the web
interface are not carried over, the README says how to ship them.
A site installed with 5.3 forces login by default, which 4.5 did not,
so its start page answers 303 to the login page and the webserver
health check refused it. The role declares 200 and 303 as the codes
that check accepts: 200 is what an upgraded site keeps answering.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The guard required a register on every async kcadm job and nothing more.
A result that is registered but no longer handed to async_reap_jobs,
directly or through the list it is collected into, passed it, although
the reap then neither polls that job nor sees it fail.
The test now follows each registered result to an async_reap_jobs
expression, by variable name with word boundaries, so kc_declared_client
does not pass for kc_declared_client_jobs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The stub answered every absolute name, so a guard that asked for the
backend twice and never for the exporter passed both tests. The stub now
records the names it is asked for, and the tests require the backend and
the exporter.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
COLLABORA_WOPI_HOST read the scheme and the port with
lookup('tls', 'web-app-nextcloud', ...) and the host with
lookup('domain', 'web-app-nextcloud'). The tls lookup aligns an app term
to its consumer: for a clearnet Collabora it answers for the clearnet
sibling of an onion-primary Nextcloud, while the domain lookup keeps the
onion name. Such a deployment renders https://<onion>:443, an origin
Nextcloud does not use, so the allowlist does not match its WOPI calls.
Nextcloud pins overwritehost and overwriteprotocol to its primary domain,
so the allowlist now asks the tls lookup for that domain instead of for
the app.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Since the targets take their scheme from the domain they probe, two
switches of the http_ready module no longer fit. Both still followed the
TLS state of the Prometheus vhost.
fail_if_not_ssl refused an http target, an onion vhost for example, as
soon as Prometheus itself was served over TLS. It is false now. The
module does not follow redirects, so the scheme of the target URL already
decides the transport.
insecure_skip_verify switched certificate checks off whenever Prometheus
itself had no TLS, also for an https target. Checks now stay on, except in
a self-signed deployment whose CA is not mounted into the Prometheus
containers, where the exporter has nothing to verify against.
The per-role test of the spec parsed every answer of the query API as
JSON. An answer that is not OK now counts as "no probe yet", so the poll
keeps waiting instead of ending on a parse error.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The check listed the torrcs without the limit through "grep -L ...; true".
With no torrc under /opt/chutney/net/nodes, or with one that cannot be
read, grep printed nothing, the trailing true turned its error into
success, and the test reported that every node runs one worker.
The check now counts the limit per file with "grep -cH" and reads the exit
status. A status above 1 or an empty answer fails the test, and a file
with a count of 0 is named as before.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Tor starts its worker threads by the number of CPU cores it detects, and
the cpus limit of the container does not change that number. On a host
with 20 cores every node of the private chutney network ran 21 threads and
grew to about 0.6 GB. The seven relays and authorities filled the 2g
limit and its swap within ten minutes, the kernel killed them one by one,
and no circuit could be built any more.
The entrypoint now writes NumCPUs 1 into the torrc of every chutney node
before it starts the network. Measured on that host over ten minutes: the
nodes run 3 threads and stay at 65 to 86 MB, the container at 0.47 GB
without swap, where it had filled 2 GB plus 2 GB of swap before.
The CLI test of the role fails when a chutney node runs without the limit.
Closes#824
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Nobody could sign in to Odoo through Keycloak. Odoo sent an http redirect
URI, and Keycloak refused it as "Invalid parameter: redirect_uri".
The addon auth_oauth_https builds that URI from web.base.url. Two things
kept it from doing so after the image moved to Odoo 20.
Its manifest declared version 19.0.1.8.0, and Odoo marks a module whose
version names another series as not installable. It only warns about that,
so the install task stayed green and the addon was missing. The manifest
now declares 1.8.0, which Odoo prefixes with its own series.
Once installed, the addon read the parameter with get_param, which Odoo 20
replaced by typed getters. The call raised AttributeError, a broad except
logged it and fell back to the URL of the request, which is http behind
the proxy. The addon reads the parameter with get_str now and no longer
catches the error.
A deploy now fails when the addon is not installed afterwards. The check
reads ir_module_module, because the output of "odoo -i" does not tell an
installed module from a skipped one.
The spec logged out by navigating to /web/session/logout. Odoo 20 accepts
POST only on that route and answered 405, so both SSO tests failed after
a successful login. The spec logs out through the user menu now.
A host that runs the suite with several workers still failed it. Odoo
computes the session token of an account from its OAuth access token, so
every new sign-in ends the other sessions of that account, and the spec
files share the administrator. The role caps its suite at one worker
with PLAYWRIGHT_MAX_WORKERS, which the central config now honours, and
the spec asserts the cap.
Closes#822
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every blackbox probe of a Tor deployment answered 404, so AppDown fired for
all deployed apps although they were up. The job built its targets from
canonical_domains_map(DOMAIN_PRIMARY), which knows no onion domain. In a Tor
deployment the vhosts answer under <sub>.<onion> only, and the front proxy
served the clearnet names from its default server.
The targets now come from lookup('domains'), the map the vhosts are built
from, and each target takes its scheme from the domain it probes instead of
from the Prometheus vhost.
Which roles are probed follows lookup('roles_with_service', 'prometheus'),
the list the role's own spec expects metrics for. It leaves out the database
roles, which have a domain in the map and no vhost, and the roles that
declare scrape: false because their vhost carries no Prometheus block.
Two roles declared more than their vhost delivers. web-svc-mirror renders a
vhost of its own with no backend behind it, and on a live stack its probe
answered 404 under the onion name as well, so it declares scrape: false now.
web-app-peertube has a backend, but its vhost template never included the
Prometheus locations. It includes them now.
The spec could not see any of this in a compose deployment. dotenv_quote
wraps PROMETHEUS_TARGET_ROLES_JSON in quotes there, JSON.parse returned a
string, the role list came out empty and the per-role test was skipped. The
value is decoded now, and the test also requires probe_success == 1 for
every role in the list.
Closes#792
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Of the 76 open code scanning alerts, 39 point at code that can change.
svc-ai-mcp-adapter:
- A path argument that is empty or made only of dots is refused.
Percent-encoding keeps a separator inside its segment, but "." and ".."
need none, so an upstream that resolves dot segments could answer from a
path the contract does not name.
- parse_duration splits on the units. The pattern it used before rescanned
a long run of digits from every start position.
Tests and the README generators:
- Every Jinja environment built with autoescape=False and a "noqa: S701"
uses select_autoescape now, as six test modules and
utils/templating/jinja.py already did. It escapes templates named
*.html, *.htm and *.xml, which none of these call sites loads.
- Dead code goes: a doubled "rel" block in the email lint, an unread
_SKIP_DIRS, and a module that _resolve returned to no caller.
- Two "except ...: pass" blocks become contextlib.suppress and
assertRaises.
- Nine test modules import unittest.mock one way instead of two, and one
lookup test imports its module once.
Closes#807
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
In the async pass the role runs kcadm jobs side by side. Every kcadm call in
the Keycloak container shares one config file. kcadm rewrites that file when
it refreshes the access token, about every 55 seconds, takes the file lock
with tryLock() and gives up at once when another kcadm process holds it. One
of two overlapping declared-client jobs then failed with "Failed to get lock
on /opt/keycloak/.keycloak/kcadm.config", and the deploy with it.
keycloak_kcadm_update now repeats a call that lost the lock, up to ten times
one second apart. The call fails while it refreshes its token, before it
sends its own request, so repeating it is safe. Every other error still ends
the call at once.
That covers the module only, so the async jobs are reduced to the ones that
go through it:
- The default-scope assignment was an async shell task. It ran beside the
shell tasks that follow it, and neither side repeats a lost lock. It runs
in line now.
- The two mail-settings jobs were async without a register, so nothing
reaped them and a failure was never seen. They run in line now.
The client job and the declared-client jobs stay async. A test fails when an
async kcadm task does not use the module or is not registered for the reap.
A config file of its own per call was the other option. It does not work by
itself, because the admin session lives only in the shared file.
Closes#805
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
nginx resolves its upstreams once, at start, so the frontend starts through
a guard that waits until `backend` and `exporter` resolve. A container
resolves with a search domain, and the zone of a deployment answers every
name with the address of the ingress. A service that Docker's DNS did not
know yet therefore resolved anyway, as `backend.<domain>`. The guard let
nginx start at once and nginx pinned the ingress as its backend. Every API
call then failed with 502 and the SPA showed "Bad Gateway" instead of the
login form, until the container was restarted.
The guard now asks for the absolute names, `backend.` and `exporter.`. The
search domain is not applied to those, so the lookup fails until the
services themselves resolve.
Closes#806
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The extension installer returned as soon as an add-on was installed, in
whatever version. A new pin in meta/addons/ therefore reached a fresh wiki
only, and an existing wiki kept the version it was first deployed with.
The installer now skips an add-on only when the installed version equals
the pin. Otherwise it calls install(id, version, namespace), which the
Extension Manager resolves as an upgrade, and it reports an error when the
add-on is not on the pin afterwards.
The check that follows the installer compared presence only. It takes the
pin as well, so a deploy fails when an add-on is installed in another
version.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The unit test of the alarm script built its Jinja environment with
autoescape switched off, which code scanning reports as a possible XSS
sink and fails the PR on. The template is a shell script, so
select_autoescape() leaves it unescaped exactly as before, and the finding
no longer applies.
Refs #790
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Stalwart closes a connection that exceeds its per-IP connection budget
without a greeting. msmtp reports that as "the server sent an empty reply"
with exit 76, and the mail health check treated exit 76 as permanent. One
such drop, in a second in which Alertmanager flushed its notifications,
failed a deploy although the same path accepted a mail one second later.
The health check now retries exit 76 when msmtp names the empty reply,
within the ten attempts it already gives to exit 69, 75 and 124. Every
other protocol error still ends the check at once.
The alarm mail was sent once, so the alarm about a failed unit was lost to
the same drop. It now makes three attempts ten seconds apart for the
transient cases of the health check and still reports the last error.
The limit on the server is left alone. Exempting the node's own senders by
source address would also exempt Tor clients and, in swarm, every external
client. That question is tracked in #793.
Closes#790
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Formatter output only: the parsed YAML and the Python AST of every touched file equal their previous versions.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
test-e2e-cli loads test.env with `set -a && . test.env && set +a`. The
unquoted `ONION_PORTS=25 80 110 ...` is not an assignment there but the
command `80` with a one-word env prefix: bash prints `command not found`,
leaves ONION_PORTS unset, and onion_ports.py reports "no onion-forwarded
ports declared; nothing to probe". The port check passed vacuously in every
tor job with more than one forwarded port (OpenResty, Listmonk, Pixelfed in
run 35661520542) and only ran where a single port was declared.
Pass the value through dotenv_quote, which quotes a space-separated value in
both compose and swarm and keeps the env-file lint satisfied. A unit test
renders the template in both modes and sources it the way test-e2e-cli does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tor attaches a stream to an onion to the rendezvous circuit that matches its
isolation and keeps it there until SocksTimeout; it never moves the stream to a
fresh circuit. The probe connected without SOCKS authentication and gave up
after 60s, so a dead rendezvous circuit would absorb both passes. That is the
most likely reading of run 35661520542 (web-app-mastodon#1, public Tor), where
<onion>:80 timed out twice right after curl had reached the same onion port.
Authenticate every connect with fresh random username/password credentials.
Tor accepts any and, under its default IsolateSOCKSAuth, builds a separate
rendezvous circuit per credential set, so a retry can no longer land on the
circuit that failed the attempt before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review had these removed across the role once already, and the 11 September
pass cleared every file that was flagged. The seal-key pin and the
recovery-key work then reintroduced the same pattern in nine new places.
Also removes two that predate the review and were never flagged: the
healthcheck note in meta/services.yml and the entrypoint note in
compose.yml.j2. Both are the same narrative-exception shape.
Param docs, file headers and the plain contract notes in the Playwright specs
stay: separate categories, none of them objected to. What the deleted
comments recorded lives in the role README.
Six assertions: the key is persisted on the deploy that creates it, it is
loaded before the logins that may need it, recovery is gated on both logins
having failed and a key existing, the rebuilt AppRole is the inventory one
rather than the recorded one, the recovery path ends by re-pinning, and a
rejected key fails loudly.
Recovering to the recorded pair instead of the inventory pair would strand
the next deploy the same way, which is why that one is asserted directly.
