Commit Graph
14682 Commits
Author SHA1 Message Date
evangelostsak 136c28f15d test(web-app-openbao): expose the RBAC account credentials and the KV mount to Playwright 2026-09-11 18:35:31 +02:00
evangelostsak 8dcfed33c3 feat(web-app-openbao): declare dedicated operator and reader RBAC accounts 2026-09-11 18:35:27 +02:00
evangelostsak 4296fbc487 feat(web-app-openbao): move the seal alert into the role that owns the metric 2026-09-11 18:35:23 +02:00
evangelostsak f5df096a2a refactor(web-app-prometheus): concatenate role-local alert rules instead of owning them 2026-09-11 18:35:19 +02:00
evangelostsak 699823f6cb test(prometheus): cover the alert_rule_roles lookup 2026-09-11 18:35:13 +02:00
evangelostsak 85abcdf627 feat(prometheus): discover role-local alert rules via the alert_rule_roles lookup 2026-09-11 18:35:03 +02:00
kevinveenbirkenbachandClaude Opus 5 8d8701d9b3 feat(apt): one Ubuntu mirror list for the package cache and image builds, with failover, a short timeout and a health test
The ubuntu CI image build (job 103277644636) hung on Canonical's
archive: apt fetched 26 MB at 417 kB/s, then 61.6 kB in 29 s, then sat
on the package download until the job was cancelled after 23 minutes,
while download.docker.com delivered 42.8 MB in under a second on the
same runner. Locally archive.ubuntu.com and security.ubuntu.com took
20-26 s to the first byte against 2 s for a university mirror.

INFINITO_APT_UBUNTU_MIRRORS in default.env is now the one list of
Ubuntu mirrors, in failover order: de.archive.ubuntu.com, ftp.fau.de,
ftp.halifax.rwth-aachen.de, archive.ubuntu.com.

- Image builds: the Dockerfile sources the list alongside the install
  scripts and runs scripts/install/apt-mirrors.sh first. In an image
  with ubuntu.sources it points both stanzas at apt's mirror+file
  method, priorities in list order, and sets Acquire::http::Timeout to
  10 s, so apt moves on from a mirror that fails or goes silent; other
  distros are untouched. In ubuntu:latest apt-get update took 8 s
  against 110 s on the stock sources, a dead first mirror was left
  behind within 25 s against 62 s on apt's defaults, and the swarm node
  packages (80 debs) came in 17 s. The Dockerfile's own RUN lines ran
  clean on ubuntu, debian, fedora and arch.
- Package cache: package.sh proxies the first entry as the primary
  Ubuntu repos and the second as their fallback. Nexus revalidates
  metadata on every request here (metadataMaxAge 0), which costs a
  full download from a mirror without an ETag; both de.archive and
  ftp.fau.de send one, and the 15 MB universe index now comes back
  from the cache in 0.02 s.
- tests/external/repository/test_apt_ubuntu_mirrors.py keeps the list
  honest: the first mirror must answer InRelease within 5 s (median of
  three) or the test names a faster later one, no mirror's -updates may
  lag the freshest by more than 24 h, and every entry must be the
  primary archive, a Canonical country alias or an enabled official
  mirror in Launchpad's registry. It caught the first draft of the
  list: mirrors.edge.kernel.org is not an official Ubuntu mirror.

After pulling, an existing checkout needs `make dotenv-force`: plain
`make dotenv` re-reads the old .env through BASH_ENV and keeps its
values.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 17:13:54 +02:00
kevinveenbirkenbachandClaude Opus 5 2ea06aac4c fix(cache): bound the frontend's wait on a primary apt repo so the mirror answers in time
test_packages_available failed locally in every quality pass with
"91 package(s) could not be checked at all ... archive.ubuntu.com:
timed out". The dev stack routes archive.ubuntu.com to the package-cache
frontend, which re-runs a request against the -mirror repo only once
the primary Nexus repo answers 502/503/504. From this host's docker
network Canonical's IPv4 addresses take seconds to connect and then
send nothing, and Nexus tries all nine addresses of archive.ubuntu.com
in turn before it gives up: apt-ubuntu answered 502 after 21-25 s, so
the frontend's first byte arrived after 27-46 s, past the probe's 20 s
timeout. apt-ubuntu-mirror (de.archive.ubuntu.com) served the same
files in 0.08 s.

The four primary locations now carry proxy_read_timeout 8s: nginx
gives up on Nexus itself, its own 504 hits the existing error_page, and
the mirror answers. A shorter Nexus upstream timeout was tried first and
dropped, because Nexus spends it once per address and still took
9-21 s. The mirror locations keep the global 300 s, and a healthy
primary answers well inside the bound, as before.

Measured after a compose restart: the ubuntu index arrives after
8-12.6 s with Origin: Ubuntu, debian still in 0.03-0.16 s, npm and PyPI
in 0.13 s; make test-external passed 13 of 13 and the full
make quality-high went green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 15:43:08 +02:00
kevinveenbirkenbachandClaude Opus 5 8fac8d0bc5 fix(bluesky): pin container inspect to the container type in the cfg heal
test_inspect_pins_type failed on heal_sso_proxy_cfg.sh from
293c67780d: a bare container inspect searches every object kind, and a
role's overlay network carries the same entity name as its main
container, so the existence check could match the network. The check
now passes --type container, so only a real container is stopped
before the directory is removed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 15:11:59 +02:00
kevinveenbirkenbachandClaude Opus 5 56c0716f64 fix(deps): lift js-yaml to 4.3.2 for GHSA-2883-xcg3-v3hh
Dependabot alert 6 on infinito-nexus/core flags js-yaml 4.3.1 in the
root package-lock.json: maxTotalMergeKeys does not limit CPU use for
empty merge sources, affecting >= 4.0.0, < 4.3.2. It arrives as a
development dependency through stylelint 17.15.0 -> cosmiconfig 9.0.2,
whose ^4.1.0 range already admits the patched release, so only the
js-yaml entry changes.

The resolved URL and integrity are the ones registry.npmjs.org serves
for 4.3.2; npm run inside the dev container had written the package
cache's /repository/npm-proxy/ path into resolved, which does not exist
on the public registry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 15:10:26 +02:00
kevinveenbirkenbachandClaude Opus 5 50a13c755b test(test-e2e-playwright): read the onion multipliers through the cached loader and the role-file constant
The flavor-coverage test from 214e7e29d1 read the role's vars with
yaml.safe_load(VARS_PATH.read_text()) and spelled "vars" / "main.yml"
out by hand. test_no_uncached_file_reads forbids a direct .read_text
and test_file_spot requires the ROLE_FILE_* constants, so the file now
loads through utils.cache.yaml.load_yaml_any and joins
ROLE_FILE_VARS_MAIN. What the test asserts is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 14:29:37 +02:00
kevinveenbirkenbachandClaude Opus 5 0d1db79d3d chore(dashboard): move to port-ui 2.1.2 now that its image is published
The bump waited for the image: until port-ui's CI was fixed, ghcr held
tags only up to 2.0.0, and a pin to a missing tag fails every deploy at
the pull. ghcr now answers an anonymous manifest request for
port-ui:2.1.2 with 200 and digest sha256:db3fb9c8, while 2.0.0 answers
200 and a nonexistent 9.9.9 answers 404. A local dashboard has run that
exact digest healthy for hours.

2.1.x keeps what core relies on: / renders without a redirect so the
OIDC return stays valid, no TRUSTED_HOSTS means no host check as
before, the Login/Account/Logout entries oidc.js looks for stay
untranslated, the new inline script is covered by the existing CSP, and
iframes still accept http for onion origins.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 14:16:55 +02:00
kevinveenbirkenbachandClaude Opus 5 ee3c485776 fix(snipe-it): mint as apache and hand Passport's keys to it so the API can verify the MCP token
Run 34421944328 Snipe-IT#1 (swarm) failed its CLI contract with
"UNREACHABLE read call snipeit_get_license_summary: the provider
answered 500". The mint ran provision_mcp_identity.php through
container exec as root, so the first deploy's passport:keys wrote the
key files root-owned (0660/0600), and the NFS export's No_Root_Squash
kept them that way. Apache serves as apache and could not read the
public key, so every token-checked request failed with 500 on every
replica. The image's startup script chowns the keys to apache:root,
but only when a container starts, and the mint ran after that.

mcp_mint.yml now runs the same chown in the resolved container before
minting, which also heals keys an earlier deploy left root-owned, and
runs the mint itself with -u apache so keys it creates start out
readable.

The PHP script checks the token it just minted against its own API and
exits 1 on anything but 200, so a broken mint fails the task that made
it, within its retries, instead of surfacing as an unreachable provider
in the CLI contract. Each retry revokes the same-name token before
minting, so retries do not pile up live tokens.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 14:04:13 +02:00
kevinveenbirkenbachandClaude Opus 5 c88ab1f518 fix(zammad): address the MCP adapter's upstream by service key so swarm resolves it
Run 34421944328 Zammad#1 (swarm) failed its CLI contract with
"UNREACHABLE read call zammad_list_organizations: the provider answered
nothing". The adapter's upstream_url named nginx by its container_name,
zammad-nginx, which only compose registers; the templates set that
field through compose_only and swarm drops it. The daemon journal shows
the A and AAAA lookups for zammad-nginx failing exactly across the CLI
window, and nginx logged no request from the adapter's network.

ZAMMAD_MCP_UPSTREAM_HOST now takes ZAMMAD_NGINX_SERVICE, the service
key that resolves in both modes and the same way env.j2 already reaches
railsserver, websocket and elasticsearch. Only admitted MCP clients
(flowise, hermes, n8n, openclaw, openwebui) join the shared zammad
network and none of them runs a service keyed nginx, so the short name
is unambiguous there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 13:59:00 +02:00
kevinveenbirkenbachandClaude Opus 5 eda6b25329 fix(pihole): pass the admin password through a dotenv-quoted env file and judge the denial before the callback URL
Run 34421944328 pihole#1 (README replay, sso off) never left
/admin/login: the native login was refused. The compose template put
the administrator password inline under environment:, and compose
interpolates every $name in it, so the container held the inventory
password with each $name removed (its sha256 prefix matches that
mangled value, not the real one). The environment moves into
templates/env.j2, which has_env wires as env_file, with the password
through dotenv_quote like every other role. Rendering the real template
with a password containing $name, ${...} and $$ and reading it back from a
compose v5.5.1 container returns it unchanged, while an unquoted RAW=S$x
control line in the same file comes back as "S". TZ now comes from the
shared sys-svc-container include, so PIHOLE_TIMEZONE goes.

pihole#0 (onion) timed out on "still handing off": oauth2-proxy serves
its 403 page for a user outside the allowed group on /oauth2/callback,
and the poll checked the callback URL before the page text. The CI
trace holds "You do not have permission to access this resource." in
that page's collapsed More Info block, which textContent reads. The
poll now judges the denial text first, then the Keycloak auth URL, and
only then treats the callback URL as a handoff in progress. The loose
"403" and "forbidden" matches go, since a reached admin page can
contain either.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 13:47:22 +02:00
kevinveenbirkenbachandClaude Opus 5 c2120395e7 fix(litellm): let the database follow the inventory and keep the network a dedicated one is reached on
Run 34421944328 web-app-litellm#1 (README host-mode replay) answered
every health probe on llm.ai.infinito.test with 502: litellm
crash-looped on "P1000: Authentication failed against database server
at postgres" while postgres logged 'Role "litellm" does not exist'. The
README inventory has no svc-db-postgres group, so sys-svc-rdbms skipped
the central provisioning, yet svc-ai-litellm pinned postgres.shared to
true and kept consuming a shared database nobody had created its role in.

postgres.shared now reads "{{ 'svc-db-postgres' in group_names }}":
an inventory with the central group keeps the shared database, the
README inventory gets a dedicated one.

That dedicated instance, rendered by sys-svc-rdbms postgres.yml.j2,
joins only the compose default network, and the networks renderer
dropped default from every svc-db-*/svc-ai-* role. A local variant-1
deploy measured litellm-database on litellm_default alone and litellm on
the shared litellm network alone. _suppress_default now keeps default
when the role's database is local. The variants pin shared to true (0)
and false (1) so the matrix exercises both paths, and the generated
READMEs show the postgres relation as 0..1.

The three attachment helpers move from render.py to attachments.py,
which render.py re-exports, and test_render.py splits along the same
seams into five files plus _render_helpers.py with every test id kept.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 13:32:36 +02:00
kevinveenbirkenbachandClaude Opus 5 c215aca058 fix(nextcloud): route sociallogin through the tor proxy, fail fast on a refused login callback, stop leaking plugin config
Run 34421944328 Nextcloud#1 timed out "waiting for a signed-in Nextcloud
shell after the login redirect". 4164f37981 turned sso on in variant 1
while ldap stays off there, so that round logs in through sociallogin
instead of oidc_login - the first variant, on this branch or on main,
that runs sociallogin against an onion issuer. oidc_login goes through
Nextcloud's HTTP client, which honours the proxy in proxy.config.php;
sociallogin builds its own client and reads only
social_login_http_client['proxy'], so its token exchange never reached
the onion Keycloak. The template now sets the same tor SOCKS proxy for
it, inside the existing "sso on and the OIDC host is an onion" branch.

The spec only treated 5xx documents as a page failure, so a refused
callback surfaced as a 60s (x5 over tor) wait for a shell that could not
come. A 4xx on /apps/sociallogin/ or /apps/oidc_login/ now counts as
the page itself failing, reported without the query so no code leaks.

The plugin loop printed every item, including the OIDC client secret,
into the include line of a public CI log; it now labels by plugin key and
honours MASK_CREDENTIALS_IN_LOGS.

_shared.js grew past 250 lines; its page and modal helpers move to
_page.js and _shared.js re-exports them, with an identical export
surface.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 13:21:39 +02:00
kevinveenbirkenbachandClaude Opus 5 293c67780d fix(bluesky): register compose handlers before the pre-compose staging and move the cfg heal into a script
Run 34421944328 Bluesky#1 stopped at "The requested handler 'compose-up'
was not found in either the main handlers list nor in the listening
handlers list", right after cloning the social-app build context. The
pre-compose staging (01_social_app.yml, 02_login_broker.yml) and the PDS
routine (03_pds.yml) notify compose-up and compose-build, but the
handlers only exist once sys-stk-backend loads sys-svc-compose, which
happens later in the role.

00_core.yml now includes sys-svc-compose with tasks_from: none.yml and
handlers_from: main right after the once flag, the same handler-only
registration sys-stk-backend and sys-svc-rdbms dedicated.yml already use.
The notifications still run at the stack's own flush, after the compose
files exist; the role has no earlier flush. A handler-order test pins
the registration ahead of the pre-compose block.

The oauth2-proxy cfg heal leaves its inline shell for
files/shell/heal_sso_proxy_cfg.sh, which receives the cfg path and the
container name through the environment. The blanket
"container stop ... || true" becomes a precondition: the container is
stopped only when container inspect finds it, so a directory is still
removed when the container is gone, and a failing stop of an existing
container now fails the task instead of vanishing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 13:13:36 +02:00
kevinveenbirkenbachandClaude Opus 5 9d0f618e23 refactor(sso): compute the app logout target once in the sso lookup
The previous commit wrote the same proxy-aware logout expression into
four javascript.js.j2 templates. The rule it encodes - sign out of
oauth2-proxy first when the app is proxy-gated, else go straight to the
provider - belongs where proxy-gating is already decided.

utils/roles/applications/services/sso.py gains logout_url(url,
is_proxy_gated) next to get_sso_config, with OAUTH2_SIGN_OUT_PATH as the
path constant; the rd value is quoted exactly as Jinja's urlencode
filter does it. The sso lookup exposes it as the logout_url term: it
renders OIDC through the templar, takes is_proxy_gated from the same
resolution, and refuses a context without OIDC. Prometheus, Checkmk,
Baserow and Jitsi now render lookup('sso', application_id, 'logout_url').

Behaviour is unchanged: the resolver tests pin the rd encoding against
jinja2's do_urlencode, and the lookup tests cover a proxy-gated app, an
OIDC-native app, a disabled SSO and a missing OIDC.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 13:06:19 +02:00
kevinveenbirkenbachandClaude Opus 5 a69f169600 fix(logout): sign proxy-gated apps out of oauth2-proxy before Keycloak
Run 34421944328 Fediwall#1: Prometheus, Checkmk, Baserow and Jitsi inject
their own logout control, and it pointed straight at
OIDC.CLIENT.LOGOUT_URL. For an app behind oauth2-proxy that ends the
Keycloak session but leaves the proxy's own session cookie on the app,
so the app stays signed in.

When lookup('sso', application_id, 'is_proxy_gated') holds, the control
now goes through /oauth2/sign_out?rd=<url-encoded Keycloak logout>: the
proxy clears its cookie, then redirects to Keycloak. oauth2-proxy only
follows rd into its whitelist_domains, which carry the primary domain
and the node onion, so the redirect is allowed in clearnet and in tor
mode; the pihole spec already drives the same sign-out path. Apps
without the proxy keep the direct URL, and to_json quotes the value as a
JS string literal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 13:03:59 +02:00
kevinveenbirkenbachandClaude Opus 5 f142f637f5 fix(onion): mirror CSS asset origins and emit plaintext onion CSP siblings
Run 34421944328 odoo#0 loaded assets over the node onion that the page
could not use.

build_csp_header mirrors every clearnet token that carries the primary
domain onto the node onion, but kept its scheme. Onion vhosts are served
in plaintext, so a sibling like https://cdn.<onion> never matched the
http://cdn.<onion> request it was meant to allow. Onion siblings now map
https to http and wss to ws; the clearnet token is unchanged.

The body filter rewrote external asset origins to the onion mirror only
in HTML, so stylesheets kept pulling https://<origin> directly. When an
app has tor enabled and external asset origins, the header filter now
also flags text/css responses on .onion hosts (dropping Content-Length,
since the body changes), and the body filter runs those through the same
buffer and mirror rewrite and returns before the HTML injections. The
flag is set under the same Jinja condition that renders the mirror
block, so a flagged response always has a rewrite to run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 12:24:03 +02:00
kevinveenbirkenbachandClaude Opus 5 3db5820b6e fix(n8n): serve the MCP trigger over Streamable HTTP on 1.100.1 so Open WebUI can connect
Run 34421944328 n8n#1: Open WebUI 0.11.0 ships only a Streamable HTTP
MCP client. n8n 1.95.3's MCP Server Trigger speaks SSE only and answered
the client's POST to /webhook/infinito-mcp/sse with 404, so "every
registered MCP server answers Open WebUI's client" failed.

n8n moves to 1.100.1 and the managed trigger to typeVersion 2, which
serves Streamable HTTP at /mcp/<path>. meta/mcp.yml declares
transport: streamable_http at /mcp/infinito-mcp (the trigger path stays
infinito-mcp) and points source_url at the 1.100.1 tree. Open WebUI's
supported_transports drops sse, which its client never spoke; no
provider declares sse any more, so no pairing is lost.

n8n#0: the guest spec probed ${APP_BASE_URL}${MCP_ENDPOINT_PATH} and
expected a refusal of >= 300 without an event stream. The trigger is
exposure: internal, so the public vhost must answer 404; the spec now
asserts exactly that through mcpEndpointUrl(..., N8N_BASE_URL), and the
bearer guard itself is proven in-cluster by the CLI MCP contract.

Requirement 035 and the MCP audit table follow the version and
transport.

Upgrade path, read from upstream at both tags: the postgres migrations
are identical (45/45) and the only new common migration is
AddProjectDescriptionColumn, an additive nullable column; BREAKING-
CHANGES.md lists only 1.98.0 in range (route metric format, Form node
tag attributes, Node >= 20), none of which this deployment uses. A fresh
1.100.1 install ran all 89 migrations and passed the MCP bridge, server,
non-member and guest specs locally.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 12:20:46 +02:00
kevinveenbirkenbachandClaude Opus 5 a24fcb3950 fix(test-e2e-playwright): assert the MCP probe URL is absolute, not merely truthy
registerMcpDisabledState and registerMcpGuestRejection opened with
expect(endpointUrl).toBeTruthy(). normalizeBaseUrl turns an unset base
into "", so mcpEndpointUrl() still returns "/mcp", which is truthy, and
Playwright resolves the relative path against baseURL (APP_BASE_URL or
http://127.0.0.1). A role whose own base URL variable was missing
therefore probed some other host and could pass.

Both probes now require /^https?:\/\/[^/]+/. Every caller builds its URL
from a base that lookup('tls', ..., 'url.base') renders as
"<scheme>://<domain>/", so the check holds in tor mode
(http://<app>.<onion>.onion/) and in clearnet mode (https:// or http://
with TLS off) alike, and fails only when the base is unset.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 12:10:22 +02:00
kevinveenbirkenbachandClaude Opus 5 303c47fe2a fix(test-e2e-playwright): retry transient onion transport errors for every API verb, not only GET
0a52433dcb gave the standalone request fixture a Tor-resilient GET,
because the bundled socks client caps the CONNECT at 30s and a cold
onion circuit fails a request regardless of its timeout. POSTs and calls
made through context.request still went out once, and a circuit Tor
rejected ("Socks5 proxy rejected connection") was not recognised as
transient at all.

apiFetchOnion now takes the verb from opts.method and calls
request.fetch, so it serves both APIRequestContexts; apiGetOnion becomes
its GET shorthand. The rejection joins the transient patterns. Retries
stay onion-only and transport-only: clearnet keeps a single attempt and
HTTP or other errors re-throw at once.

The Mattermost LiteLLM spec routes its login, users/me, users/username
and channels/direct calls through it. The /posts call that sends the
prompt stays a single attempt, because a replayed post would message the
bot twice. The Hermes administrator spec reads /v1/models through
apiGetOnion.

env.test.js pins a POST surviving a rejected circuit, the single
clearnet attempt, no retry of a non-transport error over Tor, and the
GET shorthand.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 12:05:02 +02:00
kevinveenbirkenbachandClaude Opus 5 4f44eff128 fix(svc-net-tor): bind dnsmasq statically so a failed re-enumeration cannot drop every listener
Run 34421944328 Nextcloud#13 lost the dashboard's Matomo tracker. The
dns53 sampler on mgr-01 shows no :53 socket from 01:39:52 to 01:48:44
while the dnsmasq PID stayed the same, six seconds after a short-lived
container's veth on docker0. In that window the manager could not
resolve its own onion vhosts, so sys-front-inj-matomo's site checks for
web-svc-logout (01:45:35) and web-app-dashboard (01:47:18) never reached
OpenResty, the injection was disabled as "API not ready", and the
Playwright check found no matomo.js. The other four swarm managers of
the run that carried the same listener kept it throughout.

Under bind-dynamic, dnsmasq 2.91 re-enumerates interfaces on every
netlink address event and then garbage-collects every listener it did
not re-find (network.c enumerate_interfaces), also when the enumeration
failed; release_listener has no other caller. 5aa7506e36 reproduced this
on the lab nodes and 11c98825aa already moved the bridge-less case to a
static bind. The bridge case now binds statically as well.

This supersedes two earlier rationales, both measured on the local stack:
- bb0e098a9f kept bind-dynamic because dnsmasq starts at boot before
  docker0 exists. A unit drop-in (files/dnsmasq-restart.conf,
  Restart=on-failure, RestartSec=5) now retries the start: with a missing
  listen-address the start exits 2, and dnsmasq came up 4s after the
  address appeared.
- 4739b2de99 called bind-dynamic load-bearing because docker0 can
  disappear across a docker restart. Only a start without the address is
  fatal; a bound socket survived the address being removed and re-added
  and the interface being recreated with a new ifindex, answering on the
  same socket inode each time.

The docker0 fact is re-gathered before torrc and the dnsmasq drop-in are
rendered. The play-start fact predates default-address-pools moving the
bridge (285ba1e76c), and a static bind on a stale address would make
dnsmasq exit. container_dns reads only the pool, so daemon.json is not
affected by the refresh.

The host resolv.conf moves from an inline copy content to
templates/resolv.conf.j2; both it and the drop-in render byte-identical
to what the local stack already runs.

A veth churn of 13.8k cycles did not reproduce the loss locally, under
either bind mode.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 12:01:51 +02:00
kevinveenbirkenbachandClaude Opus 5 304144ce67 fix(matrix): give each bridge declared tokens and a rendered registration, drop the swarm service removal
Every mautrix bridge generated its own registration with -g on start,
with fresh random tokens. Synapse had to read that file from a shared
directory, be restarted afterwards, and on swarm the services had to be
removed first so the registrations could be wiped (93735fd533). The
tokens changed on every redeploy and the deploy depended on a wait for
files a container writes.

Each bridge now has as_token and hs_token declared in meta/secrets.yml
and a fixed appservice_port in its addon. The bridge config reads the
tokens through lookup('config'), and registration.yaml.j2 renders the
registration from that config in the upstream -g format, so both sides
are known before anything starts and stay identical across redeploys.

The files reach the containers as secrets: matrix_bridge_mounts offers a
stack view (top-level definitions, content-hash named by compose_volumes),
a synapse view (one registration per bridge under /registrations, mode
0444) and a bridge view (the config at /config/config.yaml). The bridges
run as the synapse uid with an explicit --config and --no-update, keep
their scratch /data on a 16m tmpfs so hlth-volumes finds no anonymous
volume, and depend on a healthy synapse. Their healthcheck is
/_matrix/mau/ready, which the bridge only reports after whoami with the
as_token and an appservice ping carrying the hs_token, so a healthy
bridge proves the token pair.

The compose tasks lose the compose down, the swarm service rm block, the
registration wait, the permission fixes and both synapse restarts; this
reverts the workaround of 93735fd533.

Verified on the local compose stack: four bridges ready, synapse pings
answered, registration and config sha256 unchanged across a redeploy,
test-bridge-roster and 12 matrix Playwright specs green. A token
rotation (new pair in both files, compose up --force-recreate as the
handler runs it) came back healthy; a mismatched pair left the bridge
unhealthy. The swarm-shaped stack file passes docker stack config with
the 0444 secret modes, the 16 MiB tmpfs and content-hash secret names.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 11:43:16 +02:00
kevinveenbirkenbachandClaude Opus 5 a650801b35 fix(lookups): name file-backed extra secrets by content and hand plain types to PyYAML
compose_volumes copied extra_configs and extra_secrets through verbatim.
A file-backed extra without a name therefore reached swarm under a fixed
name, and swarm secrets are immutable: a new content under the same name
is rejected on the next stack deploy. Such extras now get the same
{role}_{key}_{sha8(content)} name that _config_secret_name already gives
meta-declared entries. An explicit name is kept.

container_volumes appended extras as handed over. Ansible passes one
lookup's result to another as proxy containers, and a TaggedInt mode
inside a proxy mapping made PyYAML raise "cannot represent an object".
The private _to_plain of compose_volumes moves to utils/templating as
to_plain, now coercing bool, int and float to the exact builtin types
(bool before int, since bool subclasses int), and container_volumes runs
its extras through it.

Only web-app-matrix passes extra secrets or configs today; the
extra_volumes path that web-app-nextcloud uses is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 11:20:26 +02:00
kevinveenbirkenbachandClaude Opus 5 a3b0606791 fix(mastodon): emit http and ws URLs when TLS is off, not only skip force_ssl
Mastodon 4.6.2 derives https = Rails.env.production? in 1_hosts.rb.
The HTTP-only override disabled force_ssl and the route defaults, but
config.x.use_https, the streaming base URL and the ActionMailer
default_url_options kept https and wss. RoutingHelper#default_url_options
delegates to the ActionMailer hash, so an onion deployment still handed
out https avatar URLs and a wss streaming endpoint that nothing serves.

The override now sets use_https to false, points streaming at
ws://<web_domain> and merges protocol http:// into the mailer defaults
once ActionMailer loads. The block stays gated on
MASTODON_HTTP_ONLY_ENABLED, which is true only when TLS is disabled for
the role, so HTTPS deployments are untouched.

Measured on the mastodon v4.6.2 image: without the override use_https is
true and streaming is wss; with it both switch to http and ws.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 11:17:41 +02:00
kevinveenbirkenbachandClaude Opus 5 b582846ab3 fix(test-e2e-playwright): make the Keycloak redirect and login-control checks able to fail
performKeycloakLogin polled the page URL with toContain(canonicalDomain).
On the Keycloak auth page the app domain already sits in redirect_uri,
so the poll was satisfied before the login had completed. The denial
path and the biber/admin iframe checks used the same substring test.
All of them now compare hostnameOf(url) with the canonical domain, which
is a bare hostname (env.js reads it with /\.onion$/).

LOGIN_CONTROL_NAME never matched a login link that carries an
icon-font glyph: Playwright's accessible name includes the private-use
character, and the pattern only allowed whitespace before the text. The
post-login waitFor({state: "hidden"}) therefore passed at once, even
before oidc.js had swapped the link. The pattern now admits \p{Co}
before the text, so that wait really waits for the swap.

dom.test.js pins both: glyph-prefixed login controls match while Logout
and Accessories do not, an IdP URL carrying the app domain in
redirect_uri is rejected, a look-alike suffix is rejected, and
about:blank yields an empty host.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 11:15:10 +02:00
evangelostsak 48e230d87f docs(requirements): cross-link the OpenBao requirement to its PR
Closes the last cross-link acceptance criterion: the requirement now names
infinito-nexus/core#611 in its header and in the criterion itself, and the PR
description links back to this document. Also deep-links the upstream story
instead of pointing at the OpenProject root.
2026-09-11 00:15:35 +02:00
kevinveenbirkenbachandClaude Opus 5 214e7e29d1 fix(test-e2e-playwright): scale onion timeouts by the tor flavor instead of a flat five
resolveTimeout multiplies every Playwright window by 5 for a .onion
target. That default arrived in 35dee215c8 (2026-07-08), before
5399b48482 (2026-07-24) gave CI the chutney flavor, a private Tor net on
localhost, and PLAYWRIGHT_ONION_TIMEOUT_MULTIPLIER was never set. Waits
that expire by design therefore cost five times their base under chutney
too, which is where most of the Tor surcharge came from.

The Playwright run now passes the multiplier for onion targets only,
looked up from svc-net-tor's services.tor.flavor: 2 for chutney, 5 for
the public network, whose circuits really are slow. An unknown flavor
fails the lookup instead of falling back, and a unit test holds every
flavor directory of svc-net-tor to an entry.

Chutney traces of run 34421944328: 700 successful timed waits across 186
tests used at most 0.42 of their unscaled window, and the longest
client-side expect took 5.1s. Across runs 34421944328 and 34419507038 no
chutney test with its own ceiling (401 of them) used more than 53% of
the new x2 ceiling, even at today's x5-stretched duration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 15:53:08 +02:00
kevinveenbirkenbachandClaude Opus 5 dcb9ddb9cf fix(playwright): stop stretching waits on server-side clocks by the onion multiplier
The matrix DM spec sleeps 120s before the administrator signs in and 30s
between the two sign-ins, and the rate-limit dialog handler cools down
for 45s, all so Synapse's rc_login bucket (per_second 0.17 in the
compose flavor) can refill. That refill runs on Synapse's clock, yet the
sleeps went through resolveTimeout and became 600s, 150s and 225s over
Tor. They now use fixed constants with the clearnet values, which pass
there as they are. The 5s pause before the marker is sent stays scaled:
Element shares its megolm session through the homeserver, so that one
does depend on the network.

Mailu's two poll intervals (500ms for a visible element, 3s for an inbox
refresh) were scaled as well, so an arrived mail was noticed up to 15s
later over Tor. They are fixed now; the deadlines still scale. Named
constants are what the raw-timeout lints reserve for intervals that must
not scale.

Across runs 34421944328 and 34419507038 the matrix DM test took 960s
over Tor against 210s on clearnet; the two refill sleeps alone account
for 600s of that difference.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 15:51:29 +02:00
kevinveenbirkenbachandClaude Opus 5 92ccfa2570 fix(test-e2e-playwright): stop the logout helpers waiting out windows whose answer is already known
confirmKeycloakLogoutIfPrompted waited the full 2s times the onion
multiplier for Keycloak's confirmation button even when Keycloak had
already redirected away because the end-session request carried an
id_token_hint. It now races the button against the page leaving the
logout endpoint (waitUntil: commit) and still clicks the button when
Keycloak renders the prompt. The dashboard's own confirmLogoutIfNeeded
stops the same way, and its poll interval no longer scales, so a slow
target is noticed sooner rather than later.

inAppLogout's first candidate poll ignored a visible settings link, so a
surface without a recognisable logout control or menu trigger, such as
the Mastodon home timeline, burned 30s times the multiplier before the
existing settings fallback ran. A visible settings link now ends that
poll as well; logout controls and menu triggers are still checked first
in every round. The settings selector moves to settingsLinksOn so both
callers share it.

Traces of run 34421944328: the dashboard's confirm probe expired after
2.03s on clearnet and 10.07s over Tor, and Mastodon's biber logout
polled 165s over Tor before reaching the settings page. Replayed in
Chromium, a Mastodon-shaped surface logs out in 3.24s instead of 33.45s
and a redirecting Keycloak logout returns in 0.20s instead of 2.00s,
while a prompting one is still confirmed. In the traced persona logouts
of six artifacts every other poll found a candidate in its first round,
so their path does not change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 15:49:10 +02:00
kevinveenbirkenbachandClaude Opus 5 0b79afdc0c fix(test-e2e-playwright): stop the OIDC login helper waiting out probes that cannot match
Every dashboard persona login spent 25s times the onion multiplier in
clickOidcLoginLink before it clicked anything. The markup probe matched
a[href*='openid-connect'], and the dashboard header renders the Account
dropdown's hidden Logout item, whose Keycloak end-session URL contains
the same string, before its Login link. .first() pinned the hidden item
until the 5s probe expired; the exact-name probe then missed for 20s
because Playwright counts the Font Awesome ::before glyph in the link's
accessible name. The probe now skips logout hrefs and resolves to the
Login anchor the loose probe used to click.

The adapter readiness wait cost every other role 15s times the
multiplier, because only the dashboard sets __oidcLoginReady. The helper
now waits for the load event and then only while the flag is false. The
dashboard loader, an inline head script, declares it false before the
Login link renders; oidc.js still sets it true once the click handler
is wired.

Traces of run 34421944328 show the two probes expiring at 5.07s + 20.00s
on clearnet and 25.01s + 100.00s over Tor. Across runs 34421944328 and
34419507038 the Tor dashboard persona test took 150-156s in all 102
compose runs against 32-34s in the 92 clearnet ones.
Replayed on the dashboard HTML those traces captured, the helper reaches
the Keycloak authorize endpoint in 0.08s instead of 40.09s, and still
waits for the adapter (1.52s) when the flag flips to true after 1.5s.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 15:46:32 +02:00
evangelostsak c9f48fca67 Merge remote-tracking branch 'origin/main' into feature/web-app-openbao 2026-09-10 14:35:40 +02:00
kevinveenbirkenbachandClaude Opus 5 60b9006506 test(mcp): fail the contract on a JSON-RPC error inside an HTTP 200
Over streamable_http the contract probe judged initialize, tools/list and the
read call by HTTP status alone. A server that answers 200 with a JSON-RPC
`error` object therefore passed initialize and the read call, and failed
tools/list only as "server serves []", which names the wrong cause. The SSE
path checked initialize and the read call, but not tools/list. mcp-adapter,
which fronts WordPress, maps invalid params to 200, and Moodle's
webservice_mcp answers a failing tool call with 200 and code -32603, so a read
that returned nothing could turn the check green.

rpc_error() extracts the error object of a JSON-RPC body, refused() is built
on it, and reject_error() fails initialize, tools/list and the read call on
both transports with the server's own message. A 200 whose body is not JSON
still carries no error object, so a provider that has not started keeps the
caller retrying as before.

The streamable_http initialize, and the one in the negative probes, sent no
clientInfo, which the MCP SDK requires. Home Assistant (mcp 1.26.0, stateless)
answers such an initialize with 200 and -32602 and serves the rest anyway, so
only the old status check let it pass. All three initialize calls now share one
INITIALIZE with clientInfo, as the SSE path already sent. Against a Home
Assistant-shaped endpoint on mcp 1.26.0 the SDK logs "InitializeRequest.params.
clientInfo Field required" for the old probe, and the fixed one prints OK.

Against HEAD, an error-on-200 at initialize and at the read call printed OK;
now all three steps print REJECTED naming the error. The MCP adapter in front
of sixteen of the nineteen streamable_http providers answers 200 only with a
result (svc-ai-mcp-adapter/files/python/server.py), so none of them changes
outcome.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 12:47:15 +02:00
kevinveenbirkenbachandClaude Opus 5 9f95127b72 fix(wordpress): serve exactly the reviewed MCP abilities on WordPress 7.1
The CLI contract test rejected WordPress in job 102705652296 of run
34421944328:

    REJECTED server serves ['mcp-adapter-discover-abilities', 'mcp-adapter-execute-ability', 'mcp-adapter-get-ability-info'] but the contract declares ['infinito-get-post', 'infinito-list-categories', 'infinito-search-posts']

There are two layers. The mcp-adapter v0.5.0 default server lists only its
three meta-tools and reaches public abilities through discover and execute; it
takes its tool list from the mcp_adapter_default_server_config filter
(includes/Servers/DefaultServerFactory.php). And the three infinito abilities
were never registered at all: the mu-plugin hooked abilities_api_init and
passed no category, while WordPress 7.1.0 core fires only
wp_abilities_api_init, refuses wp_register_ability() outside it and requires a
category (src/wp-includes/abilities-api.php:290-297,
abilities-api/class-wp-ability.php:295-297).

The mu-plugin now registers an `infinito` category on
wp_abilities_api_categories_init, registers the abilities on
wp_abilities_api_init with that category, and sets the default server's tools
to exactly the three abilities, with empty resources and prompts, at
PHP_INT_MAX. With the meta-tools gone, execute-ability no longer reaches every
public ability any other plugin registers.

The abilities return objects ({posts}, {categories}, {post}), since MCP defines
structuredContent as a JSON object. get-post now serves published posts only,
as search-posts already did; before, it returned any published type, including
customize_changeset and wp_block entries.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 12:35:05 +02:00
kevinveenbirkenbachandClaude Opus 5 93735fd533 fix(matrix): recreate synapse and the bridges together on a swarm redeploy
Every deploy of the compose flavour wipes /opt/compose/matrix/mautrix/ and
re-renders each bridge config with placeholder tokens, so the bridges generate
new registrations on their next start and synapse has to load them. Compose
gets that ordering from the `compose down` right before the wipe. Swarm had no
counterpart: the bridge services kept running on the deleted bind-mount
directories and never regenerated anything, while synapse was restarted by the
flush and found no registration.

Job 102705652096 of run 34421944328 (swarm, variant 2) hits it on the
credential-rotation redeploy, which changes homeserver.yaml, a swarm config,
and therefore restarts synapse every time:

    FileNotFoundError: [Errno 2] No such file or directory: '/data/mautrix-meta/registration.yaml'
    stuck: matrix_synapse

From 05:44:56 dockerd reported "current working directory is outside of
container mount namespace root" for the running slack and signal bridges, and
mautrix-meta, started at 03:50:50, only initialised again at 05:53:11, when its
healthcheck finally replaced it; stack_ready.sh had given up at its 180s grace
long before.

Before the wipe, swarm mode now removes the synapse and bridge services a
previous deploy left running, once from the stack host, so the flush's
`docker stack deploy` creates them together, exactly as on a first deploy
(which converged in about 32s in the same job). Removing only the bridges would
not be enough: a synapse that is not recreated keeps the old tokens, and the
new bridges would be refused before the synapse restart at the end of the file.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 12:32:27 +02:00
kevinveenbirkenbachandClaude Opus 5 347904c04a test(mattermost): keep polling for the bot reply through a transport error
firstBotReply already treats an HTTP error as "no reply yet" and returns an
empty string, but a transport failure throws out of context.request.get, and
expect.poll does not catch what its callback throws, so the attempt ends
there instead of at the poll's deadline. Over tor that is routine. Job
102705652088 of run 34421944328 lost three retries to it, each thrown at
test-litellm-agent.js:54 from inside the poll (:142):

    TimeoutError: apiRequestContext.get: Timeout 30000ms exceeded.   after 6.2 of 15 minutes
    Error: apiRequestContext.get: Socks5 proxy rejected connection - Failure   twice

A failed request now counts like a failed response: the poll asks again two
seconds later and only its deadline decides. A reply is still required, so a
transport that never recovers still fails the test, with the poll's message
rather than the last transport error.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 11:08:26 +02:00
kevinveenbirkenbachandClaude Opus 5 37501aaaa1 fix(mattermost): carry the merged Agents config to every swarm replica
The role merges the gateway service and bot into the stored Agents config and
then cycles the plugin so it adopts them, but agents/reload.yml and
agents/enable.yml both run mmctl in one container (run_once, a single cid).
mattermost-team-edition has no cluster bus, so no other replica hears of it.

Job 102705652088 of run 34421944328 (swarm, three replicas) shows exactly
that in the service log:

    02:43:51-52  "Plugin activated" mattermost-ai on .1, .2 and .3
    02:44:45     Point the Agents bot at the platform gateway (the merge)
    02:44:48     Deactivate the Agents plugin so it drops its cached config
    02:44:51     "Plugin activated" mattermost-ai on .1 only

Every post the plugin handled in the test's direct-message channel then
landed on .2 or .3, which still ran the config from 02:43 ("Notification
modified by plugin." at 03:10:30 and 03:25:32 on .2, at 03:35:42, 03:50:42
and 03:52:55 on .3), and the bot never answered:

    Timeout 900000ms exceeded while waiting on the predicate ... Received: 0

After the reload the role now forces a rolling update of the service, so
every replica restarts and activates the plugin against the merged config,
and wait_converged.yml blocks until the update has completed and every task
is running again, before the stored config is re-read and asserted. Compose
runs one container, which the reload already covers, so both steps are swarm
only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 11:05:24 +02:00
kevinveenbirkenbachandClaude Opus 5 ad7913d41a fix(decidim): create the organization that db:seed skips in production
Every task after the seed step reads `Decidim::Organization.first`
(ensure/default_pages.rb, set_organization_host.rb, ensure/admin_user.rb,
configure_oidc.rb), and only `rails db:seed` ever creates one. Decidim guards
its core seeds with `if !Rails.env.production? || ENV.fetch("SEED", nil)`
(decidim-core db/seeds.rb, v0.31.6) and this role sets no SEED, so in
production the seed step creates nothing, and `failed_when: false` hides it.
The next task then calls `CreateDefaultPages.call(nil)`, whose
`StaticPage.find_or_create_by!(organization:, slug:)` raises, and
set_organization_host.rb would follow with `nil.host =`.

The regular matrix deploys RAILS_ENV=development, where the seeds run, so
only the README replay step, which provisions a production inventory, gets
here. It has not got this far yet: the healthcheck stopped it first
(87ce146d8).

ensure/organization.rb creates the organization when there is none, with the
attributes Decidim's two creation paths agree on (the core seeds and
Decidim::System::CreateOrganization), plus the help pages and content blocks
CreateOrganization adds. It leaves out the admin invitation, because
ensure/admin_user.rb creates the administrator itself, and tos_version,
which the terms-of-service page sets when the next task creates it. Name and
reference prefix come from DECIDIM_ORGANIZATION_NAME, the host from the same
domain the other tasks use.

The task runs only where the seeds skip, with ENVIRONMENT == 'production'. In
development nothing changes, and a seed that fails there still surfaces at the
next task instead of being covered by a bare organization.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:53:55 +02:00
kevinveenbirkenbachandClaude Opus 5 87ce146d8d fix(decidim): probe with the host Rails will answer for
The healthcheck curls http://127.0.0.1:3000/up, and the role's own
initializer, templates/allowed_hosts.rb.j2, appends the public domain to
`config.hosts`. What that append does depends on RAILS_ENV (Rails 7.2.2.2,
railties/lib/rails/application/configuration.rb:39-42 and
default_middleware_stack.rb:16-17):

- development starts from `.localhost`, `.test`, 0.0.0.0/0 and ::/0, so
  loopback stays allowed and the append changes nothing;
- production starts from an empty list, so the append switches host
  authorization on with the domain as its only entry, and the probe gets 403.

Job 102705651823 of run 34421944328 shows both. Its regular deploy runs
RAILS_ENV=development and is healthy with no blocked request. Its nested
instructions deploy runs RAILS_ENV=production and never becomes healthy; the
first probe that reaches Rails is already refused:

    E, [2026-09-10T03:47:10.424204 #1] ERROR -- : [ActionDispatch::HostAuthorization::DefaultResponseApp] Blocked hosts: 127.0.0.1:3000
    curl: (22) The requested URL returned error: 403

118 such lines in total, until `Decidim | Wait for web container to be
healthy` (60 x 15s) gives up. The container itself is fine: migrations ran
and Puma is listening on 0.0.0.0:3000.

The curl flavor sends a Host header when `hostname` is set
(utils/docker/healthcheck/probes.py:54-55); decidim never declared one. It
now sends the same `lookup('domain', 'web-app-decidim')` that vars/main.yml
feeds into the initializer, which production accepts as its one entry and
development accepts anyway.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 10:50:15 +02:00
kevinveenbirkenbachandClaude Opus 5 c4c2f40018 fix(checkmk): drop the site tmpfs; OMD keeps its own tmp in the container
f85996b43a added this mount on the premise that "checkmk's entrypoint runs
`omd create --no-tmpfs`, so the site's tmp directory is never mounted and the
site cannot come up cleanly", and noted that checkmk had not been watched
starting with it. The premise does not hold. `--no-tmpfs` persists TMPFS=off, so
omdlib never mounts anything and keeps ~/tmp as an ordinary directory on the
site volume; upstream runs it that way inside containers on purpose, because a
tmpfs mount there would need privileges (Werk 6528).

The mount was not solving a problem, it was the problem. Without a mode, Docker
28 creates a tmpfs as 755 root, and the site user cannot mkdir in it; with one,
compose honours it but `docker stack deploy` does not, because the compose to
swarm conversion builds TmpfsOptions from SizeBytes alone
(docker/cli cli/compose/convert/volume.go:146-170). That is why the key flipped
four times in two days and why run 34421944328 still crash-looped on swarm with
`PermissionError: '/omd/sites/cmk/tmp/apache'` after the mode came back.

Measured on the pinned checkmk/check-mk-raw:2.4.0p32, one container with the
tmpfs and one without, same volume layout otherwise:

    healthy after       45s                     45s
    ~/tmp               tmpfs                   751, site user, 192K -> 188K
    login.py p50/p90    9.4 / 11.1 ms           8.1 / 10.8 ms

No difference that matters, and the directory OMD creates is exactly the one the
mount used to shadow. The image declares no VOLUME, so nothing anonymous takes
its place. Checkmk's own docs call the mount a performance option, and Werk
19700 recommends leaving it off because it breaks `omd restore`, which matters
here with `backup.no_stop_required: true`.

An entrypoint wrapper that chmods the path was tried first and rejected before
it landed: `docker stack config` refuses the `"$@"` it needs, and
sys-svc-compose-ca turns any longer `bash -c` argv into rc 127.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 09:22:11 +02:00
kevinveenbirkenbachandClaude Opus 5 98707e06f4 test(matrix): hold the CA passthrough guard to the repo's own conventions
The guard landed in 2acf91e18 breaking two rules it should have followed from
the start, and `make quality-high` caught both:

    test_ca_trust_passthrough.py:23: `parents[N]` index expression
    test_ca_trust_passthrough.py:23: `PROJECT_ROOT` assignment outside `__init__.py`
    test_ca_trust_passthrough.py:44: .read_text(...) uncached

So `PROJECT_ROOT` moves into the package `__init__.py`, where every other role
test suite keeps it, and both template reads go through `read_text` from
`utils.cache.files` so a second reader of the same file pays one open.

Verified with the flags the suite actually runs under. scripts/tests/code/run.sh
passes `--import-mode=importlib` and `consider_namespace_packages=true`, which
is what makes a hyphenated `roles/<role>/` directory importable at all; a plain
`pytest <path>` fails the relative import here and does so for the existing
sys-svc-mail-msmtp suite too, which is the control that says the pattern is
right and the invocation was wrong. Under those flags both suites pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 02:01:19 +02:00
kevinveenbirkenbachandClaude Opus 5 2acf91e186 fix(matrix): pass the CA bundle into the container synapse binds it from
The mdad stack runs its own Docker daemon inside the matrix container, so a
bind source in `matrix_synapse_container_extra_arguments` is resolved in that
container's mount namespace, not on the host. `vars.yml.j2` mounts
`CA_TRUST.bundle_host` while `services.yml.j2` passed only
`CA_TRUST.cert_host`, and those are two different files: the root CA and the
bundle beside it. The bundle therefore never existed where synapse looked.

`docker create` refuses the unit before Matrix logs anything of its own. From
the rescue artifact of job 102465251843 in run 34348821577, the nested
journal at containers/matrix/nested/.../journal.txt:5018:

    matrix-synapse[5612]: Error response from daemon: invalid mount config for
    type "bind": bind source path does not exist:
    /etc/infinito.nexus/ca/ca-bundle.crt

Five start attempts, the same line each time, and the only error in the 872 KB
journal. matrix-heisenbridge follows it down as a dependency, `systemctl
restart` returns rc=1, the mdad bootstrap fails and the rescue task ends the
play. Not an OOM (exit 1, not 137, `OOMKilled: false`, 9.7 GB free), not disk
pressure, not a failed pull, and 16 % of the job budget spent.

b5fcd535de is where the asymmetry began: it moved this mount from `cert_host`
to `bundle_host` for a good reason -- SSL_CERT_FILE and REQUESTS_CA_BUNDLE
replace the trust store rather than extend it -- and left the passthrough on
`cert_host`. So the bundle stays; what was missing is the line that carries it
in, at the same absolute path the nested daemon asks for.

The test that comes with it reads both templates as text and requires every
CA_TRUST path synapse binds to be passed through at that same path. Rendering
would need a full Ansible context, and the invariant is about what the two
files say to each other rather than about the values. It fails against HEAD,
where `bundle_host` is bound and only `cert_host` is passed. It also fails
loudly if the regex ever stops matching how the extra arguments are built,
rather than quietly guarding nothing.

The cause is proven to the byte; the effect is not observed. `docker create`
never succeeded, so a second synapse defect behind this one would still be
invisible. A local `make compose-deploy apps=web-app-matrix mode=update` with
TLS_MODE=self_signed, grepping the nested journal for "Started
matrix-synapse.service", settles both.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 01:56:17 +02:00
kevinveenbirkenbachandClaude Opus 5 8434a31611 fix(ci): answer a role-only selection by existence, not by variant None
Rows are indexed by `(name, variant)` and a variant is an int. A token that
names only a role carries no variant, so the lookup ran with `None` and could
never hit, and every such token was reported as matching no discovered row.

That is the form `collapse_to_roles` produces, so it is the form a retriggered
priority line arrives in. Run 34348821577 shows the result: all twenty-one
priority entries carried

    priority: '<role>' matches no discovered row
              (no row in this run's mode and lifecycle envelope)

while baserow, checkmk, decidim and the rest went on to deploy fine. The
warning was not wrong about one entry and right about twenty; it was blind. A
role that genuinely has no row in the envelope looked exactly like a role with a
full set, which is how priority work goes missing without anyone being able to
see which entry did.

A token that narrows nothing beyond the role name is now answered by existence:
it asks for the role in whatever axes the rotation picks, so any discovered row
satisfies it. `Pin.pinned` already draws that line - "whether the token narrows
anything at all beyond the role name" - and pinned tokens keep the old path,
`check_pins` included.

Three tests come with it. There was already one for a bare role that matches
nothing, and none for a bare role that matches something, which is why the
lookup could stay broken. One of the new ones uses variant 4 so a future
implementation cannot pass by quietly guessing variant 0. Checked against HEAD:
the same input returns the spurious warning there, so the tests fail without
this change.

This restores the signal, not the lost jobs. The other silent path, `priority`
being truncated to the job budget in cli/meta/ci/chunks.py, is untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 01:44:59 +02:00
kevinveenbirkenbachandClaude Opus 5 5603b5ca45 fix(ci): read the trace switch from the variable that exists
The workflow read `vars.INFINITO_PLAYWRIGHT_KEEP`. This repository defines two
variables, `GHCR_USERNAME` and `PLAYWRIGHT_KEEP_ALL`, and that is not either of
them. GitHub does not complain about an undefined `vars.*`; it expands to an
empty string, so `keepAll` in playwright.config.js has been false on every run
since the reference was introduced by db66e01020, and `PLAYWRIGHT_KEEP_ALL=true`
has never reached anything.

The consequence is that trace, screenshot and video all stayed at their
on-failure settings, so a passing test recorded nothing. That is why run
34348821577 offers no way to see inside its Playwright timings: the artifact for
the tor Nextcloud job holds fourteen junit files and fourteen HTML reports and
not one trace.zip. The one measurement that would explain where a 161s test
spends its time was switched off by a name.

The env key keeps its name, since the container, the passthrough allow-list and
the config all agree on it; only the source changes.

Worth knowing before the next run: this turns trace, screenshot and video to
"on" for every test. Artifacts grow well beyond the current 2.8MB per job, and
tracing is not free, so the runs that already die at the distro budget may die
sooner. The switch is now a variable, so it can be turned on for a diagnostic
run and off again without touching the workflow.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 01:22:01 +02:00
kevinveenbirkenbachandClaude Opus 5 b95d9a2f54 fix(pihole): make the login and denial checks able to fail
Three assertions in these specs passed whether or not the thing they name
happened.

The native login test navigates to `/admin/login` and then asserted that the URL
contains `/admin`, which it already did before anyone typed a password, next to
a check that `body` is visible, which is true the moment the document parses. A
failed login scored green. It now waits for the URL to leave `/login` and for
the password field it just filled to be gone.

The SSO login test asserted `page.url()` once, immediately after a settle whose
result is discarded, so whether it saw the finished redirect was luck. Its two
sibling tests in the same file poll for the Pi-hole base URL and wait out
`/oauth2/callback` first; this one never did. It does now, and it also rejects
being parked on the callback, which nothing checked before.

The denial test counted `/oauth2/callback` and the auth endpoint as evidence of
denial. Both appear mid-handoff on a login that is about to SUCCEED, so an
authorization regression could pass here unnoticed. The check becomes a poll
over three distinct states: still handing off keeps polling, back at the auth
endpoint or a forbidden body is denial, anything else fails and reports where
the browser actually landed. A transient no longer reads as a pass.

The obvious form of that fix — requiring the URL to leave `/oauth2/callback`
before judging — is avoided on purpose: oauth2-proxy may render its 403 on the
callback path itself, and the test would then time out on a correct denial.
Judging the state rather than the path does not care.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 01:14:55 +02:00
kevinveenbirkenbachandClaude Opus 5 b7ccff3278 fix(checkmk): give the site tmp a mode instead of trusting the engine default
The default is what varies between Docker versions, not the explicit value.
Measured by starting each engine and reading the mount back:

    engine   tmpfs without mode   tmpfs with mode 1777
    29.7.2   1777 root            honoured
    28.5.2   755 root             1777 root

A checkmk site runs as uid 1000, so a root-owned 755 tmp makes omdlib fail
`mkdir /omd/sites/cmk/tmp/apache` and the container restarts forever. That is
exactly what run 34348821577 shows, on runners reporting Docker 28.0.4.

f3eba39cae removed `mode: "01777"` here after measuring on 29.7.2 alone, where
the default happens to match, and so replaced the mechanism that behaves the
same everywhere with the one that does not. The mode comes back.

The validator loses the ban with it. Its stated reason, that `docker stack
deploy` drops the key, is a different claim from the one the removing commit
made, and neither survives the measurement above for the compose path this
failure came from; it now checks the type and nothing more.

One thing this does not settle. The note this repository kept on the earlier
attempt records run 33923190220 failing the same way on 28.0.4 while the mode
WAS set. That contradicts the 28.5.2 row and did not reproduce, so either the
patch releases differ or that diagnosis was wrong. The next run decides it:
read the engine version out of the rescue artifact and check whether the site
tmp comes up 1777.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 00:55:05 +02:00
kevinveenbirkenbachandClaude Opus 5 59d45ba665 style: apply autoformat normalisation
What `make quality-high` rewrote on its own, kept apart from the changes it ran
against so neither has to be read through the other.

The matomo healthcheck test moves from a flow sequence to a block one, which is
the formatter's threshold for a long line and not a change to the command: both
revisions parse to the same two-element list, escapes included. The healthcheck
timing lint loses two blank lines between methods.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-10 00:34:23 +02:00