Commit Graph
2780 Commits
Author SHA1 Message Date
kevinveenbirkenbach 37461b87ab Merge remote-tracking branch 'origin/main' into fix/web-app-prometheus/blackbox-targets/792 2026-10-06 19:51:20 +02:00
kevinveenbirkenbachandClaude Opus 5.5 c73bc0ca85 test(web-app-keycloak): follow every async kcadm result to its reap
The guard required a register on every async kcadm job and nothing more.
A result that is registered but no longer handed to async_reap_jobs,
directly or through the list it is collected into, passed it, although
the reap then neither polls that job nor sees it fail.

The test now follows each registered result to an async_reap_jobs
expression, by variable name with word boundaries, so kc_declared_client
does not pass for kc_declared_client_jobs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 16:40:48 +02:00
kevinveenbirkenbachandClaude Opus 5.5 601b21a8f4 test(web-app-penpot): require both service names in the start guard
The stub answered every absolute name, so a guard that asked for the
backend twice and never for the exporter passed both tests. The stub now
records the names it is asked for, and the tests require the backend and
the exporter.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 16:40:48 +02:00
kevinveenbirkenbachandClaude Opus 5.5 8834cc124a fix(web-svc-collabora): build the WOPI host from Nextcloud's own domain
COLLABORA_WOPI_HOST read the scheme and the port with
lookup('tls', 'web-app-nextcloud', ...) and the host with
lookup('domain', 'web-app-nextcloud'). The tls lookup aligns an app term
to its consumer: for a clearnet Collabora it answers for the clearnet
sibling of an onion-primary Nextcloud, while the domain lookup keeps the
onion name. Such a deployment renders https://<onion>:443, an origin
Nextcloud does not use, so the allowlist does not match its WOPI calls.

Nextcloud pins overwritehost and overwriteprotocol to its primary domain,
so the allowlist now asks the tls lookup for that domain instead of for
the app.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 16:40:48 +02:00
kevinveenbirkenbach 415a9cb91a Merge remote-tracking branch 'origin/main' into fix/web-app-prometheus/blackbox-targets/792 2026-10-06 16:40:34 +02:00
kevinveenbirkenbachandClaude Opus 5.5 2a1c5e13ce fix(web-app-prometheus): judge each probe by its own target
Since the targets take their scheme from the domain they probe, two
switches of the http_ready module no longer fit. Both still followed the
TLS state of the Prometheus vhost.

fail_if_not_ssl refused an http target, an onion vhost for example, as
soon as Prometheus itself was served over TLS. It is false now. The
module does not follow redirects, so the scheme of the target URL already
decides the transport.

insecure_skip_verify switched certificate checks off whenever Prometheus
itself had no TLS, also for an https target. Checks now stay on, except in
a self-signed deployment whose CA is not mounted into the Prometheus
containers, where the exporter has nothing to verify against.

The per-role test of the spec parsed every answer of the query API as
JSON. An answer that is not OK now counts as "no probe yet", so the poll
keeps waiting instead of ending on a parse error.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 16:40:34 +02:00
kevinveenbirkenbach 0e9354f1a0 Merge remote-tracking branch 'origin/main' into fix/svc-net-tor/chutney-node-workers/824 2026-10-06 16:40:29 +02:00
kevinveenbirkenbachandClaude Opus 5.5 4f8c1f00f1 fix(svc-net-tor): fail the worker check when no torrc can be read
The check listed the torrcs without the limit through "grep -L ...; true".
With no torrc under /opt/chutney/net/nodes, or with one that cannot be
read, grep printed nothing, the trailing true turned its error into
success, and the test reported that every node runs one worker.

The check now counts the limit per file with "grep -cH" and reads the exit
status. A status above 1 or an empty answer fails the test, and a file
with a count of 0 is named as before.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 16:40:28 +02:00
kevinveenbirkenbach 5d854cc005 Merge remote-tracking branch 'origin/main' into fix/code-scanning-alerts/807 2026-10-06 13:43:32 +02:00
kevinveenbirkenbach 40c033fc22 Merge pull request #826 from infinito-nexus/fix/web-app-keycloak/kcadm-config-lock/805
fix(web-app-keycloak): keep concurrent kcadm jobs from losing the config lock
2026-10-06 13:07:26 +02:00
kevinveenbirkenbach c4bcb9a31c Merge pull request #818 from infinito-nexus/update/version-sources-web-app-xwiki-20261005-dc2425a
update(versions): update pins with a declared source (web-app-xwiki)
2026-10-06 13:05:11 +02:00
kevinveenbirkenbach d0ef1f81a1 Merge pull request #825 from infinito-nexus/fix/web-app-penpot/frontend-start-guard/806
fix(web-app-penpot): make the frontend wait for its backend by absolute name
2026-10-06 12:22:36 +02:00
kevinveenbirkenbachandClaude Opus 5.5 7968b47cf0 fix(svc-net-tor): give every chutney node one worker thread
Tor starts its worker threads by the number of CPU cores it detects, and
the cpus limit of the container does not change that number. On a host
with 20 cores every node of the private chutney network ran 21 threads and
grew to about 0.6 GB. The seven relays and authorities filled the 2g
limit and its swap within ten minutes, the kernel killed them one by one,
and no circuit could be built any more.

The entrypoint now writes NumCPUs 1 into the torrc of every chutney node
before it starts the network. Measured on that host over ten minutes: the
nodes run 3 threads and stay at 65 to 86 MB, the container at 0.47 GB
without swap, where it had filled 2 GB plus 2 GB of swap before.

The CLI test of the role fails when a chutney node runs without the limit.

Closes #824

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 11:37:54 +02:00
kevinveenbirkenbachandClaude Opus 5.5 c35f2409a4 fix(web-app-odoo): make the OAuth addon work on Odoo 20
Nobody could sign in to Odoo through Keycloak. Odoo sent an http redirect
URI, and Keycloak refused it as "Invalid parameter: redirect_uri".

The addon auth_oauth_https builds that URI from web.base.url. Two things
kept it from doing so after the image moved to Odoo 20.

Its manifest declared version 19.0.1.8.0, and Odoo marks a module whose
version names another series as not installable. It only warns about that,
so the install task stayed green and the addon was missing. The manifest
now declares 1.8.0, which Odoo prefixes with its own series.

Once installed, the addon read the parameter with get_param, which Odoo 20
replaced by typed getters. The call raised AttributeError, a broad except
logged it and fell back to the URL of the request, which is http behind
the proxy. The addon reads the parameter with get_str now and no longer
catches the error.

A deploy now fails when the addon is not installed afterwards. The check
reads ir_module_module, because the output of "odoo -i" does not tell an
installed module from a skipped one.

The spec logged out by navigating to /web/session/logout. Odoo 20 accepts
POST only on that route and answered 405, so both SSO tests failed after
a successful login. The spec logs out through the user menu now.

A host that runs the suite with several workers still failed it. Odoo
computes the session token of an account from its OAuth access token, so
every new sign-in ends the other sessions of that account, and the spec
files share the administrator. The role caps its suite at one worker
with PLAYWRIGHT_MAX_WORKERS, which the central config now honours, and
the spec asserts the cap.

Closes #822

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 11:37:41 +02:00
kevinveenbirkenbachandClaude Opus 5.5 883b9c4f46 fix(web-app-prometheus): probe apps where their vhosts answer
Every blackbox probe of a Tor deployment answered 404, so AppDown fired for
all deployed apps although they were up. The job built its targets from
canonical_domains_map(DOMAIN_PRIMARY), which knows no onion domain. In a Tor
deployment the vhosts answer under <sub>.<onion> only, and the front proxy
served the clearnet names from its default server.

The targets now come from lookup('domains'), the map the vhosts are built
from, and each target takes its scheme from the domain it probes instead of
from the Prometheus vhost.

Which roles are probed follows lookup('roles_with_service', 'prometheus'),
the list the role's own spec expects metrics for. It leaves out the database
roles, which have a domain in the map and no vhost, and the roles that
declare scrape: false because their vhost carries no Prometheus block.

Two roles declared more than their vhost delivers. web-svc-mirror renders a
vhost of its own with no backend behind it, and on a live stack its probe
answered 404 under the onion name as well, so it declares scrape: false now.
web-app-peertube has a backend, but its vhost template never included the
Prometheus locations. It includes them now.

The spec could not see any of this in a compose deployment. dotenv_quote
wraps PROMETHEUS_TARGET_ROLES_JSON in quotes there, JSON.parse returned a
string, the role list came out empty and the per-role test was skipped. The
value is decoded now, and the test also requires probe_success == 1 for
every role in the list.

Closes #792

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 11:37:28 +02:00
kevinveenbirkenbach ab3b365fb9 Merge remote-tracking branch 'origin/main' into fix/code-scanning-alerts/807 2026-10-06 09:18:00 +02:00
kevinveenbirkenbach 0622cce916 Merge remote-tracking branch 'origin/main' into fix/code-scanning-alerts/807 2026-10-06 09:18:00 +02:00
kevinveenbirkenbachandClaude Opus 5.5 3610e7b9ef fix(code-scanning): resolve the alerts that describe real code
Of the 76 open code scanning alerts, 39 point at code that can change.

svc-ai-mcp-adapter:
- A path argument that is empty or made only of dots is refused.
  Percent-encoding keeps a separator inside its segment, but "." and ".."
  need none, so an upstream that resolves dot segments could answer from a
  path the contract does not name.
- parse_duration splits on the units. The pattern it used before rescanned
  a long run of digits from every start position.

Tests and the README generators:
- Every Jinja environment built with autoescape=False and a "noqa: S701"
  uses select_autoescape now, as six test modules and
  utils/templating/jinja.py already did. It escapes templates named
  *.html, *.htm and *.xml, which none of these call sites loads.
- Dead code goes: a doubled "rel" block in the email lint, an unread
  _SKIP_DIRS, and a module that _resolve returned to no caller.
- Two "except ...: pass" blocks become contextlib.suppress and
  assertRaises.
- Nine test modules import unittest.mock one way instead of two, and one
  lookup test imports its module once.

Closes #807

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 09:18:00 +02:00
kevinveenbirkenbach 682402e5b3 Merge remote-tracking branch 'origin/main' into fix/web-app-keycloak/kcadm-config-lock/805 2026-10-06 09:17:56 +02:00
kevinveenbirkenbachandClaude Opus 5.5 30084f986c fix(web-app-keycloak): keep concurrent kcadm jobs from losing the config lock
In the async pass the role runs kcadm jobs side by side. Every kcadm call in
the Keycloak container shares one config file. kcadm rewrites that file when
it refreshes the access token, about every 55 seconds, takes the file lock
with tryLock() and gives up at once when another kcadm process holds it. One
of two overlapping declared-client jobs then failed with "Failed to get lock
on /opt/keycloak/.keycloak/kcadm.config", and the deploy with it.

keycloak_kcadm_update now repeats a call that lost the lock, up to ten times
one second apart. The call fails while it refreshes its token, before it
sends its own request, so repeating it is safe. Every other error still ends
the call at once.

That covers the module only, so the async jobs are reduced to the ones that
go through it:

- The default-scope assignment was an async shell task. It ran beside the
  shell tasks that follow it, and neither side repeats a lost lock. It runs
  in line now.
- The two mail-settings jobs were async without a register, so nothing
  reaped them and a failure was never seen. They run in line now.

The client job and the declared-client jobs stay async. A test fails when an
async kcadm task does not use the module or is not registered for the reap.

A config file of its own per call was the other option. It does not work by
itself, because the admin session lives only in the shared file.

Closes #805

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 09:17:56 +02:00
kevinveenbirkenbach 123f85f1c0 Merge remote-tracking branch 'origin/main' into fix/web-app-penpot/frontend-start-guard/806 2026-10-06 09:17:52 +02:00
kevinveenbirkenbach 207553eeb2 Merge remote-tracking branch 'origin/main' into fix/web-app-penpot/frontend-start-guard/806 2026-10-06 09:17:51 +02:00
kevinveenbirkenbachandClaude Opus 5.5 5494fbad48 fix(web-app-penpot): make the frontend wait for its backend by absolute name
nginx resolves its upstreams once, at start, so the frontend starts through
a guard that waits until `backend` and `exporter` resolve. A container
resolves with a search domain, and the zone of a deployment answers every
name with the address of the ingress. A service that Docker's DNS did not
know yet therefore resolved anyway, as `backend.<domain>`. The guard let
nginx start at once and nginx pinned the ingress as its backend. Every API
call then failed with 502 and the SPA showed "Bad Gateway" instead of the
login form, until the container was restarted.

The guard now asks for the absolute names, `backend.` and `exporter.`. The
search domain is not applied to those, so the lookup fails until the
services themselves resolve.

Closes #806

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 09:17:51 +02:00
kevinveenbirkenbachandClaude Opus 5.5 e142d07f97 fix(web-app-xwiki): upgrade an add-on that is not on its pinned version
The extension installer returned as soon as an add-on was installed, in
whatever version. A new pin in meta/addons/ therefore reached a fresh wiki
only, and an existing wiki kept the version it was first deployed with.

The installer now skips an add-on only when the installed version equals
the pin. Otherwise it calls install(id, version, namespace), which the
Extension Manager resolves as an upgrade, and it reports an error when the
add-on is not on the pin afterwards.

The check that follows the installer compared presence only. It takes the
pin as well, so a deploy fails when an add-on is installed in another
version.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 09:17:15 +02:00
Prageeth Panicker bf37d5e7b3 fix(web-app-wazuh): address PR review feedback
Tor: use the shared tor block (bond 0.5, enabled when svc-net-tor is in the node's groups, tor pinned on in both variants) so a node serves one network, onion on a Tor node and clearnet otherwise.

SPOT and sizing: WAZUH_INDEXER_URL replaces four hand-built copies of the indexer URL, and the indexer heap is half of mem_limit instead of a hardcoded 1g (unchanged for the current 2g limit).

Fixes: bcrypt_hash rejects secrets over 72 bytes; the Keycloak group-removal helper uses the same resolver as the add helper and checks every response (the realm is no longer URL-encoded, matching the add helper), and a failed cleanup fails an otherwise passing test; the swarm certs generator removes a stale container before each attempt; vm.max_map_count is set before the early handler flush; securityadmin failed_when now needs rc 0 and the success marker.

The regenerated role README also picks up generator drift (mailu to stalwart).
2026-10-06 05:15:19 +00:00
kevinveenbirkenbachandClaude Opus 5.5 e6fbb4b3f4 chore: drop two leftovers of the Wazuh role's first commit
That commit added test_bcrypt_hash.py with an Exception comment and left a
trailing blank line in group_vars/all/11_oidc.yml. Both go.

What the comment said still holds: the test hashes with the raw bcrypt package
because Ansible's password_hash('bcrypt') routes through passlib, whose bcrypt
self-test can misreport a password well under 72 bytes as too long when
passlib and the installed bcrypt disagree on the version.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 05:15:17 +00:00
Prageeth Panicker 87c8199012 feat(web-app-wazuh): add Wazuh SIEM role with OIDC/LDAP RBAC
Add single-node Wazuh stack (manager, indexer, dashboard) as a new
Ansible role, integrated with the platform's Keycloak realm via native
OIDC and secondary LDAP authorization, with three-tier RBAC.

- Pins wazuh/wazuh-{manager,indexer,dashboard}:4.14.6 and
  wazuh-certs-generator:0.0.4 (one-shot cert bootstrap job)
- No external database: wazuh.indexer (OpenSearch) is self-contained
- Auth: native OIDC against Keycloak, working end-to-end after two
  infra fixes (JVM cacerts trust for the indexer's outbound calls to
  Keycloak, front-proxy vhost wiring)
- LDAP wired as a secondary, authentication-independent authorization
  backend (group-based backend_roles enrichment only)
- RBAC: three roles (administrator, security-analyst, readonly-auditor)
  via the platform's standard meta/rbac.yml + rbac_group_path pipeline
- Documents a genuine upstream Wazuh bug: wz-home as the default
  landing route throws an uncaught client exception for any
  non-all_access role; default route moved to OpenSearch Dashboards'
  own /app/home

Addressed two full rounds of review feedback (Kevin, Alejandro):
WEB_PROTOCOL usage, service-account naming, comment cleanup, shared
Keycloak-admin-helper extraction, a real TOCTOU race in RBAC test
setup, a Mermaid auth-flow diagram, meta/services.yml placement for
the certs-generator, CA-trust-path SPOT usage, CSP tightening
(including a genuine worker-src: blob: requirement found via live
testing), and a loud (non-silent) dev-bucket fallback warning.

Rebased onto main after a 3-month gap; brought the role in line with
~30 platform-convention changes introduced in that window (swarm
compose-template rules, Tor/onion contract, Playwright timeout/goto
wrappers, file layout, task numbering, comment policy, and more).

Live-deploy verification (9 iterations) found and fixed 6 further
real bugs invisible to lint/unit tests alone:
- credentials.* / meta/schema.yml -> secrets.credentials.* /
  meta/secrets.yml (another platform-drift casualty)
- notify: "compose up" -> "compose-up" (day-one handler-name typo)
- local.subnet collision with web-app-mailu's /24
- missing primary: false alongside tor.exclusive: false, which let
  the onion domain shadow the clearnet vhost entirely
- a second front-proxy include for the onion domain, later reverted
  (see below)
- shared sys-stk-front-proxy/sys-front-tls TLS-lookup keying bug
  (app-id-keyed vs. domain-keyed), which silently applied TLS to
  Wazuh's onion vhost

The last two are related: enabling genuine dual-stack (clearnet +
onion) reachability for tor.exclusive: false surfaced a real bug in
two shared platform roles. That fix was verified safe against all 103
apps currently in the repo, but is deliberately NOT included here -
shared-role changes deserve their own PR and review, not to ride in
on a single app's feature branch. Wazuh's second front-proxy include
was reverted accordingly; the resulting onion-reachability gap is
documented in the README as a known, platform-scoped limitation.

make test: 5508 passed, 3 failed (all three trace to untracked local
scratch files - Dockerfile.bak, scripts/lab-*.sh - not present in any
CI checkout).
2026-10-06 05:15:00 +00:00
kevinveenbirkenbach be800e8741 Merge pull request #803 from infinito-nexus/fix/sys-ctl-hlth-msmtp/empty-reply-is-transient/790
fix(mail): retry a connection the mail server drops before its greeting
2026-10-06 02:28:06 +02:00
kevinveenbirkenbach 88418016ba Merge pull request #788 from kevinveenbirkenbach/fix/ci-fork-image-wait/784
fix(ci): keep the fork image wait alive and let it find its privileged run
2026-10-06 01:36:54 +02:00
kevinveenbirkenbachandClaude Opus 5.5 89b95d751c fix(update): stop bumping pins of eol roles
The update bot opened a version PR for web-app-jira (#821) although the role is end of life: none of the updaters looked at the lifecycle. utils.update.base.is_maintained now answers that question once, and the collectors for image versions, declared version sources, pip pins, repository refs and the addon check skip a role whose lifecycle is eol.

The lint that requires every semver pin to be watched shares that scope, otherwise it would report the pins of the eol roles as unwatched. Every other stage, deprecated included, stays in the update scope.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 01:09:10 +02:00
kevinveenbirkenbachandClaude Opus 5.5 517d40484e test(mail): select autoescape for the rendered alarm script
The unit test of the alarm script built its Jinja environment with
autoescape switched off, which code scanning reports as a possible XSS
sink and fails the PR on. The template is a shell script, so
select_autoescape() leaves it unescaped exactly as before, and the finding
no longer applies.

Refs #790

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 17:19:56 +02:00
kevinveenbirkenbachandClaude Opus 5.5 4c955d080f feat(update): fail a declared update source whose pin is not a semver
The version-source resolver orders upstream versions by semver. A pin that
is not one, such as `latest` or a branch name, has no depth, so no candidate
ever matches it and the declaration never moves anything. Nothing reported
that: the watched-pins lint only looks at values that already are a semver.

test_version_keys now fails on an `update:` block whose pinned key does not
hold a semver. A monitored addon the resolver does not read is held to the
same rule as soon as it pins a `version`, because the catalog check skips a
non-semver pin just as silently. An addon without a `version` keeps tracking
the app default and is left alone.

The Stalwart README no longer asks for `webui_version` to be bumped by hand
next to the image tag. Its `update:` block names the upstream tags, the
server itself follows `releases/latest` of the bundle, and the resolver
plans v1.0.5 to v1.0.11 for it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 12:48:22 +02:00
evangelostsak 8c84f5a271 docs(tests): drop the EPEL mirror comment
The probe reads one host either way. The mirror history belongs to the
commit that moved it.
2026-10-05 12:43:23 +02:00
evangelostsak 4e44d7fb84 Merge remote-tracking branch 'upstream/main' into feature/svc-net-wireguard
No conflicts. Two files were touched by both sides and both kept each
side: the Makefile, where upstream retires the mcp-audit target and this
branch pins the throwaway cluster's vpn axis, and requirement 035, where
the role table keeps the single svc-net-wireguard row.

Incoming carries the fixes for two applications this branch's last run
failed on -- penpot probing its frontend without curl, and mobilizon's
postgres 18 migrations -- plus dependency bumps and a lint repair.

Gates on the merged tree: unit 6202, lint 523, integration 330,
ansible-lint clean.
2026-10-05 12:17:12 +02:00
kevinveenbirkenbachandClaude Opus 5.5 7c7b895ae9 fix(ci): match the privileged run of a fork PR by its number as well
The wait job picks the newest privileged run on the head commit of the
PR. Two fork PRs that share a head repository, branch and commit but
target different bases each start such a run, and the commit alone cannot
tell them apart. A failed run of the other PR would then end the wait of
this one while its images are still being built.

The run name of the workflow carries the PR number, so the match now
requires it next to the head commit. A run without that name matches
nothing, and the wait does what it does whenever no run is found: it
keeps polling for the images until they exist or the wait times out.

Raised by the CodeRabbit review of this PR.

Refs #784

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 12:07:14 +02:00
kevinveenbirkenbach 1d03c001ea Merge remote-tracking branch 'origin/main' into fix/sys-ctl-hlth-msmtp/empty-reply-is-transient/790 2026-10-05 11:45:00 +02:00
kevinveenbirkenbachandClaude Opus 5.5 b37cf2e9fe fix(mail): retry a connection the mail server drops before its greeting
Stalwart closes a connection that exceeds its per-IP connection budget
without a greeting. msmtp reports that as "the server sent an empty reply"
with exit 76, and the mail health check treated exit 76 as permanent. One
such drop, in a second in which Alertmanager flushed its notifications,
failed a deploy although the same path accepted a mail one second later.

The health check now retries exit 76 when msmtp names the empty reply,
within the ten attempts it already gives to exit 69, 75 and 124. Every
other protocol error still ends the check at once.

The alarm mail was sent once, so the alarm about a failed unit was lost to
the same drop. It now makes three attempts ten seconds apart for the
transient cases of the health check and still reports the last error.

The limit on the server is left alone. Exempting the node's own senders by
source address would also exempt Tor clients and, in swarm, every external
client. That question is tracked in #793.

Closes #790

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 11:44:58 +02:00
kevinveenbirkenbach 8e9fa797fe Merge remote-tracking branch 'origin/main' into fix/ci-fork-image-wait/784 2026-10-05 03:13:43 +02:00
kevinveenbirkenbach a306b225c3 Merge remote-tracking branch 'origin/main' into update/onlyoffice-nextcloud-connector 2026-10-05 03:13:35 +02:00
kevinveenbirkenbachandClaude Opus 5.5 69fceafb5d fix(ci): limit the fingerprint shortcut to branches of this repository
The duplicate check accepted any open PR whose head branch started with
the dedupe prefix and ended in the fingerprint of the fresh proposal. The
listing also returns PRs from forks, and the fingerprint can be derived
from public data, so a fork PR with such a branch name would have made the
bot skip its own update PR without carrying the change.

The shortcut now only applies to PRs whose head branch lives in this
repository, which only accounts with write access can create. A fork PR
is still compared by the content of its head, as before.

Raised by the CodeRabbit review of PR #781.

Refs #780

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 03:06:57 +02:00
kevinveenbirkenbachandClaude Opus 5.5 739c78bad1 feat(update): bump addon pins through their declared source
The version-source updater only read meta/services.yml. An addon pinned in
meta/addons/<id>.yml stayed where it was: the ONLYOFFICE connector for
Nextcloud sat on v10.1.2, which does not install on Nextcloud 35, until
someone bumped it by hand.

The updater now reads addon files too. An addon declares its source in the
`update:` block it already has, beside monitored, catalog and upstream_id.
A monitored addon of the github-releases catalog needs nothing more: its
upstream_id names the repository whose tags are its versions. The
update-version-sources job opens the PRs, one per role.

Every pinned addon is wired:

- nextcloud onlyoffice carries its own version pin now. Its archive URL
  stays a literal that carries the pin, because
  test_pinned_addon_archives_available reads it as written
- wordpress mcp-adapter, moodle auth_oidc and jellyfin sso-authentication
  follow the tags of their repository, moodle within its v4.5 series
- the three xwiki extensions read their Maven metadata
- pretix-oidc reads the PyPI simple index. The JSON API answers 404 behind
  the package cache the test stack routes pypi.org through
- joomla's plugin is built here and jellyfin's ldap-authentication is bound
  to the md5 of its archive; both are opted out with that reason

A bump rewrites the pin and the config.archive line together, and an
archive that does not carry the pin fails test_version_keys, so the two
cannot drift apart again. test_version_pins_are_watched covers addon pins
as well: a new one without a source or a reasoned opt-out fails the lint.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 02:50:48 +02:00
kevinveenbirkenbachandClaude Opus 5 81ab6e4ffc feat(update): let a version pin declare its own upstream source
The Docker updater watches image plus version, the repository updater
repository plus ref. A dozen pins fell outside both - an app_version, a
release, a tag in a registry neither speaks - so nothing ever told anyone
they had gone stale.

Such a pin now names its upstream next to itself in an `update:` block:
git_tags, registry_tags, npm, http_regex or a role-local script, with
`match` selecting a tag flavour and `strip` removing a prefix the pin does
not carry. utils/update/source.py resolves them beside the two existing
updaters, `python -m cli.contributing.update.source` bumps them, the
update-version-sources job opens one PR per role, and
tests/external/update/source warns while a bump is pending.

Two lints hold the rule: every semver pin is watched by one of the three
updaters or opted out with a reason, and a semver only ever sits behind
`version`, `<what>_version` or `ref` - which renames moodle's release to
app_version and simpleicons' local_version to local_tag, the latter being
the tag of a locally built image rather than an upstream version. A
declaration naming a key the entity does not carry, or a type the resolver
does not know, fails the lint instead of resolving nothing in silence.

Picked from feature/store-i18n (94d607fd0a) onto current main. Two things
differ from the original: svc-ai-lmstudio no longer carries an app_version
pin, so its declaration is dropped, and web-app-stalwart's webui_version,
added since, declares the git tags of the webui repository.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 02:50:48 +02:00
kevinveenbirkenbach f6228580be Merge remote-tracking branch 'origin/main' into fix/ci-fork-image-wait/784 2026-10-04 22:18:20 +02:00
kevinveenbirkenbach f99a41689c Merge pull request #781 from kevinveenbirkenbach/fix/update-bot-duplicate-pr/780
fix(ci): treat an edited update branch as a duplicate of its own bump
2026-10-04 22:15:56 +02:00
kevinveenbirkenbachandClaude Opus 5.5 3ab0153856 fix(ci): find the privileged run of a fork PR by its head commit
The wait for fork CI images never found the privileged workflow run it
watches. It selected a run only when the run's pull_requests array held
the PR number, and for a run whose head repository is a fork the API
returns that array empty. The job only runs for fork PRs, so the lookup
matched in no case it exists for.

The early exit for a failed privileged run was therefore unreachable.
When that run failed, the job kept polling until its timeout of 1980
attempts and held a runner for about five and a half hours.

The lookup now asks the API for the runs of the PR head commit and
selects by head_sha. That also replaces a paginated listing of every run
of the workflow with one small request per attempt. Without a known head
commit the match by PR number stays.

Refs #784

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 20:55:54 +02:00
kevinveenbirkenbachandClaude Opus 5.5 53f71d85c8 fix(ci): keep waiting for fork images when a run lookup fails
The job that waits for the CI images of a fork PR ended on the first
failed GitHub API call. The lookup of the privileged workflow run is
assigned with run_json="$(find_matching_run)" under set -e, so a failing
gh ended the script, although the loop is built for 1980 attempts. PR #782
went red that way: GitHub answered one request with HTTP 504 in attempt
31, while the privileged run was still queued.

A failed lookup now counts as "no information in this attempt" and the
loop goes on waiting for the images, which are its success criterion.
Thirty failed lookups in a row end the job with a message that names the
cause, so a broken token does not hold a runner until the timeout.

The test runs the script against a gh that fails every lookup. The
previous script fails both cases at the first lookup.

Closes #784

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 20:55:54 +02:00
kevinveenbirkenbachandClaude Opus 5.5 9bdf33b89d fix(ci): treat an edited update branch as a duplicate of its own bump
The update bot recognised an open PR as a duplicate by hashing the blobs
of the files its commit changed, read at the PR head. A follow-up commit
that edits one of those files changes the hash, so the next run opened a
second PR with the identical bump: PR #778 next to PR #775 for
web-svc-collabora, after a fix had edited the bumped services.yml.

The branch name of a bot PR ends in the first seven characters of the
fingerprint of the change the bot proposed. An open PR whose branch ends
in the fingerprint of the fresh proposal now counts as a duplicate before
its head is compared. The head comparison stays as the second criterion.

The test runs pr.sh in a throwaway repository against a stubbed gh. It
fails on the previous script with exit 1, because that one goes on to
push.

Closes #780

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 14:36:57 +02:00
kevinveenbirkenbachandClaude Opus 5.5 5b002678df docs(mcp): remove the generated audit report
The report aggregated each role's meta/mcp.yml into one table, and its
freshness test failed on every image-version bump of an MCP-serving role,
because the updater edits services.yml and nothing regenerates the committed
report. The table added nothing the sources lack: scope and credential subject
were identical for every role, so it restated its inputs.

The generator, the report, its freshness test, make mcp-audit and the i8mcau
alias go, and make docs no longer calls the generator. The metadata stays in
each role's meta/mcp.yml, where test_mcp_schema.py enforces its shape, and
requirement 035 now says so. The classification and completeness lints stay,
because they check the lists in 035 against the roles, not the report.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 14:36:52 +02:00
evangelostsak ae56878854 Merge remote-tracking branch 'upstream/main' into feature/svc-net-wireguard
No conflicts, and no file touched by both sides. Incoming is a run of
Docker image version bumps plus one CI change, which moves the diff
baseline from a force-pushed mirror of main to a fetch in each consumer --
entry-push-latest.yml and the resolve scripts, none of which this branch
touches.

Gates on the merged tree: unit 6199, lint 525, integration 330, external
19, ansible-lint clean, markdownlint clean, ruff clean.
2026-10-04 12:36:42 +02:00
kevinveenbirkenbachandClaude Opus 5 bf25eb2fc7 refactor(ci): fetch the diff baseline instead of mirroring main
The sync job force-pushed the source repository's main onto this one so that
`origin/main` would mean "upstream main" for the three consumers that diff
against it. Carrying .github/workflows/** in that push is what GitHub refuses
for an App token, and `workflows` is not a GITHUB_TOKEN permission scope, so the
job needed an app installation that may write workflow files. Run 37169357364
showed the fork has no such secret at all.

Nothing reads the pushed branch. push_ci_policy.sh, affected_roles.sh and
new_roles.sh each fetch `origin main` into their own runner's clone before
reading it, and no caller queries the branch through the API. Pointing those
fetches at the configured source repository gives the same baseline read-only:
no push, no credential, no app permission to widen.

baseline_remote.sh resolves where that baseline comes from and keeps the
normalization and the disabled-value list in one place: `origin` for the source
repository itself, for a disabled sync and for a local clone without
GITHUB_REPOSITORY, and the upstream fetch URL otherwise. The fetches now carry
an explicit `main:refs/remotes/origin/main` refspec, which a URL needs and a
remote name tolerates.

The release guard keeps fetching `origin`: it asks whether a tag commit sits on
this repository's own main, which is a different question from the diff
baseline.

Removed: the sync-main job, its `needs` edge, main_from_repository.sh and the
directory that held it.

test_diff_affected_roles.py copies only the script under test into its fixture,
so it needed the new helper too. It also inherited the full environment, and a
GITHUB_REPOSITORY present in CI would have sent the fixture's baseline fetch to
the real upstream instead of its own bare remote: three of its five cases fail
that way, and the run takes 24s instead of 0.3s because it goes to the network.
The variable is now dropped alongside BASH_ENV.

Verified by running the helper in all four contexts and affected_roles.sh from
an empty refs/remotes/origin/main in a scratch clone as the fork, as the source
repository and locally; each populated the ref and completed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-04 05:44:45 +02:00