Commit Graph
120 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 d89c74feab feat(security): harden PHP services and proxy response headers
Proxy:
- server_tokens off, X-Powered-By stripped from upstream responses,
  HSTS max-age raised to one year.
- No global X-Content-Type-Options: under nosniff, ORB (step 3.4)
  blocks no-cors cross-origin responses typed text/html, e.g. tracking
  beacons answered with an empty 204.

PHP (CVE-2026-109618, CVE-2026-109619):
- New role sys-svc-compose-php installs a shared ini (expose_php Off,
  session.upload_progress.enabled 0, fatal_error_backtraces 0 for
  PHP 8.5, interim memory_limit -1) and a PHP-FPM [www] drop-in
  (pm.max_requests = PHP_FPM_MAX_REQUESTS) once per stack host,
  before 04_files renders any compose file.
- The host file names carry a sha1 prefix of the controller-side
  source, so a content change alters every PHP app's compose.yml and
  recreates it, also on remote deploys where compose_volumes cannot
  read the stack host file and hashes the path instead.
- 20 PHP roles mount the ini as a compose/swarm config into their
  image's scan dir; moodle, nextcloud and shopware also mount the FPM
  drop-in. Magento keeps its own pool: its php-fpm.conf has no
  include= and is rendered without notify.
- Joomla's php-upload.ini now loads as zzz-joomla-uploads.ini so its
  own memory_limit wins: the media API compares Content-Length with
  toBytes(memory_limit) and rejects every upload at -1.

TODOs record the memory_limit revert condition and accepted
trade-offs, Akaunting on PHP 8.1, FusionDirectory on PHP 7.4 and the
public Keycloak admin console.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-21 21:03:38 +02:00
evangelostsak 2c435456bb feat(web-app-moodle): let a deployment put its hosts into meshes 2026-10-16 03:40:19 +02:00
kevinveenbirkenbach 402aa2fe9d Merge remote-tracking branch 'origin/main' into update/version-sources-web-app-moodle-20261005-4f455e2 2026-10-07 02:10:13 +02:00
kevinveenbirkenbachandClaude Opus 5.5 d1eee3e26f fix(web-app-moodle): run Moodle 5.3 and upgrade existing sites to it
The pin moved from 4.5.11 to 5.3.0, and the role fitted neither a new
install nor an existing one.

Moodle 5 serves from public/ and routes every other path through
r.php. nginx now uses public/ as its root, hands unknown paths to the
router and passes PATH_INFO for scripts with slash arguments, and
config.php declares the router as configured. The paths the role checks
and writes (version.php, auth, webservice, the OIDC plugin) moved below
public/, the URL of the OIDC plugin stays auth/oidc. The source archive
no longer contains vendor/, so the image installs the Composer
dependencies it declares. The OIDC plugin is pinned to v5.2.1, the
newest release.

An existing site kept its old code: the entrypoint filled the code
volume once and never again. It now records the release it copied and
replaces the tree when the image carries another one, and a new task
runs admin/cli/upgrade.php. Plugins that were installed through the web
interface are not carried over, the README says how to ship them.

A site installed with 5.3 forces login by default, which 4.5 did not,
so its start page answers 303 to the login page and the webserver
health check refused it. The role declares 200 and 303 as the codes
that check accepts: 200 is what an upgraded site keeps answering.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 02:10:13 +02:00
kevinveenbirkenbachandClaude Opus 5.5 0f7fb92e12 refactor(roles): take seven roles' memory parameters from the container limit
Memcached, Joomla, Moodle, Odoo, Shopware, Snipe-IT and SuiteCRM set
their cache size, PHP memory_limit, OPcache, worker limits and search
heap as literals next to a mem_limit they never read. They now ask
lookup('service_memory_mb') for a share of that limit. The shares are
chosen so every rendered value stays what it was.

Where the parameter lives in a file that cannot hold a template, the
value is handed in: Shopware's init script reads INIT_PHP_MEMORY_LIMIT
and refuses to start without it, SuiteCRM's image takes it as a build
argument. Shopware's worker gets --memory-limit as a command appended
to its entrypoint, and Moodle's OPcache size moves from the static ini
into the rendered one.

The performance.php.memory_limit keys of Joomla and Moodle are gone
from services.yml; nothing else read them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 23:24:22 +02:00
infinito-nexus-updater[bot] 2da51d1b78 update(versions): update pins with a declared source (web-app-moodle) 2026-10-05 19:20:27 +00:00
kevinveenbirkenbachandClaude Opus 5.5 739c78bad1 feat(update): bump addon pins through their declared source
The version-source updater only read meta/services.yml. An addon pinned in
meta/addons/<id>.yml stayed where it was: the ONLYOFFICE connector for
Nextcloud sat on v10.1.2, which does not install on Nextcloud 35, until
someone bumped it by hand.

The updater now reads addon files too. An addon declares its source in the
`update:` block it already has, beside monitored, catalog and upstream_id.
A monitored addon of the github-releases catalog needs nothing more: its
upstream_id names the repository whose tags are its versions. The
update-version-sources job opens the PRs, one per role.

Every pinned addon is wired:

- nextcloud onlyoffice carries its own version pin now. Its archive URL
  stays a literal that carries the pin, because
  test_pinned_addon_archives_available reads it as written
- wordpress mcp-adapter, moodle auth_oidc and jellyfin sso-authentication
  follow the tags of their repository, moodle within its v4.5 series
- the three xwiki extensions read their Maven metadata
- pretix-oidc reads the PyPI simple index. The JSON API answers 404 behind
  the package cache the test stack routes pypi.org through
- joomla's plugin is built here and jellyfin's ldap-authentication is bound
  to the md5 of its archive; both are opted out with that reason

A bump rewrites the pin and the config.archive line together, and an
archive that does not carry the pin fails test_version_keys, so the two
cannot drift apart again. test_version_pins_are_watched covers addon pins
as well: a new one without a source or a reasoned opt-out fails the lint.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 02:50:48 +02:00
kevinveenbirkenbachandClaude Opus 5 81ab6e4ffc feat(update): let a version pin declare its own upstream source
The Docker updater watches image plus version, the repository updater
repository plus ref. A dozen pins fell outside both - an app_version, a
release, a tag in a registry neither speaks - so nothing ever told anyone
they had gone stale.

Such a pin now names its upstream next to itself in an `update:` block:
git_tags, registry_tags, npm, http_regex or a role-local script, with
`match` selecting a tag flavour and `strip` removing a prefix the pin does
not carry. utils/update/source.py resolves them beside the two existing
updaters, `python -m cli.contributing.update.source` bumps them, the
update-version-sources job opens one PR per role, and
tests/external/update/source warns while a bump is pending.

Two lints hold the rule: every semver pin is watched by one of the three
updaters or opted out with a reason, and a semver only ever sits behind
`version`, `<what>_version` or `ref` - which renames moodle's release to
app_version and simpleicons' local_version to local_tag, the latter being
the tag of a locally built image rather than an upstream version. A
declaration naming a key the entity does not carry, or a type the resolver
does not know, fails the lint instead of resolving nothing in silence.

Picked from feature/store-i18n (94d607fd0a) onto current main. Two things
differ from the original: svc-ai-lmstudio no longer carries an app_version
pin, so its declaration is dropped, and web-app-stalwart's webui_version,
added since, declares the git tags of the webui repository.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 02:50:48 +02:00
Alejandro Roman 4b9a834546 Merge remote-tracking branch 'upstream/main' into feature/role-stalwart-mail-swarm 2026-09-21 03:25:20 +04:00
kevinveenbirkenbachandClaude Opus 5 328113e7c3 refactor(ai): declare the litellm consumer contract once
Fourteen roles repeated the same two lines to say they use the gateway: a
`shared` flag gated on group_names, and an `enabled` flag conjoining that gate
with LITELLM_CHAT_MODEL_SERVED. Both are now single-sourced.

`shared` becomes literal true. The flag means the provider is a shared instance
rather than one deployed per consumer, and a central gateway is always that;
the dynamic form only said "shared if deployed", which `enabled` already
decides. The 50 shared pins under `litellm:` leave meta/variants.yml with it.
The tree had already noticed: 20 of those 50 lines carried
`# nocheck: variants-static-override`, two of them with copy-pasted text about
container_backup. This narrows e5de4e085a for one service key rather than
reversing it; every other dynamic flag keeps the group_names form.

`enabled` becomes `{{ LITELLM_USABLE }}`, defined in group_vars/all/16_ai.yml.

That shorthand is only possible because is_explicit_truth learns about it.
The resolver reads a dependency out of the flag TEXT — "in group_names" in
value — so a constant hides the role id and the provider silently leaves the
static closure of every role referencing it. Measured: the old expression
resolves True, `{{ LITELLM_USABLE }}` resolves False. GROUP_GATED_CONSTANTS
names the constants whose own definition carries the gate, and the docstring
states the obligation: a SPOT of this kind that is not listed there drops its
provider without a word.

The wider fix, accepting any Jinja as dynamic, was measured and rejected: 35
other flags carry a Jinja value without the marker (MCP sidecars, javascript
and SSO gates, recaptcha), and they would all enter closures they are not in
today.

A literal shared flag is no longer dynamic, so test_dynamic_flags wants the
`# nocheck: dynamic-flag` its own message documents for legitimate exceptions;
where a marker already existed the rule list is extended rather than a second
comment added.

Behaviour is unchanged where it counts. The closure keeps litellm with the same
app counts (wordpress 26, openwebui 23, hermes 23, mediawiki 22, flowise 23),
unrelated Jinja still resolves False, and the truth table of the new enabled
form matches the old one in all four served/deployed combinations. Three roles
resolve without litellm, and `git show HEAD:` proves that predates this change:
zammad pins enabled false in variant 0 at line 39, nextcloud at line 97.

Six unit tests cover the resolver, including one asserting that every
registered constant is actually defined in group_vars, so an entry cannot
become a dead promise.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 19:23:59 +02:00
kevinveenbirkenbachandClaude Opus 5 8114a7a2b1 fix(moodle): give the site administrator the administrator password
install_database.php was the only place that ever wrote the Moodle site
administrator password, and its failed_when tolerates "Database tables already
present". That is not laxity: upstream aborts in install_database.php before
install_cli_database() writes the password once tables exist, so the role
swallowed exactly the path in which the write is unreachable. After the
credential rotation gate Moodle therefore kept the install-time secret while
templates/playwright.env.j2 rendered the rotated one, and the login answered
"Invalid login, please try again". The same job proves it: variant-2/sync
reports failures="0" with the spec passing in 5.6s, variant-2/async
failures="1", and only the rotation gate lies between them.

The deeper problem was the credential itself. secrets.credentials.user_password
described itself as "Initial password for the Moodle admin user" - a role-local
secret for an account that is not role-local. Moodle hosts the platform
administrator, so read that user instead, the way web-app-baserow,
web-app-gitlab, web-app-penpot, web-app-matrix, web-app-bluesky and
web-app-pihole already do. lookup(users, administrator).password is stable
because roles/user-administrator/meta/users.yml pins it to
ansible_become_password; only a user with no declared password re-renders
{{ 42 | strong_password }} on every evaluation.

vars/main.yml now holds username, password and email as one constant each, so
install, realign and the Playwright env cannot drift apart again, and the
now-unread credential leaves meta/secrets.yml.

Realigning uses admin/cli/reset_password.php. Verified against the pinned
release v4.5.11: --username and --password are non-interactive (lines 35-37,
prompts only when empty at 76 and 88), --ignore-password-policy is required
because the platform password is not chosen to satisfy Moodle policy (line 96),
and the "when" gate is required because line 84 resolves only auth=manual rows
while tasks/04_oidc.yml rewrites every row to auth=oidc.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 12:10:54 +02:00
kevinveenbirkenbachandClaude Opus 5 1cd9578b83 fix(moodle): fetch the source from a host that answers the runners
download.moodle.org returns 403 to GitHub-hosted runners while serving the identical artefact elsewhere: 12 failures across three runners over twelve hours, each 40-66ms after TLS, with curl's four retries and Ansible's three exhausted. The GitHub tag archive is content-equivalent - both tarballs extracted and diffed over 39312 entries, the only delta being githash.php, which nothing in the 4.5.11 tree includes - and needs --strip-components=1 because it roots at moodle-4.5.11/ rather than moodle/.

auth-oidc.sh was the next landmine, two Dockerfile lines behind the fixed one: an anonymous api.github.com tag query, 60 requests per hour per runner IP, no retry, -s swallowing the rate-limit body under set -e, and a floating plugin version as the build input. It now reads a new meta/addons/auth_oidc.yml, which declares bridges: [sso] and update.monitored so the version is watched by the update pipeline instead of frozen.

MOODLE_BRANCH and MOODLE_TARBALL_FILENAME die with the old URL; the latter was already dead, demanded only by its own :?required guard.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 13:46:35 +02:00
kevinveenbirkenbachandClaude Opus 5 a680b1dd21 fix(mattermost,moodle): pin litellm to a round whose gateway serves a model
svc-ai-litellm has two variants: 0 with both backends on, 1 with both off. The planner assigns a dependency round_index when it has that many variants and falls back to 0 otherwise (inventory/planner.py:111), so round 1 is the one round in which the gateway deliberately serves nothing - a topology that is green in its own job.

Commit e0048f157b gated 14 consumers on LITELLM_CHAT_MODEL_SERVED so a consumer switches itself off when no model is served. Exactly two of them pinned litellm on in round 1, and a variant's literal beats the base Jinja gate, so mattermost rendered LITELLM_SERVICE_ENABLED=true against an empty model_list. The Agents plugin then failed client-side with 'bifrost error: model is required' 47ms after the prompt, never reaching the gateway.

Moving both pins to variant 2 lands them where the fallback supplies svc-ai-litellm=v0 with svc-ai-ollama=v0. Measured with the planner: round 2 pulls both. Both polarities stay pinned, so the variant-coverage lint still holds. The contract assert now also refuses an empty model, which the stored-vs-intended comparison read as converged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 13:25:27 +02:00
kevinveenbirkenbachandClaude Opus 5 e0048f157b fix(litellm): gate consumers on a served model, not on the gateway being deployed
The gateway publishes only models it has a backend for and renders model_list: [] when neither ollama nor lmstudio is deployed and no OpenRouter key is set. LITELLM_CHAT_MODEL fell back to openrouter/auto without checking that key, so consumers were configured for a model the gateway does not carry and every prompt failed with a 400.

Observed in the Mattermost#1 CI job: litellm.log carries 12 occurrences of "Invalid model name passed in model=openrouter/auto" next to 400 Bad Request, and tests/test-litellm-agent.js failed on every attempt while the rest of the suite passed.

Make the fallback conditional on the key so the name is empty when nothing is served, promote LITELLM_CHAT_MODEL_SERVED from the role vars to group_vars where consumers can see it (it was the third copy of this logic and the only correct one), and switch the 14 consumers from "litellm is deployed" to "litellm serves a model". The Playwright service flag reads the same lookup, so the spec skips together with the Ansible gate. The bool filter carries weight: the folded scalar yields the string False, which Jinja treats as truthy.

A new unit test renders both expressions against Jinja and pins the invariant that a model name exists exactly when a model is served, including the case where ollama preloads embedding models only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 19:48:32 +02:00
kevinveenbirkenbachandClaude Opus 5 346ad875c6 refactor(mcp): drop the two provenance fields nothing reads
supported_version and minimum_version sat in 24 roles and were read by no template, no task and no lookup. Their only consumer was the audit report version column, which turned false the moment an automated image bump ran: qdrant reported a verified version nobody had verified, two releases long. Both leave MCP_KEYS, so a role that declares them again fails the schema lint instead of carrying a field with no effect.

Requirement 025 mandated minimum_version as a MUST and 035 carried it in an example; both are corrected here, otherwise the repository would contradict its own code.

The gate then surfaced five dead docs.mattermost.com links. Mattermost moved to /administration-guide/ and /deployment-guide/, and every replacement was probed for 200 rather than guessed. The MCP page is gone with no successor, so mattermost now points at mcpserver/README.md in the Agents repository, the source rather than a rendered mirror.

Both concerns share three files - mattermost's meta/mcp.yml, requirement 025, and the generated audit.md, which derives from a source of each - so splitting them would commit an artifact that does not match its sources.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-03 19:31:08 +02:00
kevinveenbirkenbachandClaude Opus 5 4164f37981 fix(mcp): give every MCP round an identity provider to sign in with
An MCP client spec signs in through the client's own OIDC button, which
redirects to Keycloak. Every provider put its MCP pairing in the lean variant,
and that variant pinned sso off - so the round that exists to prove the pairing
was the one round without an identity provider. Run 33574056480 shows what that
costs: openwebui's MCP specs reached _shared.js:131 and died on "openwebui OIDC
sign-in button must be visible on the auth page".

The rule now demands two things of the variant that carries the consumers: the
provider pins sso on, and each admitted consumer, in the variant that round
actually pulls in, does not pin it off. The second half matters because a round
deploys every dependency at its own index and falls back to variant 0 only when
the dependency has fewer, so the client shipping alongside is that one and not
the client's variant 0. A role that declares no sso, or pins it to a literal
false because upstream speaks no OIDC, is exempt from a flag it cannot have.

Twenty roles moved to satisfy it. Ten already had a second sso: false pin and
only needed the flip. Six could host the pairing in variant 0, which already
has sso on, and dropped back to two variants. Six kept a third lean variant to
carry the false polarity variants-coverage wants. moodle and mattermost took
swaps instead: moodle's variant 1 is the mandated LDAP-only branch, and
mattermost was already at the three-variant cap with sso on in its variant 2.
nextcloud also turns context_agent on in that round, the ExApp that serves the
endpoint - with it off the round proved nothing it was built to prove.

The rule moves to tests/lint/ansible/services/ because it reads meta YAML and
executes nothing, which is where integration.md says such a check belongs, and
its suppression key joins the catalog it was never listed in.

nextcloud_exapp_reregister becomes nextcloud_exapp_renew: mcp-secret-masking
matches `register:` unanchored, so the substring in the old name read as a task
that prints a secret. register.yml already masks the two tasks that carry one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 15:51:24 +02:00
kevinveenbirkenbachandClaude Opus 5 5bf4d4d8ac refactor(variants)!: let an MCP round decide for itself what it carries
The mcp-consumer-variant guard demanded three things of a provider's MCP
round: every admitted consumer enabled, each of them pinned `services: null`
so it dragged no dependencies along, and no other service left enabled. Two
of those were the rule legislating a round's contents rather than checking
that the pairing gets proven at all.

Keep the first and drop the rest. What else the round carries is the role's
own call, so the tor/litellm exemption, the datastore exemption and the
`allowed` set they fed have no reader left and go with them.

Strip the 63 `services: null` pins the rule forced across 20 variants, all
written bare as `services:`. The topic-emptying capability behind them
stays: apply_topic still normalises None to {} for the inventory payload,
and the unit tests still cover it. Only the uses and the mandate are gone.

That leaves overridden_providers scanning a repository with nothing left to
find, so its three assertions passed over an empty dict and would have
stayed green however the function broke. Rebuild them on a synthetic role
tree, which also covers the case the repository scan never held: a variant
naming a plain flag is not an override.

Removing the pins changes what deploys. Measured across all 20 variants: 18
identical, prometheus#1 grows 23 to 31 roles, moodle#2 grows 31 to 32 and
crosses the memory budget until the following commit shrinks it back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 01:01:14 +02:00
kevinveenbirkenbachandClaude Opus 5 f1635e6a59 feat(variants): let a variant dictate the config of the providers it pulls in
A round deploys a provider once, so the role that pulls it in is the only place that can say how lean it has to be. A variant entry may now carry any application config topic under a service it depends on: a mapping deep-merges, so naming one key changes that key alone, and null empties the topic. Empty was the case that needed a marker of its own, because merging an empty mapping is a no-op and would leave exactly the services the round was trying to shed.

The topic vocabulary and the merge semantics live in utils.roles.applications.topics, which is also where resources.py now gets them rather than spelling the same null-or-merge out a second time. Collecting who demands what stays in the inventory tree, refuses two roles demanding different values for one provider in the same round, and skips a role's own entity entry so the existing services.<own>.users declarations are not read as overrides.

The CI set-cover learns the same fact. Its greedy cover retires a role's variant-0 row when an earlier row of another role embeds it, on the stated premise that embedding pulls the provider's variant 0. A variant that dictates that provider's config breaks the premise from the other side: whatever ran, it was not the provider as the provider declares it. Such a pair now covers nothing, so the retired row keeps its job, its artefact and its verdict.

Measured on the current tree: 20 override pairs, 41 covered rows before and after, so nothing is revoked today. The rule bites the moment an override lands in a row that covers.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 21:08:08 +02:00
kevinveenbirkenbachandClaude Opus 5 791eee7c9f refactor(mcp)!: prove the endpoint contract with a CLI test, not a deploy probe
The contract check ran as an Ansible task in the middle of every provider deploy, and the two client checks ran the same way. A test that is part of the deploy cannot be run without deploying, cannot be re-run on its own, and reports its verdict as a deploy failure. Twenty-seven task files lose 491 lines to this.

It is now what it always was. Each provider ships files/test/test.sh, templates/test.env.j2 and meta/tests.yml, and test-e2e-cli stages one shared harness beside it: run.sh drives contract.sh in a throwaway container on the provider's own network, then isolation.sh for adapter-fronted providers. The read probe each provider offers moved from the calling task into meta/mcp.yml next to the allowlist it must satisfy, and the schema now rejects a read probe naming a tool outside that allowlist.

The bearer the test presents is the one a client resolves through mcp.credential, via the new mcp_client_bearer lookup over resolve_credential, which moved to utils.roles.applications.mcp so it is the same function mcp_servers renders every client connection with. The probe presented whatever the calling role passed, which for the fifteen adapter-fronted providers is a different secret than any client holds.

Sidecar isolation is read from the engine rather than measured inside the container, so it no longer needs a delegation to the node holding the task: attachment is compared by network name, which drops the gwbridge special case, and a swarm worker says it cannot measure instead of skipping silently.

Retrying is bounded by wall clock against MCP_DEADLINE_SECONDS, which scales through the same lookup('timeout') as the harness budget it has to stay under. A flag that is neither true nor false aborts rather than reading as off.

mcp-provider-probe becomes mcp-provider-cli-test: an enabled provider must ship the test rather than include the task. mcp-probe-bearer becomes mcp-adapter-bearer, tying ADAPTER_BEARER to the declared credential key now that no probe include carries it. A new integration test renders the shared env template for every provider through Ansible's own templar, so a later change to it breaks visibly instead of poisoning twenty deploys quietly.

Hermes and OpenClaw keep their client-side proof as CLI tests of their own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 20:46:26 +02:00
kevinveenbirkenbachandClaude Opus 5 30e43cce88 refactor(services): state a refused pairing as off instead of gating it
83c1c729de gave every provider a group_names-gated services entry per MCP
client, including the clients it refuses with mcp_consumer: false. The gate
is what makes the inventory resolver plan provider and client into one round,
so 19 of the repo's 82 co-deploy pairs started a client for a pairing the
provider had already declined. flowise is a live MCP client of qdrant,
homeassistant and prometheus; the other 17 providers were bringing up its
node stack for nothing.

The entries now read enabled: false, shared: false, mcp_consumer: false, and
the variants that pinned them to true are gone. Setting those variants to
false instead is not an option: test_variants_no_static_override forbids any
variant override of a literal base value in either direction.

bond leaves with them, and a new lint holds that line: collect_role_resources
skips a literally disabled entry before it reads any resource key, so a bond
there describes a cost nobody pays. Thirteen entries unrelated to MCP carried
one and are cleared too.

test_mcp_consumer_services_entry demanded the group_names shape for every
client entry, refused ones included, and now demands the opposite there.
dynamic-flag keeps its rule and takes its own documented per-key exemption.

make docs redraws twenty graphs: the edges go, the nodes stay as
'openwebui ❌' and 'flowise ❌', so the refusal is visible rather than read as
an integration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 17:20:24 +02:00
kevinveenbirkenbachandClaude Opus 5 ffdef28da3 refactor(mcp)!: derive a surface's presence from its peers
enabled and shared were hand-maintained in 24 roles and wrong in both
directions. Both now resolve from the mcp_consumer edges already declared
in meta/services.yml: a server takes the consumers it admits, a client
takes the servers that admit it, and the pair renders as "'peer' in
group_names or ...", or False when no peer is admitted at all. setdefault
keeps meta/mcp.yml and meta/variants.yml as the override channel, so
pinning either key still wins.

The resolved block replaces the mapping instead of being written into it.
_build_role_base_config parks the YAML cache's own object under mcp, so an
in-place setdefault injected the derived keys into every later raw read and
made test_mcp_schema pass alone but fail in a directory run.

The derivation keys on peer presence, never on a peer's own value: a
re-entrant read under _RENDER_GUARD returns the unrendered dict.

Four lints now ask the resolver instead of reading meta/mcp.yml, which they
could not do while the key was hand-written. That is what exposed the eleven
missing hostile-path specs the previous commit closed.

Carries one change that belongs to the WordPress probe but shares a file
with this one: the tool allowlist moves to the hyphenated names the adapter
actually serves.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 16:13:01 +02:00
kevinveenbirkenbachandClaude Opus 5 83c1c729de refactor(mcp)!: derive admission from the client
allowed_consumers was twenty hand-kept lists of which fourteen were identical, so every new client meant an edit in every provider. A client now declares services.<own-entity>.mcp_consumer: true once and a provider refuses one with mcp_consumer: false, which is also what tells a deliberate exclusion apart from a forgotten entry.

The group_names-gated consumer entries give the inventory resolver the co-deploy edge meta/mcp.yml never had: one of sixty-nine matrix rounds carried an MCP client before, twenty-three role/round pairs do now. bond 0.5 keeps the clients out of round 0 so the host memory budget still holds.

BREAKING CHANGE: meta/mcp.yml no longer accepts allowed_consumers; declare services.<entity>.mcp_consumer on the client instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 18:33:22 +02:00
Alejandro Roman 333079cc2a feat(UI): fix migration, errors happening in local deploy and swarm integraton + merged main 2026-08-30 21:39:52 +04:00
kevinveenbirkenbachandClaude Opus 5 7162b7bf4c Merge fork/feature/svc-net-tor into feature/ai-agent-employees
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.

The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.

Resolutions that were not unions:

Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.

plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.

compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.

tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.

Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.

01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.

Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.

roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.

config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.

READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.

Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 16:32:30 +02:00
kevinveenbirkenbachandClaude Opus 5 b23653adf8 feat(moodle): point the AI subsystem at the platform gateway
Moodle 4.5 ships aiprovider_openai in core, where every action carries its own
endpoint rather than one shared base URL, so the provider is configured per
action. aiprovider_openaicompatible would be the natural fit but requires
Moodle 5.0, which the role does not pin, so the per-action endpoints are the
only surface available on this branch. MOODLE_LITELLM_SETTINGS carries that as
a list of component/name/value triples and also enables the courseassist and
editor placements: a configured provider with no enabled placement exposes no
AI surface to any user, so both halves are required for the feature to exist.
An earlier revision set a global `enableai`, which does not exist in 4.5.

tasks/utils/ai_gateway.yml verifies the shared gateway contract, writes each
setting through admin/cli/cfg.php, then purges caches. Without the purge Moodle
keeps answering from its config cache and the new endpoint appears to have been
ignored.

The egress rule is the half that is easy to miss. Moodle's cURL security helper
blocks private address ranges outright, so a correctly configured in-cluster
endpoint is still refused, and it surfaces as a generic provider error rather
than as a blocked request. 01_manager_ops.yml therefore writes
curlsecurityblockedhosts and curlsecurityallowedport, appending the gateway's
port only while litellm is enabled. This deliberately narrows a security
control: 172.16/12 and 192.168/16 become reachable so the container network
resolves, while 10.0.0.0/8, link-local and loopback stay blocked. The README
records the trade-off.

MOODLE_EGRESS_NEWLINE is double-quoted on purpose. Moodle stores both settings
one entry per line; a single-quoted YAML scalar hands Jinja a literal
backslash-n, the whole list collapses into one unparseable token, and the net
effect is that every port is blocked rather than the intended two. Verified in
the container with negative controls: keycloak and litellm reachable,
10.208.0.5 and 127.0.0.1 still refused.

The spec opens a site-administrator session -- through Keycloak when sso is on,
because 04_oidc.yml rewrites every account to auth: oidc and no password
reaches it then, and through the native form otherwise -- asserts the endpoint
is http on an undotted host that is not Moodle's own domain, then calls
aiplacement_editor_generate_text and requires a non-empty completion. That last
step is a prompt actually answered through the gateway, not a config read-back.

Verified: the deploy-time read-back assertion ran ok in a variant-2 round and
the spec executed rather than skipping. The egress newline defect and its fix
were both measured in-container.
Not verified: I confirmed the spec ran but not that its final assertion passed,
and the cfg.php loop is changed_when: false, so the deploy log cannot show
whether a setting actually moved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 13:21:00 +02:00
kevinveenbirkenbachandClaude Opus 5 596b10c403 test(lint/csp): demand reason for unsafe-* grants
unsafe-inline re-admits the inline execution an XSS needs and unsafe-eval re-admits string-to-code, so a CSP relaxation must be verified against the running app rather than copied from another role. The new lint requires a nocheck marker plus a Reason on the flag's own line, so a marker cannot drift to the neighbouring directive.

The 197 pre-existing grants across 70 roles are checked out with a TODO: they were never verified and still have to be.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 23:28:14 +02:00
Alejandro Roman 7332bd66ae Merge upstream/main into feature/role-stalwart-mail-swarm 2026-08-18 01:42:23 +04:00
kevinveenbirkenbachandClaude Opus 5 e16eb5728d test(lint): drop the minimal variant's exemption from the onion
72fd9c870 required the onion in every variant but exempted variant 1, on the
argument that the minimal round is the one place the stack runs without it. That
argument does not hold: the onion is part of the deployment under test, not a
feature bolted onto the full shape. A round that skips it is not a smaller
deployment, it is a different one.

So the exemption is gone and 95 variant-1 entries pin the onion on. Two
exclusions remain, both read from data rather than a comment: a role with no tor
block has no onion to pin, and a role whose meta/services.yml pins tor.enabled to
a literal false has already declared it has none.

That put the rule in conflict with test_coverage, which requires both polarities
of every dynamic flag: with tor on everywhere, the false side has no variant left
to live in. 94 roles take the exemption that contract already provides for
exactly this - "flags whose group-derived value is a hard invariant that variants
must never override", the same one 4e6ef2542 used for the seaweedfs frontend.
web-app-nextcloud needs none: its variant 8 pairs the clearnet-only PeerTube and
keeps the false polarity.

Measured, because the declaration is not what CI runs. assign_tor gives the onion
to every second capable entry with the run number as the parity switch, so more
capable variants do not mean proportionally more onion jobs: across five roles
and 21 entries the count goes 8 -> 10 on an even run and 9 on an odd one. The
clearnet path also survives - checkmk reports v1=false on run 0 and v1=true on
run 1, so both states are still covered across two consecutive runs. What the
94 markers give up is a declarative guarantee in variants.yml, not the coverage
itself.

The real cost is weight, not job count. web-app-nextcloud variant 1 goes from
3.09GB reservation / 7.27GB limit to 18.26GB / 43.01GB once svc-net-tor and its
closure join, so the cheap round is no longer cheap. Every variant stays under
the 32/64GB budget, and the same effect applies to the other 94 unmeasured.

Verified: tests/lint 336 passed / 307 subtests, tests/integration/roles/meta/
variants plus the role-meta lints 39 passed. The rule was checked in both
directions - it reported the variant-1 offenders before the flips and none after.

Not measured: what the heavier rounds do to wall-clock, against swarm jobs capped
at 355 minutes and one that has hit that ceiling before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 06:09:02 +02:00
kevinveenbirkenbachandClaude Opus 5 ab72cff929 test(lint): require the onion in every variant, not just SSO ones
7a70d27b7 made a variant that switches SSO on also switch the onion on, because
that pairing is where the platform breaks in ways clearnet never shows. The same
argument reaches further than SSO: a Secure cookie is dropped, an https identity
provider refuses to embed, and crypto.subtle is undefined for any feature on a
non-secure origin, not only for a login. A variant that never runs the onion is
not testing the deployment users get.

So the rule now covers every variant of a role that declares a tor block.
Variant 1 stays exempt - it is the minimal, partner-free round every role
declares and the one place the stack is exercised without the onion at all - and
a role whose meta/services.yml pins tor.enabled to a literal false is skipped
outright, because that declaration already says the role has no onion and needs
no comment repeating it.

That found 137 variants. 101 were variant 1. Of the rest, 28 are the LDAP-only
auth-isolation rounds, which gain real coverage from running LDAP over the onion
and are flipped on here. Four cannot: svc-net-tor is the provider itself, and
nextcloud's variant 8 pairs the clearnet-only PeerTube. Both carry a marker
naming the reason; opentalk and coturn need none, since their services.yml pins
the literal false the rule now reads.

Flipping them exposed a contract that contradicted itself. test_auth_isolation
and test_non_baseline_explicit_disables demanded `shared: false` beside
`enabled: false`, while variants-static-override flagged that same line as a
static override - which is why all 133 of them carried a nocheck marker. The
demand only makes sense where `shared` is dynamic: a literal base value cannot
leak through deep-merge, so restating it protects nothing. Both contracts now
ask for it only in that case, and the 133 lines are gone.

The result is a stricter rule with fewer lines: 189 insertions against 193
deletions.

Verified: tests/lint 336 passed / 307 subtests, tests/integration/roles 107
passed / 610 subtests. The rule is checked in both directions - it reported 137
offenders before the resolution and none after.

Not measured: 28 additional onion rounds cost matrix wall-clock, against swarm
jobs already capped at 355 minutes and one that has hit that ceiling before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 06:09:02 +02:00
kevinveenbirkenbachandClaude Opus 5 b2a0a317fa refactor(tor): loosen the onion bond to 0.5
Every variants.yml header states the rule the bond feeds: variant 0 carries each
tightly-coupled (bond>=1, same-host) service, while loosely-coupled (bond<1,
separate-host) partners are excluded from the closure and distributed across
later variants to keep every host under budget. tor was declared bond: 1 in all
101 consumers, so it had to sit in every variant-0 closure.

It does not belong there. The onion is a separate-host provider reached over the
node's hidden service, the same shape as the bridged partners that already carry
0.5, and f1dff50fb established both the reasoning and the precedent when it
dropped xwiki and bigbluebutton from 1 to 0.5: bridged partners with their own
role and their own spec, whose bond declaration was the outlier rather than
their placement.

This also unblocks the variant-level tor contract. Requiring a non-baseline
variant to pin the onion on while tor still counted as tightly coupled would
have pulled variant 0's closure requirements along with it.

Verified: tests/lint 336 passed / 307 subtests and tests/integration/roles 107
passed / 610 subtests, including test_resource_budget over every variant's
deduplicated footprint and the bond-aware test_order closure.

Not measured, and the honest cost: f1dff50fb's own caveat applies unchanged
here. Whether redistributing a service out of the baseline closure buys back
wall-clock is not established; role-runtimes has the per-role seconds and no run
has been made with this shape.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 06:09:01 +02:00
kevinveenbirkenbachandClaude Opus 5 0c2ce12c7a feat(healthcheck): compose a probe from prefixes, and gate Moodle's mail on it
Moodle renders the same msmtprc as WordPress and mounts it into two services,
but nothing ever proved a message left the container: its healthcheck opens the
FPM port and nothing else. Moodle 0 of run 31658241728 passed green while mail
could not have worked -- the defect 2da285067 fixed for WordPress was invisible
here, and would have stayed invisible until a user reset a password.

WordPress proves delivery with the msmtp_curl flavor: the probe sends one mail
and writes a marker, and a task after convergence waits for that marker. Moodle
cannot use it, because curl cannot judge a port that answers FastCGI. Rather
than add a second msmtp_* class per liveness shape, the declaration now composes:

    healthcheck:
      flavor:
        - msmtp
        - connect
      port_key: fpm

msmtp stops being a flavor and becomes a prefix -- shell that runs on the way
past and never decides liveness. connect joins the probes as the one that only
asks whether the port accepts a TCP connection. A declaration names exactly one
probe and any number of prefixes, as a string or a list; msmtp_curl stays as an
alias for [msmtp, curl], and the lint that validates declarations now shares
resolve_flavors with the lookup instead of keeping its own idea of a flavor.

The module became five responsibilities in one file, so it is a package:
probes.py answers which probes exist, prefixes.py what runs before them,
compose.py how a declaration becomes a rendered block. The curl helpers sit with
the probes and mail_branch with the prefix -- each next to its only caller,
rather than together in a module header.

Two defects in this work were found by measuring, not reading:

  - the timing merge took the maximum across prefixes only, which cut Moodle's
    start period from 15m to 120s and would have marked a heavy PHP app dead
    while it was still installing. A prefix may now raise a timing and never
    lower one, pinned by test.
  - composing moved msmtp_curl's interval from 30s to 1m, because the composed
    class derives from Curl where MsmtpCurl derived from Probe. That is the
    interval every other curl probe uses, but it was not intended; it now has
    its own test rather than riding along unseen.

Regression measured across the whole repo rather than argued: every declared
healthcheck was rendered with the pre-refactor code and with this one. 58 render
byte-identically, 1 is Moodle's intentionally changed declaration, and 2 differ
-- friendica and wordpress, in that interval alone, with the probe command
itself unchanged.

Coverage was checked in both directions. No test was removed: the four deleted
lines are an import, the registry literal, one class name in a url test, and the
helper, which now goes through compose() instead of reaching into PROBES -- a
tightening. New code had two gaps, both closed: connect was the only probe whose
emitted command no test pinned, and compose() raised KeyError for an unknown
name unobserved.

Not verified: no deploy has run any of these probes. The composed class is built
dynamically, so what it emits is proven for all 59 services while docker
accepting it is not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 02:02:31 +02:00
kevinveenbirkenbachandClaude Opus 5 750476b26a feat(lint): give every mount target one spot in meta/volumes.yml
A volume that reaches several services repeated its container path once per
mount. Written out each time it has no single point of truth: a rename that
misses one line still parses, still deploys, and mounts one service somewhere
nobody intended. Nothing fails until that service looks for content which is
not there.

YAML already solves this, and 62722a6ab used it here first -- web-app-openproject
anchors await_migrations_target and aliases it for the second service. Nineteen
roles now follow, and tests/lint/repository/spot/test_volumes_target_single_spot.py
fails on any target spelled out twice. An alias carries no path, so the rule
cannot see it, which is the point.

The conversion was mechanical and is provable rather than reviewed: every file
was rewritten and then reloaded, and any file whose yaml.safe_load result
differed from the original was discarded instead of kept. Nineteen written,
none discarded. Anchors and aliases are a serialisation form; the loader
resolves them before any consumer sees a document.

validate_volumes_meta over all 81 meta/volumes.yml reports no violation.
make test-lint: 328 passed.

web-app-prometheus carries two unrelated deletions: comments that predate this
change and fell under the comment lint once the file was touched. They explained
a 0444 mode and a compose-only bind; both facts are visible in the entries
themselves.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 02:02:29 +02:00
kevinveenbirkenbachandClaude Opus 5 0be527abb0 fix(web-app-moodle): give the local cache a node-local home
Moodle kept everything under dataroot, which is the shared NFS export in swarm
mode - including localcachedir. Upstream is explicit that this is the one
directory that must not be shared: config-dist.php marks tempdir and cachedir
as 'MUST BE SHARED by all cluster nodes' and localcachedir as 'Intended for
local node caching'.

The path is declared once, as the mount target in meta/volumes.yml, and every
other place derives from it: vars/main.yml reads it back through
lookup('volume', ...), compose passes it as a build argument, the Dockerfile
exports it, the entrypoint requires it with :? and creates it with the runtime
owner, env.j2 hands it to config.php as MOODLE_LOCALCACHEDIR. The two mounts of
the same volume share one YAML anchor rather than repeating the literal.

The cron service mounts it too, because it runs the same code and would
otherwise write its cache somewhere else than the web service reads it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 22:05:43 +02:00
kevinveenbirkenbachandClaude Opus 5 28475455db feat(mcp): give every provider its own service account from the registry
Nine roles reached their MCP surface as the deployment administrator or with
an account password kept beside the account in credentials. A Moodle site
admin bypasses every capability check, a Nextcloud app password is not
scopable, a Gitea token reads every private repository, and a GitLab token on
root reads every private project. Each role now declares its own principal in
meta/users.yml and provisions it, so the exposed surface is what the service
role permits. jellyfin, zammad and snipe-it kept their account but took its
password from a credential named token; they read the registry too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 15:44:12 +02:00
kevinveenbirkenbachandClaude Opus 5 ef5ea7cb55 refactor(roles): name the credential file meta/secrets.yml
meta/schema.yml held credential definitions and their runtime values, but the
name said 'schema' and read like structural metadata next to services.yml and
volumes.yml. Nothing in the file is a schema in the sense the other meta files
use the word, and a contributor had to open it to learn it holds secrets.

62 role files renamed. ROLE_FILE_META_SCHEMA becomes ROLE_FILE_META_SECRETS and
keeps its role as the SPOT the path is built from.

Two readers bypassed that constant and spelled the name themselves:
utils/cache/applications.py built meta_dir / 'schema.yml' twice, and
plugins/lookup/application_gid.py carries the filename in the marker list it
detects application roles by. Left behind, the rename would not have raised —
credentials would have resolved empty and the role would have stopped counting
as an application.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 12:08:06 +02:00
kevinveenbirkenbach 06bc856c32 refactor(mcp): make MCP a role topic instead of a service entry
The block sat under services but never behaved like a service. A service has
exactly one providing role, declared with provides, so the registry can map the
key to that role and consumers bind their flag to its presence. No role declares
provides: mcp. There are seventeen providers and the pairing is done by
allowed_consumers inside the block itself.

Every rule that enforces the service model therefore needed an exemption for it:
a shared-provider suppression in fourteen roles, a playwright-service-flag
suppression, and two freshly added exemptions in the dynamic-flag and
variants-static-override checks. Four independent rules, four escape hatches,
one cause. They are all gone now, together with a bond key that only the
resource budgeter reads and only for entries under services, and a wordpress
addon bridge that pointed at a service block that no longer exists.

The block moves to meta/mcp.yml. The applications loader maps meta/<name>.yml to
applications.<app>.<name> generically, so the move needs no wiring; the file
takes the same shape as meta/rbac.yml. Every config path follows, the variant
overrides are lifted out of their services map, and the vocabulary module leaves
the services package because it no longer describes one. roles_with_service
gains an explicit topic parameter rather than an implicit fallback, so nothing
silently resolves against the wrong container.

Files that are otherwise unrelated to this move are included where the committed
tree would otherwise keep pointing at a path that no longer resolves: thirteen
vars files, eleven playwright env templates and one task file.
2026-08-06 20:36:00 +02:00
kevinveenbirkenbach 10de64ff78 fix(networks): give MCP overlay membership a direction
A provider overlay keyed on services.mcp.enabled and .shared admitted every
MCP-capable role, servers included, because those flags carry no direction. Each
provider's network therefore collected every other provider and the result was a
mesh in which any sidecar could reach any application. The new mcp_client
consumer kind additionally requires services.mcp.direction to be client or both,
and the four provider overlays switch to it.

A sidecar rendered its networks with the same lookup as its provider and
inherited that provider's consumer attachments, which it neither needs nor
should have. own_network_only keeps it on the provider network alone. A provider
whose own topology is default_net contributes no attachment line at all, and a
bare networks: key is not a list, so the flag returns nothing in that case and
lets the sidecar fall back to the project default, which is that provider's own
network.
2026-08-06 14:27:07 +02:00
kevinveenbirkenbachandClaude Opus 5 e355bbdb8b feat(mcp): gate Open WebUI tool servers on the serving role's mcp group
Eight server roles declare an `mcp` RBAC role, so the platform mints
roles/<app>/mcp and the administrator joins it. Open WebUI now honours those
groups instead of flattening them to user/admin, and each MCP tool server is
restricted to the group of the role that serves it.

The grant cannot be rendered: Open WebUI matches it against a group id that
insert_new_group mints as a fresh UUID, so tasks/01_mcp.yml resolves the group
after the container is up and writes the grant through the admin API, the way
every instance-minted credential in this repository is provisioned.

Entries render disabled. An empty access_grants is not 'nobody', Open WebUI
reads it as every administrator, and ENABLE_PERSISTENT_CONFIG=false makes the
env authoritative again on every start. Enabling only together with the grant
keeps the fallback at no access rather than too much.

The provisioning account is the role-internal openwebui-api-bot, never a
platform identity: OAUTH_MERGE_ACCOUNTS_BY_EMAIL ships disabled and an OIDC
login whose email a local account already holds is refused, so seeding a
person's address would lock them out.

Known limits are documented rather than claimed away: a bare restart leaves MCP
off until the next deploy, a user's roles list carries no application context so
roles: [mcp] grants every deployed mcp group at once, and Open WebUI drops a
stale membership only while the groups claim is non-empty.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 22:06:47 +02:00
kevinveenbirkenbachandClaude Opus 5 f2fe5e94b1 feat(moodle): serve MCP through the build-time webservice plugin
Install the protocol plugin during the image build, because a Moodle
plugin must match the pinned release and installing it at runtime would
leave the two free to drift.

Provision the webservice and its token through PHP, and attach only
read-only functions, so the read-only tool policy is enforced by the
service definition rather than declared in metadata.

Address the endpoint as its canonical origin. Moodle compares every
request against wwwroot, so a probe over the internal HTTP origin was
answered with an HTML redirect instead of JSON and every retry hit the
same wall; the verify step now sends the canonical Host and the
forwarded-proto header.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 22:21:04 +02:00
kevinveenbirkenbach e04c448cdd Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Three conflicts, none resolved by picking a side wholesale:

- Makefile: took the incoming ACT_INPUTS, which routes through SWARM_DISTROS
  instead of inlining the same expression a second time.
- test-e2e-playwright/02_run_one.yml: took the incoming rc handling. An absent
  rc now counts as failure; the previous default(0) let an aborted Playwright
  run pass silently, contradicting the debug message right above it.
- web-app-peertube spec: combined both. Kept gotoOnion and resolveTimeout,
  which this branch's lint enforces, and adopted the incoming retry-inside-poll
  for the OIDC button plus its networkidle wait.
2026-07-30 15:38:15 +02:00
kevinveenbirkenbachandClaude Opus 5 028f4b85f4 chore(storage): set every image-backed min_storage to its image size plus 200MB
The declared values were guesses that ran up to 60x the size of the image they belong to: seaweedfs asked for 30GB against a 104MB image, nextcloud for 20GB against 353MB. Because min_storage feeds the storage guard, the runner packer and the sufficient-storage filter, those numbers made hosts look too small and pruned caches that deploys did not need to lose.

Every service that declares min_storage together with image and version now carries the measured compressed transfer size of that image plus 200MB of headroom. Sizes were read from the registries through utils.docker.registry.manifest_transfer_size, the same path the headroom test uses, so the declarations and the gate agree by construction. Values are written in MB throughout, matching the existing 500MB / 100MB / 50MB style.

119 declarations across 89 roles. Range: 206MB (memcached) to 3473MB (ollama); four exceed 1GB. The 84 min_storage entries that name no image are untouched, since there is nothing to measure them against.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 15:08:51 +02:00
Alejandro Roman 871b00670a Merge kevinveenbirkenbach/feature/swarm-nfs into stalwart integration branch 2026-07-27 11:25:42 +02:00
kevinveenbirkenbach 1a0af0ce09 Merge remote-tracking branch 'fork/feature/swarm-nfs' into feature/svc-net-tor
# Conflicts:
#	roles/sys-ctl-mtn-cert-deploy/tasks/main.yml
#	roles/sys-svc-compose-ca/tasks/main.yml
#	roles/web-app-dashboard/templates/javascript/oidc.js.j2
#	roles/web-app-mastodon/tasks/01_overrides.yml
#	roles/web-app-peertube/templates/compose.yml.j2
#	scripts/github/guide/compose_deploy.sh
#	scripts/github/guide/host_deploy.sh
#	scripts/make/alias.sh
#	tests/lint/ansible/roles/test_service_core_first_task_run_once.py
2026-07-24 04:30:56 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0a3570ecea feat(lint): require replicas:1 for scheduler services
A scheduler (cron, clock, celery beat, a framework scheduler process) enqueues
recurring jobs; running more than one replica fires the same jobs each tick,
causing duplicate work, lock contention, and unique-index collisions (e.g.
GoodJob's (cron_key, cron_at)). On swarm the default replica count is
len(hosts), so a scheduler with no explicit replicas silently fans out one per
node. Worker services (sidekiq, worker) are excluded: they scale horizontally.

Add tests/lint/ansible/services/test_scheduler_single_replica.py to enforce
replicas: 1 on any service whose key ends in cron/scheduler/beat/clock, with a
nocheck opt-out. Fix the two offenders it found, moodle cron and zammad
scheduler, and justify openproject cron with the required single-replica nocheck.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-24 04:06:39 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e99d8a8813 feat(cdn): serve role frontend dependencies from the internal CDN
Add lookup('asset', app, pkg, path) and lookup('asset_host'): both resolve
the asset origin through one shared resolve_host, returning the internal
CDN domain when web-svc-cdn is deployed with flavor internal and
cdn.jsdelivr.net otherwise, so template URLs and CSP origins can never
diverge. Versions come from the role's committed files/package-lock.json.

Roles declare frontend dependencies in files/package.json; web-svc-cdn
aggregates every role lockfile, runs one npm ci per role in a node
container and mirrors the packages under /npm/<pkg>@<version>/ following
jsdelivr's path scheme, so internal and external URLs differ only by host.

Convert web-app-dashboard (keycloak-js 24.0.5 UMD) and web-app-littlejs
(bootstrap, fontawesome) to the asset lookup, drop the vendored keycloak
blob, and add a repository lint forbidding hardcoded external CDN hosts
in templates and CSP metas; pre-existing external CDN entries are checked
out with nocheck markers and a migration TODO.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:31 +02:00
Alejandro Roman ac101d2efa Merge kevinveenbirkenbach/feature/swarm-nfs into stalwart integration branch 2026-07-21 10:00:17 +02:00
kevinveenbirkenbachandClaude Fable 5 d88f272f9f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 15:31:08 +02:00
kevinveenbirkenbachandClaude Fable 5 194671d90f feat(readme): uniform description schema and clearer role titles
Rewrite all 254 role descriptions to a uniform noun-phrase schema
(function-first, no leading verb, no trailing period, <=120 chars) and
enforce it via the extended description lint. Rename 12 role README H1
titles for clarity (e.g. Backup Container to Local -> Backup Docker
Volumes, CSS Service -> Corporate Design, NGINX Legal -> Imprint) and
regenerate their Cosmos and Quick Setup sections. Overview table
regenerated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 01:11:48 +02:00
kevinveenbirkenbachandClaude Fable 5 da598a2d67 feat(readme): roles overview table with homepage/video backfill
Add cli.build.readme.overview (make readme-index / make docs) rendering a
roles overview in the root README: Name, Lifecycle, Homepage, Video,
Description, Integrated with, Install. New info-media lint requires
homepage + video in every invokable role's meta/info.yml (opt out via
'# nocheck: info-media' in meta/main.yml); backfilled 147 roles with
researched, adversarially-verified homepages and official videos.

Retire the 'unsupported' lifecycle value: fold it into 'eol' as a valid
live end-of-life status (shipped but untested/unmaintained) across the
stage lists, symbol glossary, allow-list lint and docs. Retag confluence,
jira, minio, phpldapadmin to eol and magento to pre-alpha.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 19:10:01 +02:00
kevinveenbirkenbachandClaude Opus 4.8 10a218ed6e fix(nfs): correct volume ownership for non-root NFS-backed containers
Per-volume nfs uid/gid/mode (erpnext/frappe 1000, jenkins 1000, moodle/www-data 33, ejabberd 9000) so NFS-backed named volumes get the container runtime owner. nfs_prep falls back to default(omit) instead of re-forcing root:0755, which reverted the docker copy-up ownership for non-root images on every redeploy. compose_volumes test asserts the x-infinito-nfs marker.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 14:44:34 +02:00