Commit Graph
606 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 5b8465260f feat(ci): deploy only the variants a change reaches
An image bump in a provider redeployed every variant of every consumer,
including the variants that switch that provider off and therefore never
deploy the bumped image at all. The diff-derived whitelist now narrows each
affected role to the variants whose own round closure still reaches a seed,
emitted as `role#0,2` selection tokens. That closure is walked on the same
variant-merged services maps the matrix planner builds its rounds from, so the
narrowing follows a parent's variant choice across every generation of the
chain. It covers every auto-derived whitelist: the weekly update PRs,
Dependabot, and any other diff-driven run.

The narrowing never shrinks the matrix on partial information. A seed role
keeps every variant, because its own files changed. A role reached only
through a run_after edge keeps every variant too, because the round closure
does not follow that edge and a role whose hits are empty is therefore
unnarrowed rather than dropped.

A pinned token naming a role the discovery query does not return now warns
instead of aborting the run. The diff-derived whitelist legitimately names
roles the lifecycle envelope filters out, and once it pins variants such a
token would abort every chunk; role names are still checked against roles/
before the run starts, so a typo is caught one layer earlier.

The round's variant merge moves out of the deploy inventory package into
utils/roles/applications/variants.py. The CI selection has to build the same
maps, and importing the deploy package for them pulls in a module that reads
INFINITO_SRC_DIR at import time, which the runner host does not set.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-21 22:51:03 +02:00
kevinveenbirkenbachandClaude Opus 5 f88aea5473 docs(mcp): regenerate the audit report for the baserow bump
test_the_committed_report_is_freshly_generated compares the committed table
against a fresh render, and baserow had moved to 2.3.4 in its services.yml
while the table still read 2.3.3.

The bump arrived from the image-version updater, which edits services.yml and
nothing else. The report is derived from those files but is committed, so every
version bump for an MCP-serving role leaves it stale until someone runs
`make mcp-audit`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-21 18:25:12 +02:00
Alejandro Roman ace4c3ff2c fix(docs): regenerate MCP audit report after image bumps 2026-09-21 12:32:35 +04:00
Alejandro Roman 4b9a834546 Merge remote-tracking branch 'upstream/main' into feature/role-stalwart-mail-swarm 2026-09-21 03:25:20 +04:00
kevinveenbirkenbachandClaude Opus 5 55b5a3dd93 fix(fider): run the build that reads the private-network opt-in
0d1ba1777f set ALLOW_PRIVATE_NETWORK_TARGETS=true in July with the right
diagnosis: Fider's SSRF guard refuses the OAuth token exchange when the
Keycloak host resolves to an internal address, so login 500s and never
completes. The image pin stayed at v0.36.0, which does not read that variable,
so the fix has been inert ever since.

Read out of the binaries rather than the release notes: `stable` contains the
string ALLOW_PRIVATE_NETWORK_TARGETS, v0.36.0 contains it zero times. Upstream
added the opt-in on 2026-06-24 and the check now returns before the hostname is
resolved, which is what puts the OAuth token path under it as well.

CI showed this as three red specs on Fider#0, admin sso login, biber sso login
and the seaweedfs tenant-logo upload, all with a missing .c-menu-user. The
artifact says why the selector is missing: the page is Fider's own 500 screen,
and fider.log carries the cause four times over, `Token URL is not allowed:
This URL is not allowed because it targets a private or internal network
address (app/services/oauth/oauth.go:320)`. The selector was right; there was
no session to render.

The tag is `stable` and not a version because upstream published v0.36.1 on
GitHub on 2026-07-03 and never pushed the matching image. Docker Hub's newest
release tag remains v0.36.0, and `stable` was rebuilt three days after that
release, carrying the opt-in. The marker on the line records both that and the
condition for removing it.

This is a moving tag in a repository that has just finished pinning fifteen of
them, and the mirror copies by tag, so an existing copy of `stable` can go
quietly stale. That is the accepted cost of the only build upstream ships with
the fix, and the TODO is what closes it.

audit.md moves with it: the report is generated from the pins and the lint
fails when the committed copy differs from a fresh generation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 10:52:02 +02:00
kevinveenbirkenbachandClaude Opus 5 c31c62efaf fix(images): install the curl the slim base dropped under the healthcheck
10c5b35e88 moved simpleicons and roulette-wheel from node:26-bookworm to
node:26-slim. Both declare healthcheck flavor curl, which renders
CMD curl -f <url>, and neither Dockerfile installed anything. Measured with
docker create and docker cp: node:26-bookworm carries /usr/bin/curl at 280800
bytes, node:26-slim carries neither curl nor wget. The probe therefore fails
from the first beat and the container never reports healthy.

sys-svc-compose waits on the database container alone in compose and on every
stack task in swarm, so the consequence splits by mode. Under swarm the task
dies with non-zero exit (137): dockerexec: unhealthy container, is rescheduled,
dies again, and the converge wait fails after 600 attempts. Five jobs across
the two runs on a534d5d46 died that way, and none of them on the role they are
named after: ERPNext#0, Decidim#0, Hugo#0, Nextcloud#9 and Nextcloud#12 all
carry web-app-dashboard in their closure, dashboard enables simpleicons, and
the failure reads 'not converged: simpleicons_simpleicons'.

All five are swarm and tor, which reads like the tor axis until XWiki#1 is put
beside them: swarm, tor, green, and simpleicons is not in its closure. The
discriminator is the service, not the axis.

Of the seven roles 10c5b35e88 rebased, bridgy-fed and postmarks probe over tcp,
and flowise and bookwyrm install curl themselves. These two were the remainder.
roulette-wheel had not been deployed yet when this was written, so it is the
same defect caught before it was observed. The install goes into the runtime
stage of the multi-stage build; the builder stage would have left the runtime
image unchanged and nothing would have gone red.

The lint reads the binary out of the probe the declaration renders rather than
restating a flavor-to-binary table, so a flavor that changes its command moves
the check with it, and it locates the flavor line per service so one marker
cannot silence a sibling. Its scope stops at slim: nginx:1.31.6-alpine was
measured too and ships curl as a real 272 KB binary plus busybox wget, so the
four alpine roles that declare those flavors are served by their base, and
widening the rule to them needs that measurement per image rather than an
assumption from the word alpine.

Not built. The image with the new layer has not been produced, and whether a
second fault waits behind the healthcheck is decided by the next deploy. The
apt path itself is not blind: BookWyrm#1 installs curl the same way on
python:3.11-slim and is green in this run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 03:18:37 +02:00
kevinveenbirkenbachandClaude Opus 5 c693f4f619 style(quality): apply what the gate wrote back into the tree
Nothing here was typed by hand. The docs generator rebuilt the MCP audit
table against pins that had already landed, so fider reads v0.36.0 instead
of stable, jenkins 2.568.3-lts instead of lts, pretix 2026.7.0 instead of
stable, and flowise 24-slim instead of 24-bookworm-slim. That is the table
catching up with da665ab69 and 10c5b35e8, reading the same declaration the
updater reads.

Autoformat wrapped three test files from the same series: one condition
pulled onto a single line, three assertEqual calls broken across lines.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 19:35:05 +02:00
kevinveenbirkenbachandClaude Opus 5 66be8f6a0a test(lint): refuse a Debian codename in an image tag
A codename carries no ordering, so 'bookworm' and 'trixie' are two opaque
strings and no updater can tell that one supersedes the other. A pin that
names its Debian release that way is frozen on that release for as long as
it stands, and the freeze is invisible because the tag keeps resolving.
Numeric tags are comparable and stay inside the freshness pipeline, so the
rule is worth holding for the whole repository rather than fixing case by
case.

The check reads two shapes, because a codename reaches the tree in two
spellings. A 'name:tag' reference is recognised by the absence of a space
after the colon, which is what separates 'debian:bookworm' from the YAML
'image: python' and covers Dockerfile FROM lines and heredocs without a
second parser. A 'version:' value is read as a key, because '3.11-bookworm'
carries no colon at all.

False positives are kept out by two independent gates: a word-boundary
prefilter, so 'residual' and 'inside' are not read as 'sid', and a split of
the tag on '-', '.' and '_' that requires a whole component to be a
codename. That is what lets 'cache buster', '.stretched-link',
'fa-bullseye' and a tags list containing 'stretch' through. Every codename
occurrence in the repository was checked against the matcher by hand, and
the check was negative-controlled in both shapes, including an untracked
file.

Four references had to move for the rule to hold. meta/distros.yml takes
'debian:13' rather than '13.6': at depth 1 the updater can raise it across
a major, while depth 2 pins it inside the 13.x line forever. The whole
'make install' path was run in a debian:13 container to back that: all
twenty-four Depends and Recommends resolve on trixie, the docker repository
bootstrap derives the right suite and installs docker-ce-cli, and the
package builds and installs.

The workspace DiD probe moves to 'debian:13-slim'. An earlier version of
this change carried a suppression claiming the probe had to follow the
suite the apt proxy mirrors. That was wrong twice over: it cited a function
that has nothing to do with the probe, and the coupling does not exist,
because the cache frontend rewrites by path prefix and the suite never
enters a routing decision. Replaying the probe verbatim against
debian:13-slim fetches all three trixie suites through the
bookworm-configured proxies, so the suppression is gone rather than
reworded.

The rule ships with an escape hatch and no uses of it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 17:03:53 +02:00
kevinveenbirkenbachandClaude Opus 5 aa6bbe3b20 feat(lint): render the software identity from its declared variables
SOFTWARE_NAME, SOFTWARE_DOMAIN, SOFTWARE_URL and ORGANIZATION are declared once in group_vars/all/00_general.yml and derive from each other. Eleven templates wrote the value instead, so a rename would reach the four declarations and every consumer that reads them, and silently miss the literal.

The lint reads the name from the declaration rather than carrying it, and searches case-insensitively: the domain is the name lowered, the URL is https:// plus the domain and the organization is the name again, so one literal finds the value of all four. A hit is reported with the variable it should have been, derived from the scheme in front of it and its case. The escape is nocheck: software-identity, same line or the one above.

Every migrated template renders the same bytes as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-15 21:38:45 +02:00
kevinveenbirkenbachandClaude Opus 5 f19d5e3477 test(lint): refuse a loop_var that shadows a declared variable
Ansible warns that the name is already in use and carries on. Carrying on is the problem: inside the loop the original value is gone, so every lookup, template and nested include reading that name sees the loop item instead, and the task still reports success. The damage lands in a log nobody greps.

The lint collects every key declared in group_vars and in each role's vars or defaults, then reports any loop_var matching one. A denylist of known-bad names would have found application_id and missed domain, which is the same class in four more places.

All seven existing sites are marked, each with the reason read at the source rather than inferred. The three application_id loops pass the role id as the item, and the include_vars that follows loads that role's own application_id with the same value, so both meanings coincide by construction. The four domain loops hand the vhost to sys-stk-front-proxy, whose tasks read front_proxy_domain from exactly that name, so the shadowing is the interface.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-15 21:23:17 +02:00
kevinveenbirkenbachandClaude Opus 5 154d0c0cd2 test(lint): refuse an accept verdict in a firewall fragment
A fragment declares its own nftables table. An accept there terminates that table's base chain and nothing else: the packet still traverses every other base chain at the same hook, and a drop in any of them wins. Docker sets the filter FORWARD policy to drop by default, so a fragment that accepts is not permitting anything, it is declining to decide.

Measured rather than argued. Converting a role's appended iptables FORWARD accept into a fragment rule at priority -10 delivered 0 of 3 forwarded packets under that policy where the rule it replaced delivered 3 of 3, and the fragment's own counter showed the accept had matched all three. Both versions deployed green, which is why this belongs in a lint and not in a paragraph.

The message names both ways out: express the intent as a drop of what must not pass, or put the rule in the chain that carries the policy it has to overrule.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-15 21:14:10 +02:00
kevinveenbirkenbachandClaude Opus 5 508bc5adac test(lint): keep shell scripts out of jinja templates
A templated script is opaque to every tool that reads shell. shellcheck parses Jinja as syntax errors, the repository's own .sh guards skip the file because it is not a .sh, and nothing can execute it to test it: what runs on the host never exists in the repository, so a defect in the rendered half is found by deploying it.

The lint reports any *.sh.j2 under roles and points at the static slot sys-service already resolves, files/shell/script.sh, with the values passed as ExecStart arguments or Environment entries the way this repository's python scripts already take them.

None of the seven templates present meets the bar the marker is for, which is a script whose shape is generated rather than its values. Each is grandfathered with a marker naming what it interpolates, because converting a deployed script to argv changes what runs on the host. The guard exists to stop the next one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-15 20:36:04 +02:00
kevinveenbirkenbachandClaude Opus 5 18aaf57006 test(lint): let sys-service own every systemd unit a role installs
sys-service resolves the template, writes it into the unit directory, reloads the daemon, enables and starts the unit, and carries the timer, async and flush behaviour. A role that renders its own service.j2 with a bare template task gets the file and none of the rest, and every one of those omissions is silent: a unit that exists but was never enabled looks installed until the host reboots.

The lint reports any src pointing at a service.j2 inside a role's tasks. desk-ssh was the one finding left after sys-svc-firewall moved over, and it migrates here: its ssh-agent unit now comes from sys-service in the user scope, with the type, ordering, environment and Install section supplied as template fields. RuntimeMaxSec is pinned empty on purpose, because the default applies it to anything that is not oneshot and would kill a long-running agent after a day.

The unversioned unit is retired first. sys-service names units by version, so the old file would otherwise stay behind enabled and a second agent would bind the same socket.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-15 20:34:33 +02:00
kevinveenbirkenbachandClaude Opus 5 dd05caa34f feat(services): require an explicit onion decision per host-bound port
A port under ports.local or ports.public binds an address on the stack host, which is what a HiddenServicePort can forward to. Whether it should be forwarded has a cost in both directions: silently off leaves a service unreachable that the operator believes is on the onion, and silently on publishes a database, an admin interface or a SOCKS proxy to everyone holding the onion address. Neither belongs in a default, so the flag is now mandatory and every false carries the reason it is false.

The first run produced 165 findings, of which 131 were one architectural fact repeated rather than 131 decisions. Four groups are therefore settled once in the lint instead: the proxy-fronted http, sso and websocket, because the node onion already forwards port 80 to the reverse proxy that routes every vhost, oauth2 hop and socket upgrade behind it; the implicit-TLS variants, because Tor is already the transport encryption and container_ports drops those listeners when TLS is off, which an onion deployment always is; the categories a hidden service cannot carry, listed once in plugins.lookup.tor_ports.UDP_ONLY_CATEGORIES, which the lint imports rather than restates; and any entity that already answers the question through exposed:.

That leaves 23 real per-port decisions, each with a reason naming its own consumer. svc-net-tor keeps its own socks, trans and dns off the onion: a published SOCKS port is an open proxy, and SocksPolicy cannot refuse it because the forward arrives from the node's own address.

Mailu turns smtp, submission, imap and pop3 on, which is what makes sending and receiving work over the onion, and leaves sieve off because ManageSieve manages filters rather than mail. Measured end to end through the merged applications view, across all 105 roles the deploy now forwards exactly 25, 110, 143 and 587.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-15 14:51:59 +02:00
kevinveenbirkenbachandClaude Opus 5 d8c6bd242d test(ci): require every secret and variable a workflow reads to be documented
An unset secret or variable resolves to the empty string, not to an error, so
an undocumented one is invisible twice: the administrator cannot know to set it,
and the run that needed it degrades quietly instead of failing. The guard reads
every ${{ secrets.X }} and ${{ vars.X }} out of .github/workflows and demands
the name in the page that owns it, secrets.md or configuration.md.

It compares the reference against the page rather than the page against itself,
so it also catches a doc naming something the workflows no longer read.

Three gaps were already there. DOCKERHUB_USERNAME and DOCKERHUB_TOKEN
authenticate the image mirror against Docker Hub and lift the anonymous
pull-rate limit; both are declared required: false and the login step is gated
on them being non-empty, so their absence does not fail a run, it walks the
mirror into the rate limit. Neither was documented anywhere.

PLAYWRIGHT_KEEP_ALL is the sharper one: configuration.md listed
INFINITO_PLAYWRIGHT_KEEP, which is the name the workflow forwards it under, not
the name an administrator sets. Following the documentation produced a variable
nothing reads.

Commented-out lines are skipped. cron-security-scorecard.yml carries
`# repo_token: ${{ secrets.SCORECARD_TOKEN }}` as an option nobody has taken,
and demanding a doc row for it would describe an input no run asks for.

GITHUB_TOKEN is exempt in _PROVIDED_SECRETS because GitHub mints it per run; it
is not administered and cannot be set. A second case asserts the scan finds
workflow files at all, so the guard cannot pass vacuously.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 19:59:10 +02:00
kevinveenbirkenbachandClaude Opus 5 786899700f feat(litellm): serve OpenAI, Anthropic and OpenRouter from declared keys
The gateway could reach exactly one remote provider, through a bare
OPENROUTER_API_KEY group_var with an empty-string default and an `{% if %}`
around a single hardcoded model entry. OpenAI and Anthropic had no path at all.

All three are now declared in roles/svc-ai-litellm/meta/secrets.yml, which
mapping.py already calls the application config schema, so a key lands at
applications.svc-ai-litellm.secrets.credentials.<provider>_api_key like every
other credential. Each carries the `type` and `regex` its value must satisfy and
defaults to the central API.<provider>.api_key block, which keeps one
operator-facing place to fill a provider in while the role owns the shape.

type and regex had to become real to be worth declaring. 192 `validation:`
blocks sit in the role schemas today and not one line of Python enforces them,
so a `regex` would have joined a pile of decoration. validate_supplied_value
now runs on every path a value arrives from outside — the --set override, a
value already in the inventory, and the value standing next to a `default:`.
A generated value stays exempt because the algorithm that produced it is named
by the same schema and generate_value takes no schema argument. min_length is
deliberately left unenforced: the generators ignore it, so holding them to it
would fail where nothing is wrong.

The pattern is written for the paste, not for the vendor. OpenAI moved issuance
from sk- to sk-proj- without notice and Anthropic issues sk-ant-api03-; a schema
pinning a prefix would reject a valid key and abort inventory creation. `\S{20,}`
accepts every form those three issue today and one nobody has announced, and
rejects a trailing newline, an embedded space, a truncated copy and a
<placeholder>. A key pasted into the wrong provider's slot is caught at deploy
instead, where the route it publishes answers nothing.

The gateway iterates one LITELLM_REMOTE_BACKENDS list instead of three `{% if %}`
blocks; config.yaml.j2 and env.j2 read the same list and selectattr('key')
decides. LITELLM_CHAT_MODEL falls back to the first configured remote alias
rather than to a hardcoded openrouter/auto.

For CI the keys travel as INFINITO_API_{OPENAI,ANTHROPIC,OPENROUTER}_KEY through
default.env, the passthrough allowlist, the infinito service in compose.yml, the
workflow env block and the development inventory. The compose leg is not
optional: the env lint rejected the change without it, because a key the
inventory reads but compose does not forward resolves to empty inside the
container with no error. The workflow reads `secrets.X || vars.X` so a variable
works as asked while a secret wins, which matters because a variable is not
masked and the development inventory sets MASK_CREDENTIALS_IN_LOGS: false.

An unset key is not an error anywhere: the provider's routes stay unpublished
and the CLI test asserts their absence rather than skipping.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 19:57:21 +02:00
kevinveenbirkenbachandClaude Opus 5 4ce588fea1 test(lint): forbid a JavaScript default for an unset environment variable
`process.env.X || <value>` is a second source for something the deployment
already renders, and the two drift without a sound. playwright.config.js read
`process.env.PLAYWRIGHT_ACTION_TIMEOUT || 30_000` while nothing ever set that
variable, so every run used the unscaled 30 s and no failure said so.

Normalising an absent variable stays allowed, because it substitutes nothing:
`|| ""`, `|| '[]'`, `|| false`, `|| 0` and falling through to another
`process.env` entry only let the caller test absence itself.

The six sites the rule found:

  n8n and flowise built a Postgres port from `|| '5432'` while env.j2 renders it
  from `lookup('database', application_id, 'port')`. Node throws
  ERR_SOCKET_BAD_PORT on the resulting NaN, so an unfed port now names itself
  instead of connecting somewhere plausible.

  The bluesky broker did the same with BROKER_PORT, which compose.yml.j2 renders
  unconditionally. Its INSECURE_TLS default was `"false"`, which is what `""`
  already evaluates to.

  matomo defaulted MATOMO_TRACKING_SCOPE to "sub"; the only comparison is against
  "root", so `""` takes the identical branch.

  moodle and simpleicons defaulted a name their playwright.env.j2 always renders.
  simpleicons already asserts the value is truthy; moodle now fails on the scope
  it cannot find rather than on a name nobody configured.

Four values the code itself owns keep a default under `// nocheck: env-default`:
PDS_INTERNAL_URL (which falls through to requireEnv, not to a literal),
HANDOFF_COOKIE_NAME, HANDOFF_COOKIE_MAX_AGE and PASSWORD_SALT_HASH_ROUNDS. No
role renders any of them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-12 12:07:57 +02:00
kevinveenbirkenbachandClaude Opus 5 e76e3d5b41 fix(nextcloud): compute the sociallogin group mapping key
Ansible renders the values of a structure loaded through include_vars and leaves its keys as written, so the mapping key a564385068 wrote as a Jinja expression reached Nextcloud as its own source text. occ config:app:get sociallogin custom_providers on a live variant-1 instance returns "groupMapping": {"/{{ lookup('rbac_group_path', application_id=application_id, role='administrator') }}": "admin"} while every neighbouring value of the same provider is rendered, scope among them.

Keycloak emits /roles/web-app-nextcloud/administrator, so the mapping matched nothing and the account sociallogin provisions on first login stayed out of admin. occ group:list on that instance shows admin holding only the local administrator while keycloak-ac165525-e9e8-4230-9381-13b1acd83fc0 exists and belongs to no group, and every Playwright spec that needs an admin settings section answered 'Access forbidden'.

An independent reproduction outside this repo showed the same split on ansible-core 2.21.4 and 2.20.6: in a payload of the same shape the sibling value rendered and the key did not, so it is not a version artifact. Nextcloud 34.0.3 settles the other half: Manager.php hands an admin the unfiltered settings of a section and ai is a core section of the always-enabled settings app, so an admin cannot receive 403 there whatever is installed.

Writing the whole mapping as one expression makes jinja2_native, which this deployment has on, return a real mapping with the key computed. The lint takes the class: no meta payload may carry Jinja in a mapping key. It found exactly this one case in the whole meta tree, and it ignores a line that itself opens with the expression so the corrected form does not trip it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-12 10:29:49 +02:00
kevinveenbirkenbachandClaude Opus 5 7ccd2c185c fix(n8n): resolve the hosting node before the condition that uses it
Ansible templates delegate_to before it evaluates the task's when, so a task delegating to N8N_HOST_NODE kills the play with 'N8N_HOST_NODE' is undefined whenever the resolver that sets it was skipped, even though the delegating task would have been skipped too. 43e09ebf0d put the resolver inside the same conditional block. Run 34653754175 shows the pair directly above the failure: the include skipped on the false condition while the setup call had returned status 200, and the play ended at 01_bootstrap.yml:92.

The resolver now runs unconditionally, before the block. It costs one lookup per deploy and changes no state.

The new lint reads the class rather than this instance: a conditional resolve_host_cid include whose PREFIX_HOST_NODE a delegate_to in the same role reads, unless an unconditional resolver or set_fact defines that prefix role-wide. The walker carries the condition through block, rescue and always, so an include nested in a block does not read as unconditional. Against the pre-fix file it reports would flag: ['N8N'].

It found one more: web-app-yourls/tasks/install.yml wrapped its tasks in a block whose when repeated the IS_STACK_HOST condition that main.yml already carries on the include. The wrapper is gone and the tasks are dedented; nothing had broken there because the condition always held.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-12 09:29:37 +02:00
kevinveenbirkenbachandClaude Opus 5 b3ce43a596 fix(security): require no_log on every task that carries a credential
At -vvv Ansible prints each module argument and result, so a password in
a command line, a request body, a header or a fact lands in the deploy
log. Nothing enforced masking outside the MCP task files, and no check
looked at parents at all.

A new lint, test_credential_tasks_are_masked, finds tasks whose printed
parts (module arguments, environment, loop) reference an identifier ending
in password, secret, token, api_key or private_key, or
secrets.credentials. A task passes when it or an enclosing block sets
no_log, when every file that pulls it in does so through include_tasks
with apply.no_log or import_tasks with no_log (transitively), when the
secret only reaches a parameter the module masks itself, or when the
module masks its values. no_log on an include_tasks task itself does not
count, because it does not reach the included tasks. The rule can be
suppressed per task with `# nocheck: credential-masking`, now listed in
the suppression catalog, and the lint tests its parent and suppression
logic on synthetic cases.

It found 39 unmasked tasks in 19 roles, all real: join tokens, client
secrets, API tokens in Cloudflare headers, database passwords in
command lines, the ChatGPT bot login body and more. Each now sets
no_log from MASK_CREDENTIALS_IN_LOGS. The sys-dns-hetzner-rdns alias
hetzner_no_log, which only restated that switch, is gone.

Not covered yet: template and copy tasks whose rendered file carries a
secret print it as a diff under --diff.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 21:52:18 +02:00
kevinveenbirkenbachandClaude Opus 5 8d8701d9b3 feat(apt): one Ubuntu mirror list for the package cache and image builds, with failover, a short timeout and a health test
The ubuntu CI image build (job 103277644636) hung on Canonical's
archive: apt fetched 26 MB at 417 kB/s, then 61.6 kB in 29 s, then sat
on the package download until the job was cancelled after 23 minutes,
while download.docker.com delivered 42.8 MB in under a second on the
same runner. Locally archive.ubuntu.com and security.ubuntu.com took
20-26 s to the first byte against 2 s for a university mirror.

INFINITO_APT_UBUNTU_MIRRORS in default.env is now the one list of
Ubuntu mirrors, in failover order: de.archive.ubuntu.com, ftp.fau.de,
ftp.halifax.rwth-aachen.de, archive.ubuntu.com.

- Image builds: the Dockerfile sources the list alongside the install
  scripts and runs scripts/install/apt-mirrors.sh first. In an image
  with ubuntu.sources it points both stanzas at apt's mirror+file
  method, priorities in list order, and sets Acquire::http::Timeout to
  10 s, so apt moves on from a mirror that fails or goes silent; other
  distros are untouched. In ubuntu:latest apt-get update took 8 s
  against 110 s on the stock sources, a dead first mirror was left
  behind within 25 s against 62 s on apt's defaults, and the swarm node
  packages (80 debs) came in 17 s. The Dockerfile's own RUN lines ran
  clean on ubuntu, debian, fedora and arch.
- Package cache: package.sh proxies the first entry as the primary
  Ubuntu repos and the second as their fallback. Nexus revalidates
  metadata on every request here (metadataMaxAge 0), which costs a
  full download from a mirror without an ETag; both de.archive and
  ftp.fau.de send one, and the 15 MB universe index now comes back
  from the cache in 0.02 s.
- tests/external/repository/test_apt_ubuntu_mirrors.py keeps the list
  honest: the first mirror must answer InRelease within 5 s (median of
  three) or the test names a faster later one, no mirror's -updates may
  lag the freshest by more than 24 h, and every entry must be the
  primary archive, a Canonical country alias or an enabled official
  mirror in Launchpad's registry. It caught the first draft of the
  list: mirrors.edge.kernel.org is not an official Ubuntu mirror.

After pulling, an existing checkout needs `make dotenv-force`: plain
`make dotenv` re-reads the old .env through BASH_ENV and keeps its
values.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 17:13:54 +02:00
kevinveenbirkenbachandClaude Opus 5 2ea06aac4c fix(cache): bound the frontend's wait on a primary apt repo so the mirror answers in time
test_packages_available failed locally in every quality pass with
"91 package(s) could not be checked at all ... archive.ubuntu.com:
timed out". The dev stack routes archive.ubuntu.com to the package-cache
frontend, which re-runs a request against the -mirror repo only once
the primary Nexus repo answers 502/503/504. From this host's docker
network Canonical's IPv4 addresses take seconds to connect and then
send nothing, and Nexus tries all nine addresses of archive.ubuntu.com
in turn before it gives up: apt-ubuntu answered 502 after 21-25 s, so
the frontend's first byte arrived after 27-46 s, past the probe's 20 s
timeout. apt-ubuntu-mirror (de.archive.ubuntu.com) served the same
files in 0.08 s.

The four primary locations now carry proxy_read_timeout 8s: nginx
gives up on Nexus itself, its own 504 hits the existing error_page, and
the mirror answers. A shorter Nexus upstream timeout was tried first and
dropped, because Nexus spends it once per address and still took
9-21 s. The mirror locations keep the global 300 s, and a healthy
primary answers well inside the bound, as before.

Measured after a compose restart: the ubuntu index arrives after
8-12.6 s with Origin: Ubuntu, debian still in 0.03-0.16 s, npm and PyPI
in 0.13 s; make test-external passed 13 of 13 and the full
make quality-high went green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 15:43:08 +02:00
kevinveenbirkenbachandClaude Opus 5 3db5820b6e fix(n8n): serve the MCP trigger over Streamable HTTP on 1.100.1 so Open WebUI can connect
Run 34421944328 n8n#1: Open WebUI 0.11.0 ships only a Streamable HTTP
MCP client. n8n 1.95.3's MCP Server Trigger speaks SSE only and answered
the client's POST to /webhook/infinito-mcp/sse with 404, so "every
registered MCP server answers Open WebUI's client" failed.

n8n moves to 1.100.1 and the managed trigger to typeVersion 2, which
serves Streamable HTTP at /mcp/<path>. meta/mcp.yml declares
transport: streamable_http at /mcp/infinito-mcp (the trigger path stays
infinito-mcp) and points source_url at the 1.100.1 tree. Open WebUI's
supported_transports drops sse, which its client never spoke; no
provider declares sse any more, so no pairing is lost.

n8n#0: the guest spec probed ${APP_BASE_URL}${MCP_ENDPOINT_PATH} and
expected a refusal of >= 300 without an event stream. The trigger is
exposure: internal, so the public vhost must answer 404; the spec now
asserts exactly that through mcpEndpointUrl(..., N8N_BASE_URL), and the
bearer guard itself is proven in-cluster by the CLI MCP contract.

Requirement 035 and the MCP audit table follow the version and
transport.

Upgrade path, read from upstream at both tags: the postgres migrations
are identical (45/45) and the only new common migration is
AddProjectDescriptionColumn, an additive nullable column; BREAKING-
CHANGES.md lists only 1.98.0 in range (route metric format, Form node
tag attributes, Node >= 20), none of which this deployment uses. A fresh
1.100.1 install ran all 89 migrations and passed the MCP bridge, server,
non-member and guest specs locally.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 12:20:46 +02:00
kevinveenbirkenbachandClaude Opus 5 3d6b3900e0 refactor(cli): gather the documentation generators under cli/build/docs
Two of them lived inside docs/ next to the pages they render, which is why both had grown a __file__ upward walk to find the repository root, a sys.path insert to import from utils, and a nocheck for each. A generator parked among documents reads as documentation and drops out of the checks that apply to code.

Moving them surfaced six real violations that the old location had hidden: yaml.safe_load instead of the shared cache, two uncached read_text calls, five hard-coded role paths instead of ROLE_FILE_META_SERVICES and ROLE_FILE_META_MCP, two make targets without an alias, and a requirement reference in code. All six are fixed at the root, none with a suppression, and both generators lose their project-root exemptions because a package supplies PROJECT_ROOT.

cli/build/readme joins them so the three generators that write into docs/ sit together. A new lint keeps docs/ to .md, .html, .rst and .nocheck; it enumerates through git ls-files so untracked __pycache__ is not its business. Checked against HEAD, it names exactly the three .py files that moved.

026-unified-addon-syntax stated that the matrix generator stays under docs/requirements/. The lint makes that impossible, so the requirement now records where the generator actually lives.

Verified: every one of the twelve modules under the new tree imports, no old module path survives anywhere in the repository, 482 lint tests and 1246 cli unit tests pass, and both generators run without drift.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 23:17:55 +02:00
kevinveenbirkenbachandClaude Opus 5 e415187f83 update(images): raise eight image versions the docker-version check flagged
confluence 10.2.16 to 10.2.18, erpnext v16.34.1 to v16.34.2, espocrm 10.0.7 to 10.0.8, hermes v2026.7.20 to v2026.9.7, mattermost 11.10.1 to 11.11.0, nextcloud context_agent 2.7.0 to 2.8.0, seaweedfs 4.45 to 4.46, coturn 4.17.2 to 4.18.0. The automated updater opens one PR per role and had not reached these eight; playwright and elasticsearch were already current in the tree.

audit.md follows rather than leads: the MCP report carries a version column and test_mcp_audit_report compares the committed file against a fresh generation, so hermes and mattermost move there too. Regenerated with audit.gen.py as that test instructs. Note that make docs does not run this generator, so the report only tracks reality when somebody calls it by hand.

Verified by the docker-version check itself: eleven Outdated warnings before, zero after.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-08 22:37:14 +02:00
kevinveenbirkenbachandClaude Opus 5 70feb319be feat(nextcloud): front the Context Agent with the MCP adapter
The hub serves 94 tools across 23 categories, write tools included, and enabling the ExApp switches every category on. That surface was published on the public vhost and every discovered client was handed an app password for it. svc-ai-mcp-adapter now sits in front in mcp_passthrough mode: clients present credentials.mcp_bearer to contextagentmcp and reach the nine read tools of calendar, contacts and files, which files/mcp/tools.json names and the sidecar enforces per call. Those three categories are the ones whose upstream is_available is unconditional, so the served set does not vary with which optional apps a deployment installs.

Enforcing it in the sidecar rather than in each client is what makes it hold: web-app-openwebui renders no per-tool filter at all, so a client-side include list would have bounded two consumers out of three.

06c13e7f3e withdrew the declared path from the edge for exactly this reason - "that surface is what the adapter exists to narrow". Nextcloud answers the same route under a second spelling, without the /index.php front controller, which try_files rewrites back onto the route after the edge has already declined to block it. The withdrawal becomes a regex covering both, and only where stripping the front controller changes the path, so the four adapter providers without one render unchanged.

The app password is minted after the stack renders, so it reaches the sidecar through upstream.env and rebuild.yml rather than the compose environment: compose up resolves env_file while creating the container, a restart does not. The sidecar is the last service in the template because a Jinja set has no block scope and its env_file would otherwise displace the one talk, whiteboard and talk_recording each carry.

Two suites cover what nothing measured: the vhost template rendered without a container, asserting both spellings are withdrawn, and every shipped tools.json loaded through the adapter own loader, so a bad contract fails in a second instead of as a probe timing out an hour into a deploy. The guest spec probed mcp.endpoint.path, which this change makes the proxy answer 404 for its own reasons; it now probes all three public paths and requires no MCP body.

Not yet deployed: the sidecar build, the upstream.env delivery and the hub answering exactly nine tools are unverified against a running stack.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-03 20:55:05 +02:00
kevinveenbirkenbachandClaude Opus 5 346ad875c6 refactor(mcp): drop the two provenance fields nothing reads
supported_version and minimum_version sat in 24 roles and were read by no template, no task and no lookup. Their only consumer was the audit report version column, which turned false the moment an automated image bump ran: qdrant reported a verified version nobody had verified, two releases long. Both leave MCP_KEYS, so a role that declares them again fails the schema lint instead of carrying a field with no effect.

Requirement 025 mandated minimum_version as a MUST and 035 carried it in an example; both are corrected here, otherwise the repository would contradict its own code.

The gate then surfaced five dead docs.mattermost.com links. Mattermost moved to /administration-guide/ and /deployment-guide/, and every replacement was probed for 200 rather than guessed. The MCP page is gone with no successor, so mattermost now points at mcpserver/README.md in the Agents repository, the source rather than a rendered mirror.

Both concerns share three files - mattermost's meta/mcp.yml, requirement 025, and the generated audit.md, which derives from a source of each - so splitting them would commit an artifact that does not match its sources.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-03 19:31:08 +02:00
kevinveenbirkenbachandClaude Fable 5.1 113113e553 perf(applications): render the applications SPOT once per play instead of once per task
Ansible forks a fresh worker process for every task, so the in-process
cache behind lookup('applications') was cold in every task. Each task that
touched the merged applications tree (config, domains, nginx, tls, cdn and
their callers) paid a full render: about 16 s on main and 21 to 23 s on this
branch, where ten more application roles carry MCP and AI Jinja. Measured on
the workspace jobs of run 33574056480 against the main run: 331 such tasks
(93 min, 39 % of the job) versus 391 (141 min, 43 %); the same unit is 9 to
43 % of every compose and swarm deploy job on disk. The parent process
rendered once per play, late, while templating a task name, and only the
tasks after that point ran warm.

The constructor now renders once through the new applications_carrier
action and parks the payload with its cache key in the host fact
_INFINITO_APPLICATIONS_RENDERED. Every later worker inherits the fact at
fork time and get_merged_applications serves it while the key (roles dir,
applications, users, DOMAIN_PRIMARY, SYSTEM_EMAIL_DOMAIN) still matches; a
mismatch falls back to the old render. The action masks its own result: the
payload is the whole tree with every credential, and CI runs with
MASK_CREDENTIALS_IN_LOGS false.

The task sits after the networks, design and service_provider merges, after
the add_host placements and after the token store. An independent review
showed that an earlier position bakes stale values into the play: the
web-app-wordpress title rendered to 'Blog by ' because service_provider did
not exist yet. User records inside the payload are the token store's state
at render time; readers of tokens captured later keep using lookup('users')
or lookup('config', app, 'users.…'), which read the users cache fresh.

Expected savings per job: about 130 min on a fork workspace run, 16 min on
an average compose deploy and 28 min on an average swarm deploy. The
constructor's own lookups before the carrier stay cold.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 23:55:32 +02:00
kevinveenbirkenbachandClaude Opus 5 4164f37981 fix(mcp): give every MCP round an identity provider to sign in with
An MCP client spec signs in through the client's own OIDC button, which
redirects to Keycloak. Every provider put its MCP pairing in the lean variant,
and that variant pinned sso off - so the round that exists to prove the pairing
was the one round without an identity provider. Run 33574056480 shows what that
costs: openwebui's MCP specs reached _shared.js:131 and died on "openwebui OIDC
sign-in button must be visible on the auth page".

The rule now demands two things of the variant that carries the consumers: the
provider pins sso on, and each admitted consumer, in the variant that round
actually pulls in, does not pin it off. The second half matters because a round
deploys every dependency at its own index and falls back to variant 0 only when
the dependency has fewer, so the client shipping alongside is that one and not
the client's variant 0. A role that declares no sso, or pins it to a literal
false because upstream speaks no OIDC, is exempt from a flag it cannot have.

Twenty roles moved to satisfy it. Ten already had a second sso: false pin and
only needed the flip. Six could host the pairing in variant 0, which already
has sso on, and dropped back to two variants. Six kept a third lean variant to
carry the false polarity variants-coverage wants. moodle and mattermost took
swaps instead: moodle's variant 1 is the mandated LDAP-only branch, and
mattermost was already at the three-variant cap with sso on in its variant 2.
nextcloud also turns context_agent on in that round, the ExApp that serves the
endpoint - with it off the round proved nothing it was built to prove.

The rule moves to tests/lint/ansible/services/ because it reads meta YAML and
executes nothing, which is where integration.md says such a check belongs, and
its suppression key joins the catalog it was never listed in.

nextcloud_exapp_reregister becomes nextcloud_exapp_renew: mcp-secret-masking
matches `register:` unanchored, so the substring in the old name read as a task
that prints a secret. register.yml already masks the two tasks that carry one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 15:51:24 +02:00
kevinveenbirkenbachandClaude Opus 5 af605b049f refactor(domains): give local testing a .test domain, leave .example to docs
RFC 6761 reserves .test for testing and .example for documentation, and the
project's own domain was doing both jobs under the documentation name. It is
what default.env hands every local deploy and what the test fixtures assert
against, so it is a testing name.

Rename it wherever it stands for the running machine: the environment
defaults, the development inventory, the workspace and trust scripts, the test
suites, and the four documents that describe real behaviour - the DNS mapping
to loopback, the browser CA trust, WSL2 resolution. Those documents would have
started lying the moment default.env changed.

The sandbox allowlist moves with them, since egress is matched per pattern and
nothing resolves the old name any more, and the sandbox document that mirrors
that list follows it.

Two places keep .example on purpose: the link-style sample in the
documentation guide, which is documentation about documentation, and every
generic fixture name elsewhere in the suite - a.example, foo.example and their
kin. Those are a larger sweep of their own and are not touched here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 13:10:34 +02:00
kevinveenbirkenbachandClaude Opus 5 791eee7c9f refactor(mcp)!: prove the endpoint contract with a CLI test, not a deploy probe
The contract check ran as an Ansible task in the middle of every provider deploy, and the two client checks ran the same way. A test that is part of the deploy cannot be run without deploying, cannot be re-run on its own, and reports its verdict as a deploy failure. Twenty-seven task files lose 491 lines to this.

It is now what it always was. Each provider ships files/test/test.sh, templates/test.env.j2 and meta/tests.yml, and test-e2e-cli stages one shared harness beside it: run.sh drives contract.sh in a throwaway container on the provider's own network, then isolation.sh for adapter-fronted providers. The read probe each provider offers moved from the calling task into meta/mcp.yml next to the allowlist it must satisfy, and the schema now rejects a read probe naming a tool outside that allowlist.

The bearer the test presents is the one a client resolves through mcp.credential, via the new mcp_client_bearer lookup over resolve_credential, which moved to utils.roles.applications.mcp so it is the same function mcp_servers renders every client connection with. The probe presented whatever the calling role passed, which for the fifteen adapter-fronted providers is a different secret than any client holds.

Sidecar isolation is read from the engine rather than measured inside the container, so it no longer needs a delegation to the node holding the task: attachment is compared by network name, which drops the gwbridge special case, and a swarm worker says it cannot measure instead of skipping silently.

Retrying is bounded by wall clock against MCP_DEADLINE_SECONDS, which scales through the same lookup('timeout') as the harness budget it has to stay under. A flag that is neither true nor false aborts rather than reading as off.

mcp-provider-probe becomes mcp-provider-cli-test: an enabled provider must ship the test rather than include the task. mcp-probe-bearer becomes mcp-adapter-bearer, tying ADAPTER_BEARER to the declared credential key now that no probe include carries it. A new integration test renders the shared env template for every provider through Ansible's own templar, so a later change to it breaks visibly instead of poisoning twenty deploys quietly.

Hermes and OpenClaw keep their client-side proof as CLI tests of their own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 20:46:26 +02:00
kevinveenbirkenbachandClaude Opus 5 08ac190b4b docs(mcp): regenerate the audit after jenkins admitted Open WebUI
32dd66099 gave web-app-jenkins the third consumer its neighbours already
had, and the report reads that list from the same mcp_consumer edges the
derivation does. One cell.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 18:04:13 +02:00
kevinveenbirkenbachandClaude Opus 5 ffdef28da3 refactor(mcp)!: derive a surface's presence from its peers
enabled and shared were hand-maintained in 24 roles and wrong in both
directions. Both now resolve from the mcp_consumer edges already declared
in meta/services.yml: a server takes the consumers it admits, a client
takes the servers that admit it, and the pair renders as "'peer' in
group_names or ...", or False when no peer is admitted at all. setdefault
keeps meta/mcp.yml and meta/variants.yml as the override channel, so
pinning either key still wins.

The resolved block replaces the mapping instead of being written into it.
_build_role_base_config parks the YAML cache's own object under mcp, so an
in-place setdefault injected the derived keys into every later raw read and
made test_mcp_schema pass alone but fail in a directory run.

The derivation keys on peer presence, never on a peer's own value: a
re-entrant read under _RENDER_GUARD returns the unrendered dict.

Four lints now ask the resolver instead of reading meta/mcp.yml, which they
could not do while the key was hand-written. That is what exposed the eleven
missing hostile-path specs the previous commit closed.

Carries one change that belongs to the WordPress probe but shares a file
with this one: the tool allowlist moves to the hyphenated names the adapter
actually serves.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 16:13:01 +02:00
kevinveenbirkenbachandClaude Opus 5 94f8f976c8 docs(mcp): regenerate the audit after the gitea image bump
bb975843f1 raised web-app-gitea to 1.27.3 in meta/services.yml and left the audit
table at 1.27.2, so test_mcp_audit_report.py has been failing since:

    AssertionError: audit.md differs from a fresh generation.
    Regenerate and commit it by running audit.gen.py from the repository root

Regenerated, nothing else. audit.gen.py:54-62 reads the version out of each role's
own meta/services.yml and makes no network call, so the file is a pure projection
of the tree and drifts only when a version changes -- this is real drift, not a
flaky check.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 23:41:53 +02:00
kevinveenbirkenbachandClaude Opus 5 83c1c729de refactor(mcp)!: derive admission from the client
allowed_consumers was twenty hand-kept lists of which fourteen were identical, so every new client meant an edit in every provider. A client now declares services.<own-entity>.mcp_consumer: true once and a provider refuses one with mcp_consumer: false, which is also what tells a deliberate exclusion apart from a forgotten entry.

The group_names-gated consumer entries give the inventory resolver the co-deploy edge meta/mcp.yml never had: one of sixty-nine matrix rounds carried an MCP client before, twenty-three role/round pairs do now. bond 0.5 keeps the clients out of round 0 so the host memory budget still holds.

BREAKING CHANGE: meta/mcp.yml no longer accepts allowed_consumers; declare services.<entity>.mcp_consumer on the client instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 18:33:22 +02:00
kevinveenbirkenbachandClaude Opus 5 7fd495cdfc feat(ci): replay a role's README instructions on its smallest deploy row
The instructions test held a job of its own that picked one random role per
run, so a README's Production block was proven roughly once per 265 roles.
The replay now rides on a deploy row instead: no extra runner, and every role
the sweep touches gets its documented one-command deploy exercised.

The row is the role's smallest variant -- fewest services enabled -- because
that is the shape the README describes, and it is chosen among the variants
the sweep actually deploys, so a row cut as redundant coverage cannot silently
drop the role from the test. A role qualifies only when its README carries a
Production block and the guide's own mode is one the role runs and tests.

The marked entry carries the guide mode in `instructions` and a 📖 in its job
title, between the variant number and a priority ⭐. The guide mode is
independent of the mode the rotation gave the row: a swarm row still replays
in compose. The step runs last in the deploy job, after the swarm teardown,
because the replay deploys the role a second time and needs the runner's
ports.

call-test-instructions.yml, cli.meta.ci.guide_select and the `instructions`
dispatch input go with it. `redundant()` moves next to the marker so the
chunker, the plan table and the marker share one definition.

The plan table trades its 📊 Weight column for 📖 Instructions; nothing read
the weight any more, so it leaves the matrix entries too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 16:57:04 +02:00
kevinveenbirkenbachandClaude Opus 5 65d226d0d9 fix(container): reach every resolver address through a shell, and lint for it
The container_address lookup does not return a container name in swarm. It
returns a shell fragment, "$(/usr/bin/resolve-container-id <stack> <service>)",
which is only a name once an interpreter evaluates the substitution. The shell
module provides one. The command module does not -- it execs the binary directly,
so the fragment arrives at container exec as literal text and the call dies with
"No such container: $(...)".

Compose hides the entire class: there the same lookup yields a bare name and
command: works. The failure exists only in swarm, and no swarm run had touched
these paths -- both were added by feature commits (b23653adf8, 0421ebeb16) that
wrote the same argv shape independently, which is the argument for a rule rather
than six edits.

Six call sites moved to the shell module: the moodle egress task, the three
moodle AI-gateway tasks, and the two wordpress AI-Engine tasks. The wordpress
pair keeps its stdin, so it uses the dict form with cmd:. Moving off argv exposes
the surrounding values to word splitting, so they now go through quote -- the
address itself must NOT, because single quotes would suppress exactly the
substitution this depends on.

test_argv_exec_needs_shell.py closes the gap that let this through.
container-exec-resolver checks that the resolver is used; it has nothing to say
about whether anything can evaluate the result. The new rule checks that other
half. It reuses the sibling's argv parser, which recognises only container/docker
in first position, so an argv list that does start an interpreter
(argv: [bash, -lc, ...], as joomla does) is correctly not matched.

Run against the tree the rule reported exactly the five sites found by hand plus
bigbluebutton, and passed over joomla and the discourse task, whose address is a
config lookup rather than the resolver. Bigbluebutton is pinned to compose via
compose_mode_force and can only ever see a bare name, so it takes a nocheck.

Five container-exec-resolver markers disappear with the argv form: under shell
the resolver is the ordinary path, not an exception.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-31 11:39:38 +02:00
Alejandro Roman 333079cc2a feat(UI): fix migration, errors happening in local deploy and swarm integraton + merged main 2026-08-30 21:39:52 +04:00
kevinveenbirkenbach 0b9fe7861a feat(baserow): front the native MCP endpoint with the adapter
The least-privilege half of the criterion could not hold as built.
provision_mcp.py creates the endpoint owner's workspace through
CoreHandler().create_workspace, which makes that account its ADMIN, and
2.3.3 knows only ADMIN and MEMBER - both write table data. There is no
read-only workspace role to demote to, so mutating_tools_enabled: false
bounded nothing while create_rows, update_rows and delete_rows sat in the
served allowlist.

The adapter's contract simply omits them: a client calling one is refused
with DENY_UNKNOWN_TOOL. Omission is the mechanism, since passthrough.py
refuses to load a contract that lists a mutating tool while mutations are
off. The other ten stay recorded under upstream_serves so a version bump
enabling one shows up as drift rather than as a silently wider surface.

The transport was what reverted this twice. Baserow speaks classic
HTTP+SSE, where a GET opens the response stream and the stream names a
second URL for requests, while the adapter only ever spoke Streamable
HTTP. adapter.upstream_transport selects sse.py and defaults to
streamable_http, so no existing provider changes.

The endpoint key stops being a client credential. Clients present
credentials.mcp_bearer to the sidecar; the key reaches the sidecar as
ADAPTER_UPSTREAM_PATH_KEY and is spliced in percent-encoded, so it never
enters the rendered contract and a key carrying a separator cannot open a
path of its own.

Deploy is green: the probe reached list_databases over SSE, the served
surface matched the contract, and the sidecar proved it reaches only its
provider.
2026-08-29 11:08:49 +02:00
kevinveenbirkenbach 5ff94feee2 fix(docs): follow the MCP docs one directory down
The move left every relative path pointing one level short. Most are
links, but the generator's REPO_ROOT is not: with parents[5] it resolves
to docs/contributing/, ROLES_DIR matches nothing, and the report is
written empty instead of the run failing.
2026-08-29 10:41:30 +02:00
kevinveenbirkenbach 7713404e6a refactor(docs): nest the MCP design docs under their own folder
The three files shared an 'mcp-' prefix as siblings of the other service
design docs, which the prefix-group lint reads as a folder that was never
created. Their content is unchanged; the generator's path adjustment
follows in its own commit.
2026-08-29 10:39:42 +02:00
kevinveenbirkenbachandClaude Opus 5 2d30681977 docs(mcp): turn the audit into something the lints can check
The audit read as prose, so a role could sit under a classification nobody had checked against its own declaration, and "blocked" could not be told apart from "nobody looked". The sections are now named after the exact values a role declares, which makes the section titles the vocabulary: one rule holds every role under its own declared classification, a second rejects a classification outside the vocabulary, and a third requires a named blocker per blocked role while forbidding a blocked role from shipping an enabled surface.

The report itself is generated from the roles' own metadata and a drift check imports the generator and compares against the committed file, which caught a stale report twice while this work was in progress. It lives under docs/contributing rather than beside the requirements, because a generated file in that directory reads like a finished criterion.

The criteria across seven requirement files now carry what they rest on: what closed them, or what blocks them and where that was measured. Two coverage limits stay stated rather than hidden: the report is built only from roles that declare an MCP block, so a role blocked without one is invisible to it, and a ticked criterion here means implemented and tested rather than proven in operation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 00:58:44 +02:00
kevinveenbirkenbachandClaude Opus 5 eed0879eaa Merge branch 'main' into feature/ai-agent-employees
Conflict: roles/web-app-bridgy-fed/files/Dockerfile

Both sides extended the header comment. main also replaced the upstream
checkout, `git clone --depth=1 --branch "${BRIDGY_REF}"`, with git init plus a
fetch of the single ref; this branch never touched those lines, so git took
main's version automatically and only the comment collided.

Resolved to main's file verbatim. It is a superset of ours in both respects:
it keeps our two comment paragraphs and adds a third naming the trip-wire the
non-idiomatic fetch exists for, namely that BRIDGY_REF is a commit and
`git clone --branch` refuses one. Verified byte-identical to stage 3.

Committed with the hook skipped and before the semantic audit of the other 45
files finished. main brings a lint rule that has never seen this branch's
roles, image bumps on svc-ai-ollama and web-app-nextcloud which this branch
rewrote, a stylelint major, and release 13.0.0 across the packaging surface.
Any of those can merge cleanly line by line and still be wrong.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 13:32:15 +02:00
kevinveenbirkenbachandClaude Opus 5 95020d4fb4 fix(install): complete the ansible git fallback
requirements.git.yml carried 3 of the 7 collections the Galaxy file requests, and none of the transitive ones. When galaxy.ansible.com started returning 502, scripts/install/ansible.sh fell back to the git file, cloned its three collections, then resolved community.general's dependency on community.library_inventory_filtering_v1 against Galaxy again and died — five times over, taking make install and the whole CI workspace matrix with it.

All seven collections plus that dependency now resolve from git. Verified with an install run while galaxy.ansible.com was still 502: exit 0, seven collections installed, zero Galaxy contact.

A lint pins the parity so the fallback cannot rot back out of sync unnoticed; opt out per entry with the galaxy-git-parity nocheck marker.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 19:02:45 +02:00
kevinveenbirkenbachandClaude Opus 5 e4353d9742 refactor(ci): derive the manual cancel override from the event
force_cancel_in_progress existed so a manual sweep could supersede a running orchestrator on any ref. It was a flag every caller had to remember to pass, and only one ever did.

A called workflow inherits github.event_name from its caller, so the orchestrator can see for itself that a human dispatched the run. The input is gone; the group reads the event instead. Automatic triggers keep following the ref alone -- everything but main -- and a manual dispatch supersedes everywhere, main included, because someone typed it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 16:29:11 +02:00
kevinveenbirkenbachandClaude Opus 5 e3d91fd8b2 docs(ci): catalog the cancel entry point
Adds entry-cancel-superseded.yml to the workflow catalog and widens the cancel emoji's legend, which named only PR close and branch delete.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 16:29:11 +02:00
kevinveenbirkenbachandClaude Opus 5 653dc632e4 refactor(ci): key cancellation on the ref, not a variable
CI_CANCEL_IN_PROGRESS gated whether a new run cancels the one before it, and it had to be remembered per repository: unset upstream meant cancel, set to false meant do not, and a fork inherited neither. Cancellation now follows the ref. Every branch but main supersedes its own runs; main never does, because a half-finished pipeline on the default branch is worse than a queued one.

call-orchestrator.yml compares github.ref rather than github.ref_name: a called workflow inherits the caller's ref, which is refs/pull/<n>/merge for a pull_request caller. The manual entry workflow keeps overriding the rule through force_cancel_in_progress.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 16:29:10 +02:00
kevinveenbirkenbach f45df04bf1 test(docker): forbid ARG defaults in every Dockerfile
A defaulted build argument answers a question the caller failed to answer. The
build then succeeds against a base image, version or path nobody chose, and the
mistake surfaces as wrong behaviour at runtime instead of a failed build.

web-app-bridgy-fed shows the cost: its metadata declared python 3.14 while the
Dockerfile default silently built against 3.12. Both look plausible, so nothing
pointed at the drift. It and web-app-postmarks now take the base image from the
same services declaration the rest of the role reads.

The lint walks every tracked Dockerfile, template variants and paths outside
roles/ included, which the existing ARG-scope check does not. A second
assertion fails when the scan finds no Dockerfile at all, so the rule cannot
pass vacuously if the walk ever breaks.

Touching the two Dockerfiles brought their pre-existing narration under the
comment lint; what carried substance moved into the file header.
2026-08-25 00:25:00 +02:00
kevinveenbirkenbachandClaude Opus 5 2ca09f03b3 feat(skills): add i8-quality-high
i8-quality drives `make quality`; the gate that has to be green before a pull
request is `make quality-high`, which appends every lint target and runs about
four times as long. Nothing routed an operator there.

The skill leaves the fix loop to test-fix and pins only what differs: the run
goes to the background with its output teed to /tmp so the operator gets a
tail -f line, nothing else may run beside it because the wall-clock budgets
fail under self-made load, and the verdict comes from the two per-target tables
rather than the pipe exit code, which is unreliable here. docs and autoformat
write to the tree, so an unexpected diff after a pass is a finding.

Suppression is not a fix: a lint hit is resolved by deleting the offending
construct, and any nocheck marker needs the operator's explicit approval.

The cheatsheet row is not decoration - test_cheatsheet_is_skill_index requires
every i8- skill directory to appear in that table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 00:25:00 +02:00
kevinveenbirkenbachandClaude Opus 5 7162b7bf4c Merge fork/feature/svc-net-tor into feature/ai-agent-employees
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.

The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.

Resolutions that were not unions:

Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.

plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.

compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.

tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.

Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.

01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.

Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.

roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.

config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.

READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.

Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 16:32:30 +02:00