One conflict, in the category catalogue: this branch added a virt category for
the sandboxed container runtimes, the other did not.
Keeping our side alone would have closed the conflict and broken the build. The
same merge introduces a test requiring every category to declare a hanzi label,
unique among its siblings, and virt predates that rule and carries none. It now
declares 沙箱, checked against its ten siblings for a collision.
The build script is copied into the CDN volume directory, and this makes its
existence explicit at the point of use rather than relying on an earlier task
having created it as a side effect of creating its child.
A second branch needs its own checkout to be worked on in parallel, with its own
slot, subnet, ports and container names so two deploys do not take each other's.
The two skills route to the make targets that create and release one.
The sandbox gains the worktree directory as a writable path, without which the
checkout cannot be created and the skills are unusable. It is the only change
to the permission file: one write path, no deny or ask rule touched.
The schema lint demands a delegation block whenever a role claims to act on the
caller's behalf, pinning refresh, revocation and audience binding at one exact
version, because a rendered deployment bearer is a service account whatever the
metadata says. The lint can reject a missing block but cannot say why those
three facts and no others, so the design note carries that.
The requirement is brought up to date, and the agent cheatsheet gains the two
worktree skills.
The MCP discovery lookup answers with both the selected servers and the
rejected ones with their reason, but the global bound its whole result to
MCP_DISCOVERED_SERVERS, which every consuming role iterates as a list of
servers. The rejections are kept under their own name so the reason a provider
was dropped stays available, and consumers read the selected list.
The probe was left behind when the mcp block moved out of services, so every
lookup it makes resolves against a path that no longer exists and the contract
check fails before it reaches a provider.
It survived the sweep because the check for dangling references filtered paths
containing mcp-adapter, meant to spare the genuine services.mcp-adapter service
entry, which excluded this role's whole directory by name.
WordPress serves MCP through two addons rather than a sidecar, because the
application already carries the extension point.
The mu-plugin is the entire tool surface. The MCP Adapter publishes an ability
only when it carries meta.mcp.public, so whatever is not registered in that file
cannot be reached over MCP no matter what else the installation has. Every
ability reads already-published content and carries its own permission
callback, which is not decoration: the adapter authorizes the transport, the
callback authorizes the operation, and dropping it would let any authenticated
caller reach the ability.
The second addon fetches the release asset rather than the tag's source
archive. The source archive unpacks as mcp-adapter-<version>, which no longer
answers to the slug the generic activation step uses, and it ships no vendor
directory.
Three Playwright specs cover what needs no credential: both addons active, the
abilities registered, and a guest served nothing. The mcp role joins the role
catalogue instead of replacing it.
n8n is the only role that speaks MCP in both directions, consuming other
servers and serving one of its own.
The provisioner carries the auth cookie by hand rather than through a
CookieJar. n8n sets it with Secure, which is right for the browser-facing
origin, so a jar drops it over the plain-http internal address and every call
after the login answers 401. Setting N8N_SECURE_COOKIE=false would have fixed
the provisioning at the cost of the cookie the browser actually relies on, so
the session is carried explicitly instead.
The mcp role is declared in the role's rbac metadata, without which the group
path lookup hard-fails, and a Playwright spec proves the surface serves a guest
nothing. That check needs no credential, which is why it belongs in Playwright
while the contract probe with the real bearer stays at deploy time and out of
the trace.
Four defects in one chain kept the tool servers from ever being served.
The connections never arrived. The provisioner read them back from the config
API, but ENABLE_PERSISTENT_CONFIG=false means the environment feeds the running
process and is never written to the table that API serves, so a fresh
deployment answered with an empty list however long the deploy waited. It now
falls back to the declared list it was handed.
The groups stayed empty. Group resolution returned only the id and dropped the
group itself, so membership could not be written. It returns both, and the
member list is reconciled explicitly, because the OIDC claim cannot revoke on
its own: Open WebUI drops a stale membership only while the claim is non-empty,
so a user who loses their last group keeps it until some later login carries
another.
Success was not checkable. The script printed a count and exited zero even when
nothing was granted. It now refuses to report success when servers are declared
and none was granted, and emits the resolved group ids, which are persisted and
rendered back into the environment. Without that second render a bare container
restart drops the grants, since the environment is authoritative on every start
and the ids did not exist when it was first written.
The signing key is pinned because Open WebUI otherwise generates it into the
process environment only, where a container exec sees a different value than
the server.
A user password reaches its application through Ansible, a shell, a container
exec -e and finally the runtime's own environment reader. The generator seeded
it from the same alphabet as the become password, punctuation included, and
that does not survive the chain intact.
Three roles had already worked around it by declaring a separate alphanumeric
credential for an identity that their meta/users.yml already declares, and
saying so in the description. Removing those without fixing the generator would
have reintroduced exactly the bug they were built against, so the generator is
fixed first and all three now read the user's own password. At length 64 the
smaller alphabet costs nothing worth measuring, and the become password keeps
the old one because it travels a different path.
Zammad additionally used the bot's password for its wizard-bypass admin, one
secret for two users. Each has its own declaration and now its own value.
A test asserts the generated password is alphanumeric, so a future change to
the alphabet fails there rather than inside a Ruby ENV.fetch in a container.
The published image lies about its version: flowiseai/flowise 3.1.3, 3.1.4 and
latest all ship package 3.1.2, measured in the running container rather than
inferred from the tag. An unpinned global npm install layer is reused between
releases. We build from the declared version instead.
The build installs locally rather than globally because flowise declares node
^24 from 3.1.3 on while resolving better-sqlite3 to 9.6.0, whose binding.gyp
pins -std=c++17 against node 24 headers demanding C++20. That version cannot
build there at all, and an npm override is only honoured for a local install.
Three upstream behaviours were read at the pinned source before being worked
around: cookie authentication is routed only behind an x-request-from header,
the key controller rejects an empty permissions list, and key creation answers
with the whole key list instead of the new key.
The identity password is written straight into the user row because flowise
accepts a password at registration only, and registration works once; a
redeploy carrying a rotated credential otherwise meets an account whose hash
still encodes the previous one.
Server registration refuses anything that does not speak SSE. This flowise
authorizes with an SSE toolkit, so registering another transport would claim a
connection that cannot be made.
A defaulted build argument answers a question the caller failed to answer. The
build then succeeds against a base image, version or path nobody chose, and the
mistake surfaces as wrong behaviour at runtime instead of a failed build.
web-app-bridgy-fed shows the cost: its metadata declared python 3.14 while the
Dockerfile default silently built against 3.12. Both look plausible, so nothing
pointed at the drift. It and web-app-postmarks now take the base image from the
same services declaration the rest of the role reads.
The lint walks every tracked Dockerfile, template variants and paths outside
roles/ included, which the existing ARG-scope check does not. A second
assertion fails when the scan finds no Dockerfile at all, so the rule cannot
pass vacuously if the walk ever breaks.
Touching the two Dockerfiles brought their pre-existing narration under the
comment lint; what carried substance moved into the file header.
The traffic light said "gate" everywhere else in the pipeline, on the PR
scope check and on the dependency gate, so reusing it for the deploy-mode
order made an ordering knob look like another pass/fail barrier. A crawler
reads as what sequencing is: segments following each other in a line.
Both failure-report entries in the orchestrator move to the ladybug in the
same commit, because they already held the caterpillar and one workflow
cannot spend the same glyph on ordering and on failures. The ladybug is what
the bug-report issue template already uses.
Only the test fixture spells the glyph out; the parser derives its segment
heads from the workflow's own run-name, so it follows the change on its own.
The traffic light stays where it means a gate, and as the release-candidate
symbol in the glossary.
A run title created before this commit still carries the traffic light, so
recovering its sequencing input from the title yields nothing and a retrigger
leaves that one input on the workflow default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The retrigger already sent its role selections as display names, so its run
title read like the deploy jobs it spawned. The two aliases that dispatch a
manual run went straight to `gh workflow run` with raw ids, and their titles
kept showing the dash-joined prefixes the display names exist to replace.
`python -m utils.roles.display` gains the encoding direction it was missing;
decoding stays the default so the whitelist resolver's existing call is
untouched. i8puma and i8puwl pipe their role list through it, after
`i8source`, so the venv and the repo root are in place by then.
Nothing on the consuming side changes. The whitelist resolver, the discovery
query and the sequencing and plan CLIs already decode, and decoding passes a
raw role id through untouched, so a run dispatched by hand from the GitHub UI
keeps working with plain ids.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ten applications with no MCP server of their own get one without a line of code
inside them: qdrant, checkmk, fider, jellyfin, listmonk, pretix, prometheus,
snipe-it, zammad and libretranslate.
The sidecar is confined rather than trusted. It runs read-only, drops every
capability, forbids privilege escalation, publishes no port and joins only its
own provider's network, carrying neither that provider's alias nor the consumer
networks the provider itself joined. Its tool surface is a JSON contract, not
code: which tools exist, which upstream path each maps to and what shape the
arguments take. Adding a tool is a line of JSON, not a new image.
The image is built locally from the audited adapter source with the base image
as a build argument, so all ten share one reviewed build instead of ten upstream
dependencies. Each role provisions its own read-only identity in the language
its application speaks, and every adapter include is guarded by the role's MCP
flag, so switching it off leaves no container behind.
The 21 env templates added two commits ago left the persona flags off entirely
in most roles. biber.js has no native-login branch at all -- with
SSO_SERVICE_ENABLED false it never authenticates and then hard-fails at :201
rather than skipping -- and admin.js gates its native fallback on !oidcEnabled.
CI deploys every matrix variant, and variant 1 of these roles pins
services.sso.enabled false, so each of them was a guaranteed red job that had
never been observed because the specs had never run.
Nine flags are static true: chess, confluence and jira pin sso.enabled false in
meta/services.yml with no variant override and provision no native admin, so
there is no login surface in any variant. Eleven follow services.sso.enabled in
the ternary web-app-baserow already uses, for the roles that do have an
SSO-enabled variant where the personas should run. pgadmin and phpldapadmin
lose the credential keys that a permanently blocked persona can never read.
Eleven nocheck rationales cited "the sys-svc-mail integration tests" as the
coverage that justifies not gating on EMAIL_SERVICE_ENABLED. No such tests
exist: roles/sys-svc-mail* contain no tests at all, and
tests/integration/infrastructure/services/test_mailu_dependency.py is a static
YAML parity check. The real delivery coverage is web-app-mailu's own spec,
playwright.spec.js:154. Each is now a fact checkable inside its own role.
Eight more claimed the CSS injector's origin is asserted by the shared CSP
baseline. personas/utils/csp.js:96 bails with `if (!host) continue`, and
injectorBaseUrl("css") reads CSS_BASE_URL, which no consumer role renders --
so the assertion is a guaranteed no-op everywhere. Rewritten to the
mechanically true reason: assertCspInjections resolves the injector through a
computed loop variable, never a literal gate argument, so test_spec_env_gates'
literal-only regex cannot see it.
The CSP hole itself is left standing: rendering CSS_BASE_URL would arm the
injection assertion for every consumer role at once, which is a change of a
different size than correcting a comment.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The eight roles whose MCP server is built into the application now end their MCP
step with the shared contract probe. It builds URL and Authorization header from
the role's own declaration, calls initialize and tools/list, invokes one
deterministic read tool per role, and compares the served surface against the
declared allowlist. It cannot be a Playwright test: the check needs the real
provider credential and a Playwright run would capture it in its trace.
Eleven sites wrote the MCP token into the store under the administrator's
identity while their metadata declared an owner of their own. The schema lint
forbids administrator as a credential owner, so declaration and storage
disagreed, and the probe would have looked for the bearer under the declared
owner and found nothing. They now read the declared owner.
Blocking a persona removes the only end-to-end proof that a role's auth chain
works. docs/contributing/artefact/files/role/playwright.specs.js.md:146 has
always required each flag to carry a one-line nocheck stating the role contract
that justifies it, plus a matching paragraph in the role's README or TODO.
Nothing enforced it, so 89 flags across 46 roles carried neither.
The lint mirrors test_has_spec's shape and uses is_suppressed_at with
mode="line-above", so the marker must sit directly above its flag -- a blank
line between them breaks the association, which is what the persona helper's
reader will look at. A literal =false blocks nothing and is exempt; the
sso-conditional ternary is not, because it blocks in every variant that pins
services.sso.enabled false.
Writing 89 reasons turned out to be the point. Five roles audited in one pass
blocked for five entirely different mechanisms, all of which read as the same
shrug behind a bare =true: pgAdmin's login id is the account e-mail rather than
the Keycloak username; phpMyAdmin authenticates as the MariaDB root account;
GitLab's seeded password belongs to the built-in root, not the administrator;
Mastodon's tootctl accounts create never sets a password; phpLDAPadmin binds
with an LDAP DN and a different secret. Further out: MediaWiki removes the form
that would accept the Keycloak secret ($wgPluggableAuth_EnableLocalLogin=false)
and bounces anonymous requests back into the IdP, so the unauthenticated
landing the contract asserts can never be observed; Mattermost Team Edition has
no native OIDC, so Keycloak rides the GitLab OAuth slot behind a relabelled
link; MinIO's console returns redirectRules: null and renders no SSO button at
all; XWiki's administrator is an XWikiUsers object with no password property.
Two flags turned out to be unjustifiable and are gone rather than annotated.
web-app-mailu gated PERSONA_GUEST_BLOCKED on services.dashboard.enabled, but
runGuestFlow does one anonymous goto and asserts no signed-in marker -- it
never touches the dashboard. A leftover from a helper version that entered
through the dashboard account menu. And web-app-mig's rationale claimed the
role is an in-app module of web-app-dashboard; it has its own compose stack,
healthcheck and canonical domain, and dashboard is merely a tile consumer.
Nine of the reasons say plainly that the block is not a role property but a gap
in the shared persona helpers -- decidim's sign-out is a data-method link the
generic click cannot activate, fider's SSO entry is a two-step modal, taiga,
wordpress, xwiki, odoo and penpot bind OIDC on a path the personas never visit,
jenkins blocks unconditionally although its OIDC variant looks runnable, and
opentalk's own spec proves both personas work. That list did not exist before;
it is the actual yield here, not the 89 comment lines.
45 pre-existing comments came into the comment lint's scope because this sweep
touched their files -- it only inspects files that differ from HEAD. 28 of them
were the banner "Service gate flags rendered for closure.", every one sitting
above a line that is not a service gate flag at all, orphaned by the
alphabetical re-sort these templates once got.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The block sat under services but never behaved like a service. A service has
exactly one providing role, declared with provides, so the registry can map the
key to that role and consumers bind their flag to its presence. No role declares
provides: mcp. There are seventeen providers and the pairing is done by
allowed_consumers inside the block itself.
Every rule that enforces the service model therefore needed an exemption for it:
a shared-provider suppression in fourteen roles, a playwright-service-flag
suppression, and two freshly added exemptions in the dynamic-flag and
variants-static-override checks. Four independent rules, four escape hatches,
one cause. They are all gone now, together with a bond key that only the
resource budgeter reads and only for entries under services, and a wordpress
addon bridge that pointed at a service block that no longer exists.
The block moves to meta/mcp.yml. The applications loader maps meta/<name>.yml to
applications.<app>.<name> generically, so the move needs no wiring; the file
takes the same shape as meta/rbac.yml. Every config path follows, the variant
overrides are lifted out of their services map, and the vocabulary module leaves
the services package because it no longer describes one. roles_with_service
gains an explicit topic parameter rather than an implicit fallback, so nothing
silently resolves against the wrong container.
Files that are otherwise unrelated to this move are included where the committed
tree would otherwise keep pointing at a path that no longer resolves: thirteen
vars files, eleven playwright env templates and one task file.
The role's env template shipped no persona flags, so both authenticated
personas ran unconditionally. Every CI variant is deployed, and variant 1 pins
services.sso.enabled false -- there the administrator has nothing to log into:
tasks/05_administrator.yml creates the account with `tootctl accounts create`,
which neither sets nor returns a password, so ADMIN_PASSWORD (the Keycloak
secret) cannot drive Mastodon's native /auth/sign_in. biber is worse off: no
Mastodon account is provisioned for him at all, and the OIDC sign-in wired at
templates/env.j2:61-72 is the only path that would create one. Both flags now
follow services.sso.enabled, in the same ternary web-app-baserow already uses.
files/playwright/test-seaweedfs.js gated only on seaweedfs, but it drives the
same admin journey -- Keycloak sign-in, /settings/profile, avatar upload.
services.sso.enabled and services.seaweedfs.enabled are independent
group-membership expressions, so an inventory carrying seaweedfs without
keycloak was a hard failure. It now skips when the administrator persona is
blocked.
meta/services.yml carried `# nocheck: playwright-service-flag — object store,
no Playwright surface` above the seaweedfs key. The commit that added
test-seaweedfs.js made that false, and the suppression was dead anyway: the env
template declares SEAWEEDFS_SERVICE_ENABLED, so test_env_services_match never
needed an exemption. Deleted rather than reworded.
Two nocheck rationales asserted things that are not true: that mail delivery is
"covered by the sys-svc-mail integration tests" -- no such tests exist, the
coverage is web-app-mailu's own spec -- and that the shared CSP helper asserts
the CSS injector's origin, which personas/utils/csp.js:96 refutes with
`if (!host) continue` since no consumer role renders CSS_BASE_URL.
Not verified: test-seaweedfs.js has still never run against a live stack. Its
selectors were authored against upstream Mastodon's UI without execution; the
new guard only keeps it out of the variant where it could not pass, and says
nothing about whether it passes in the variant where it now runs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every entry in LISTMONK_SETTINGS was one boot late. Listmonk reads settings
into koanf once at startup (cmd/main.go:186 initSettings) and builds the media
store from them (:205); the only reload trigger is SIGHUP (:315). The role
writes them with a direct UPDATE from 02_manager_ops.yml:34, after the single
`notify: compose up` in the whole role has already been flushed at :8. A DB
write reaches neither koanf nor the settings API, and `changed_when: false`
meant the task could not notify a handler even if one existed.
It never self-healed either: notify fires only on a changed config.toml, which
on the second run is identical, so no recreate. In the CI shape -- fresh stack,
one deploy, then Playwright -- the settings were guaranteed inactive.
What was therefore never in effect: upload.provider=s3 and the whole
upload.s3.* block (uploads went to the container filesystem, not the bucket),
security.oidc (so /admin/login rendered no SSO control at all), smtp, and
app.root_url -- which is why the deployed instance's public pages linked to
http://localhost:9000.
Proven against listmonk/listmonk:v6.2.0 with a throwaway Postgres: with the
settings written and no reload, /admin/login has no OIDC form; after
`docker kill --signal HUP` the same URL renders
`<form method="post" action="/auth/oidc">`. `container recreate` is the
existing shared handler, already used for exactly this in
web-app-peertube/tasks/oidc/enable.yml.
The OIDC control is still not reachable by the shared persona helper after the
reload: clickOidcLoginLink prefers the strict locator, and Listmonk's native
submit button's accessible name is exactly "Login" while the OIDC button is
"Login with <name>" -- so the strict matcher takes the wrong control. There is
no setting that hides the native form; cmd/auth.go renderLoginPage hardcodes
PasswordEnabled. appearance.public.custom_js is a real v6.2.0 setting, injected
into /admin/login and served same-origin, so a few lines of JS submit the OIDC
form and the second hop disappears. The shared helper stays untouched, which
keeps the 61 other roles that route through it out of this.
Measured with the helper's own source driving a real browser: no reload ->
no form; reload alone -> reached: false; reload plus custom_js -> reached: true.
The administrator persona is unblocked accordingly. biber stays blocked, and
for the third distinct reason this role has been given: Listmonk matches OIDC
users by e-mail and provisions no biber account, cmd/auth.go OIDCFinish 404s
unless security.oidc.auto_create_users is set, and createOIDCUser needs a real
default_user_role_id -- the seeded database holds only Super Admin.
Three nocheck rationales in playwright.env.j2 asserted things that are false:
that mail delivery is "covered by the sys-svc-mail integration tests" (no such
tests exist; the coverage is in web-app-mailu's own spec), and twice that the
CSP helper asserts the injector origin (personas/utils/csp.js:96 bails with
`if (!host) continue`, and no consumer role renders CSS_BASE_URL). Rewritten to
what is actually true.
Not fixed here, recorded in TODO.md: security.enable_captcha,
security.captcha_key and security.captcha_secret do not exist in v6.2.0 -- the
live table carries one security.captcha object -- so all three UPDATEs match
zero rows and hCaptcha has never been active. Repairing that would switch a
captcha on for the public subscription form, which is a behaviour change nobody
asked for.
Unverified: the logout leg. Whether inAppLogout finds Listmonk's Vue user menu,
and whether assertCspInjections passes against the admin SPA, has not been run.
The first deploy settles both.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The category labels now run together into a single block instead of being
separated from each other, the glyph sits flush against that block, and the
only `·` left is the one between the categories and the role's own fragment:
`🐳网络应用·Nextcloud` where it used to read `🐳 网络·应用·Nextcloud`.
The deploy-job parser drops the whitespace requirement behind the glyph in
the same move. Without it no job name from a fresh run matches any more, and
the retrigger would find zero roles without reporting an error.
The name still never holds a space, so the heading keeps collapsing its own
whitespace to `·` and the space-separated whitelist and priority inputs stay
parseable with more than one role. Empty parts are dropped when joining, so a
role with no category at all (pkgmgr, test-e2e-*) does not gain a leading
separator.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The script returns its anomaly count as the exit code and three of the five
cases assert on it, so the run must not raise. check=False is the default,
but stating it keeps someone from adding check=True and turning those three
assertions into a CalledProcessError.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
slaptest converts slapd.conf into slapd.d and only then runs slap_startup,
which opens the database that this very entrypoint seeds a few lines later.
On a fresh volume that open fails, slaptest exits 1 with "Restore from
backup!", and under set -e the container dies. It heals by accident because
slapd.d is already written, so the task swarm starts next takes the
has_cn_config branch, but it costs a dead task and its startup delay on every
fresh volume.
Measured against slaptest 2.6: the failing run writes cn=config.ldif,
olcDatabase={-1}frontend, {0}config, {1}mdb and every schema, so the
conversion is complete and only the exit code is misleading. A parse error
behaves the opposite way and writes nothing at all, as does an unwritable
target directory, so the has_cn_config check right below still catches every
case where the conversion did not happen.
Not -u: that switch makes slaptest a pure dry run. It exits 0 and leaves the
target directory empty, which would turn a single dead task into a container
that can never start.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first version asked dockerd which containers it still knew and treated
the rest as orphaned. That predicate can never fire in time. dockerd keeps
reporting an orphan as running for minutes, then synthesises an exit it never
verified, so the container is either still listed as running or already listed
as exited, and `docker ps -a` covers both. On one captured manager 17 of 18
orphaned scopes carried an Exited container that was still listed, so they
were skipped; the reap found nothing and the unconditional dockerd restart
that followed took the node's tasks down for no gain.
A restarted containerd re-adopts none of the shims it left behind: across 13
observed restarts, 96 of 96 were leaked and the counts match the orphaned
scopes node by node. Age is therefore the exact criterion, and it holds the
moment the damage happens rather than minutes later. It also drops the docker
CLI from the script entirely, so a wedged daemon can no longer hang the play.
Around that: the stop tolerates systemctl's "no such unit" because the scope
vanishing is the wanted outcome, both calls are bounded so a hung unit cannot
block forever, each stopped scope is verified gone, and dockerd is polled
until it is active instead of trusting the restart's return code. A unit that
systemd no longer knows answers 0 for its start timestamp, which would sort
before every real one, so that answer is treated as unknown rather than old.
The tests drive the real script against a stubbed systemctl and cover the
selection, both filters, and every failure branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI deploy jobs were titled with the raw role id, so a run page read as a
column of dash-joined prefixes. They now carry a display name: the category
path as hanzi labels, then the role's README heading or the part of the id
the categories do not already say, whichever is shorter.
utils/roles/display.py owns both directions. Encoding happens once, at
discovery time, as a `label` key on every matrix entry; the workflows render
`<glyph> ${{ matrix.label }}` and keep the glyph as a literal because the
retrigger reads the deploy mode off it.
Decoding is what makes the retrigger keep working. It accepts a display name
with or without its variant shard and a bare role id unchanged, so job names
from runs that predate this still parse and a hand-typed dispatch never has
to spell hanzi. The parser therefore stopped matching an id-shaped tail and
asks the codec instead; a job the codec does not recognise is skipped rather
than counted as a role.
The role selections go out as display names too, so the run title reads like
the jobs it spawns. Four consumers decode them back, each the single place
the value is actually used: the whitelist resolver before it checks the role
directory, the discovery query on its env, and the sequencing and plan CLIs
on their arguments.
Every fragment is joined by `·`, including before the title, and the heading
loses its whitespace and any emoji variation selector. A display name holding
a space would make the space-separated whitelist and priority inputs
unparseable with more than one role.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds a `hanzi` field to every node in meta/categories.yml plus a lint that
keeps it there. The labels are compact two-character fragments that compose
into a display path (sys-ctl-cln reads as 系统·控制·清理), so a category tree
that is otherwise only navigable by dash-joined role prefixes gains a short
human-facing form.
The lint asserts presence on every node and uniqueness among siblings, not
across the whole tree: the labels are path fragments and repeat across
branches on purpose (web-svc and svc both carry 服务), so sibling scope is
what actually guarantees a unique display path.
test_paths.py gained `hanzi` in its attribute skip list; without it the new
key reads as a sub-category and the test demands roles/web-hanzi,
roles/svc-hanzi and friends.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The backend has never carried a healthcheck, in neither the include-template
era nor after the probes moved into services.yml. That is why it stayed
invisible when its containerd shim was orphaned: docker had dropped the task
record while the process lived on, every exec into it returned "task not
found", and swarm kept reporting the replica as running because nothing ever
declared it unhealthy. Frontend, postgres, redis and openresty recovered in
that same run precisely because they are probed.
The tcp flavor asks for nothing the image does not already provide: it opens
a bash socket against the port declared under ports.internal and greps the
response for a status line, so it needs neither curl nor wget. Sixteen roles
across node, ruby, python and php images already probe this way. The five
minute start period covers the JVM boot.
The one assumption left is that the server binds a loopback-reachable
address. Penpot documents no host setting and its upstream compose does not
probe the backend either, so this is inferred from the frontend reaching
backend:6060 from another container. A probe that cannot pass would turn the
next CI run red rather than damage anything.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The extension tarballs ship no composer.lock, so `composer install --no-dev`
degrades to a full solve that still resolves require-dev, as composer states
itself. Both configured addons pull mediawiki-codesniffer 50.0.0 there, which
pins php_codesniffer 3.13.5, and once an advisory landed against that version
composer refused to load it and the solve became unsatisfiable. The extension
install failed with rc 2 without a single line of this repository changing.
Stripping require-dev from the manifest before composer sees it keeps the
solve on the runtime constraints, which is what --no-dev was asking for. The
stripper decodes to objects rather than associative arrays, because an empty
JSON object would otherwise round-trip into an empty array and composer
rejects `[]` where its schema wants `{}`.
The work that runs inside the container moved out of the embedded shell string
into install/helpers/, which removes three levels of quoting from the calling
script and makes both helpers runnable by hand for debugging. The whole helper
directory is copied in, so adding a helper means dropping a file in there and
nothing else.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The worker healthcheck passes trivially while the bootstrap marker is
absent, which is what keeps swarm from reaping the worker before the app
bootstrap runs. That pass also ends the start period early, so the grace
the 300s start_period was meant to give is gone by the time the marker
appears and the probe turns into a real horizon:status check.
Marker visibility and horizon startup are coupled: the worker entrypoint
blocks on the same file, so horizon only begins starting at the moment the
probe starts demanding it. In swarm the marker additionally lives on the
NFS-backed cluster state, so a worker on another node sees it appear with
attribute cache latency on top.
With retries: 1 the first probe that lands mid-boot was terminal. The task
was killed with exit 137 roughly a minute into a nominal five minute start
period, and the container it left behind kept its anonymous bootstrap volume
attached. Five retries at a 60s interval restore the five minutes the start
period documents, and match the retry count the http probe flavors use.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The bootstrap check and the report disagreed about which containers hold an
anonymous volume. The check enumerated running containers and filtered them
in the Go template, the report asked the engine for all containers including
stopped ones, and both carried their own copy of the mount-path template. A
container reaped mid-init therefore kept its anonymous bootstrap volume
attached, escaped the whitelist, and was counted as an anomaly, so the health
unit exited non-zero on every retry while the cleanup it triggers could not
prune a volume that is still attached.
The check also concatenated the destinations of every holder into one string
before comparing it against the bootstrap path, so two containers holding the
same volume yielded a doubled path and never matched.
Both lookups now go through one helper each and the bootstrap decision is per
container, which removes the divergence by construction rather than by
keeping two call sites in sync. Asking the engine to filter also drops the
inspect call per container that the scan performed for every volume.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A containerd restart under a live dockerd orphans the container shims. The
new containerd does not re-adopt them, dockerd keeps stale task handles and
force-deletes the container records while the processes stay alive and keep
their file locks, so every replacement task is blocked by its own
predecessor: mariadb aborts on the Aria and InnoDB locks, slapd returns
LDAP_OTHER from the LMDB reader lock, and an exec into a healthcheck-less
service fails with "task not found".
Package versions do not reveal that state, because containerd restarts into
the version it was upgraded to and dockerd is not upgraded at all. The
signal is the unit start order: containerd entering active later than
dockerd means it came up underneath it.
Reaping is done on the systemd scopes rather than through the engine,
because dockerd has already dropped the records and its own force-kill
cannot reach the surviving processes.
Runs one node at a time: a package upgrade restarts containerd on every node
of the cluster at once, and simultaneous dockerd restarts can cost the swarm
managers their quorum.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A user definition that declares no password falls back to
{{ 42 | strong_password }}. That is a template, not a value: it is re-rendered
at every use site, so the task that creates an account and the task that later
authenticates as it read two different secrets. The account exists and the
login fails. Inventory creation now decides the value once, after the roles are
resolved and next to the role credentials that were already generated there.
The scope follows the deployment: only roles resolved into this inventory
contribute users, an existing password is never overwritten, and a second run
writes nothing.
Building it duplicated what the package already carried three to seven times
over: an ensure-map helper defined three times, the same fifteen-line
round-trip load repeated seven times, and the vault encrypt-and-reparse dance
twice. They now live in one module, the round-trip counterpart to the existing
read-only yaml_io. host_vars drops from 213 to 130 lines, and its two override
entry points share the merge they used to copy. Behaviour is unchanged, and
that is measured rather than asserted: the committed versions and the new ones
were driven against the same host_vars and produced byte-identical output for
all six writers, comment placement and vault block layout included.
The adapter probes move into their own folder because a shared file-name prefix
reads as a folder in disguise, and the lint that enforces it scans the tracked
file list, so the repository was failing on it. A test that was defined at
module level instead of inside its class, and therefore never ran, is indented
back in.
The lint shipped in the previous commit had three defects, all reproduced in a
throwaway copy of the tree rather than argued.
Its documented opt-out could not be used. The docstring said to put
`# nocheck: playwright-env-template` on the first line of the role's spec.
Every spec is a CommonJS .js file, where a leading `#` is a hard parse error --
`node --check` refuses it. Following the instruction broke the file the lint
was protecting. It is `//` now, and the marker may sit anywhere in the head
rather than on line 1, which the mandatory `require` already occupies.
Its suppression check was a bare substring match: `if _RULE in
read_text(...).split("\n", 1)[0]`. Any prose containing the rule name disarmed
it -- proven by deleting a template and writing
`// TODO: someone still owes this role a playwright-env-template`, which the
lint then passed. A typo in the rule name disarmed it just as well. It now uses
`is_suppressed_in_head` from utils/annotations/suppress.py, the shared grammar
every other lint here consumes; this was the only one with a hand-rolled parse.
Its scope followed test_has_spec's `web-app-` prefix, but the mechanism it
guards does not: discover_playwright_roles.py globs
`rglob("templates/playwright.env.j2")` with no prefix filter. Six web-svc-*
roles ship specs. All six carry a template today, so the hole was latent, but
consistency with a narrower sibling is not correctness.
Verified in the copy: missing template -> fails; bare mention of the rule name
-> still fails; a real `// nocheck` in the head -> passes, and node --check
accepts the file. In the tree: tests/lint + tests/integration 532 passed,
1548 subtests, and the lint itself green at 82 specs against 82 templates.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role was a scaffold: tasks/main.yml was a two-line comment, there was no
compose.yml.j2, no domains.yml, no volumes.yml, no env.j2. It deployed nothing
while carrying a Playwright spec and two matrix variants.
It also described the wrong project. meta/main.yml and meta/info.yml pointed at
jaywink/socialhome -- a Django ActivityPub hub. The target here is
social-home-io/socialhome: created 2026-04-21, MPL-2.0, aiohttp + SQLite +
Preact, federating peer-to-peer over Ed25519/WebRTC rather than ActivityPub.
Different codebase, no fork relationship.
It turns out to be the simplest web-app in the repo. One published multi-arch
image (ghcr.io/social-home-io/socialhome:2026.6.16, pinned), no Postgres, no
Redis, no queue, no worker, no collectstatic -- SQLite in one volume, one port,
migrations applied in-process at startup.
Two things the source forced that reading the README would not have:
- [standalone].external_url is reachable only through the TOML file, never
through SH_* env (config.py:238-261). Without it routes/pairing.py answers
422 NOT_CONFIGURED and federation pairing is dead, so the role renders a
config and mounts it as type: config.
- provision_admin() runs from SH_ADMIN_PASSWORD but never calls
mark_complete(), so / keeps serving the setup wizard instead of the login
form -- upstream's own docstring claims otherwise. 01_setup.yml closes it
with an idempotent POST to /api/setup/standalone.
No SSO sidecar, deliberately, against the pgadmin model this otherwise copies:
oauth2-proxy would redirect POST /federation/inbox and /api/pairing/accept to
Keycloak and break the one thing the software exists for.
Verified against the real image rather than reasoned about. The http_status
probe's exact argv ran inside the container and returned 0. /healthz answers
with the byte-for-byte shape the spec asserts. The setup POST returns 201 then
409 and flips setup_required true -> false, which is the whole justification
for 01_setup.yml existing. [standalone].external_url lands in the inbox_url of
POST /api/pairing/initiate. The container runs as uid 10001, matching
meta/users.yml.
Both persona blocks are proven, not assumed: biber gets 401 from the token
endpoint, and the 97 static files contain zero occurrences of logout / sign out
/ abmelden, so utils/logout.js has nothing to click. Injecting a control the
way web-app-prometheus does was rejected -- that model points its button at
OIDC.CLIENT.LOGOUT_URL, which does not exist for a role with no OIDC.
swarm stays disabled for the first round. The orchestrator chains
swarm -> compose -> host under mode_fail_fast, and sequencing measures serial
at 198 jobs, so a red swarm job here would skip 92 unrelated jobs and leave 85
roles untested. Compose-only drops that to 10. Flip it once compose is green.
Adversarial review caught five defects before this landed, three of them
certain rather than likely:
- SOCIALHOME_INTERNAL_URL hardcoded 127.0.0.1, which sys-svc-container rewrites
to the bridge gateway whenever DOCKER_IN_CONTAINER is set -- from inside the
Ansible container that address is the Ansible container, so both setup tasks
would have burned five minutes of retries and failed the play.
- SH_WEBRTC_STUN_URL sat outside the coturn guard while the TURN pair sat
inside it. Upstream reads by env PRESENCE, so the empty value would have
beaten the in-app default and left WebRTC with no STUN at all.
- meta/csp.yml whitelisted blob: on worker-src. The bundle registers no service
worker and constructs no Worker; the only blob of JavaScript is an audio
worklet, and worklet modules load under script-src. Voice recording would
have died with the app's own "Audio processing unavailable".
- tasks/main.yml hand-rolled meta: flush_handlers instead of including
sys-svc-compose's finalize.yml, which silently skipped utils/post_deploy.yml.
- 01_setup.yml was gated on IS_STACK_HOST but not run_once, so a three-manager
swarm would fire three concurrent setup POSTs at one ingress port.
TODO.md carried a claim that steered the CSP design and was simply false --
that img-src is not a directive csp_filters.py emits. It appends
"img-src * data: blob:" unconditionally at line 383.
lifecycle: alpha puts the role in the CI deploy matrix before it has ever run
anywhere, deliberately: the first CI round is its first execution. README and
TODO say so rather than claiming a maturity it does not have.
Still unverified: no browser has enforced the CSP, the swarm path was only
rendered, TURN against a real coturn is untested, and arm64 has not been
exercised. TODO.md lists them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The mcp role sat in the administrator's unscoped roles list. That list carries
no application, and the RBAC builders admit any user whose roles contain the
name, so the single entry granted mcp in every deployed application at once.
It is replaced by application_roles, a mapping of application to roles, which
the user cache now reads and passes through. Both directory builders resolve
membership through one shared function, so the Keycloak and LDAP paths cannot
drift.
The member list never reached its consumer. mcp_group_members yielded username
strings while the Open WebUI reconciler calls member.get("email"), so the first
real grant would have crashed the deploy. It went unnoticed because the
reconciler returns early on an empty list and nobody held the role. The filter
now yields username and email, which is what the reconciler needs to match
accounts Open WebUI created from an OIDC claim, whose display name need not be
the username. A test drives the real filter output through the real script so
the two sides cannot diverge again.
The Playwright scenario walks the boundary end to end: biber is served no MCP
tools, the administrator adds him to the provider's LDAP mcp group in LAM, and
after signing in again the tools are served. LAM is reached only when it is
present in the inventory, so no service edge or variant is added for it.
The administrator spec inverts its last assertion. With the role scoped, an
administrator holds no application's mcp group, and BYPASS_ADMIN_ACCESS_CONTROL
is false, so the tools must not be served to them either.
A provider overlay keyed on services.mcp.enabled and .shared admitted every
MCP-capable role, servers included, because those flags carry no direction. Each
provider's network therefore collected every other provider and the result was a
mesh in which any sidecar could reach any application. The new mcp_client
consumer kind additionally requires services.mcp.direction to be client or both,
and the four provider overlays switch to it.
A sidecar rendered its networks with the same lookup as its provider and
inherited that provider's consumer attachments, which it neither needs nor
should have. own_network_only keeps it on the provider network alone. A provider
whose own topology is default_net contributes no attachment line at all, and a
bare networks: key is not a list, so the flag returns nothing in that case and
lets the sidecar fall back to the project default, which is that provider's own
network.
test_has_spec.py forces every roles/web-app-* to ship a Playwright spec and
calls a missing one a hard error, because the persona contract must never be
silently absent from the deploy capstone. Nothing forced the other half. The
runner discovers roles by rglob('templates/playwright.env.j2')
(discover_playwright_roles.py:53), so a spec without that file is staged
nowhere and executed never.
21 of 76 web-app roles were in exactly that state. Verified rather than
inferred: running the discovery filter directly returns 61 roles and none of
the 21 is among them, `make compose-playwright` refuses without a rendered
.env from a prior deploy, and `git log --all` on those paths shows the file
never existed on any branch -- the single hit is web-app-gitlab on a fork
branch. Their specs have never run once. 15 of the 21 deploy in CI today,
reporting green while their only functional verification never fires.
The new lint mirrors test_has_spec and carries a per-role opt-out that is
used nowhere: uniform beats exceptions.
Every one of the 21 now has a template written against its own
meta/services.yml rather than a shared skeleton -- espocrm splits
ADMIN_PASSWORD from ADMIN_NATIVE_PASSWORD because OIDC_ALLOW_ADMIN_USER
leaves both login paths open, the four permanently auth-less roles declare
their personas blocked instead of shipping a knowingly-red spec, and
web-app-mastodon gained a real test-seaweedfs.js because no honest
suppression existed for a role that does route Paperclip to S3.
web-app-magento moves pre-alpha -> alpha here: it now has a spec that runs,
which is what the lifecycle gate is about.
Six meta/services.yml files change only where adding a template falsified an
existing suppression -- gitlab and shopware both claimed the role 'ships no
templates/playwright.env.j2', which stopped being true in this commit.
Not resolved here, and the adversarial review says so plainly: the specs are
enabled, not proven. None has executed against a live stack. The review found
three classes of defect that land in follow-up commits rather than being
smuggled in silently -- persona blocks that are missing or should be tied to
sso.enabled, eleven suppressions naming a sys-svc-mail test suite that does
not exist, and eight claiming a CSP assertion that cannot fire because no
consumer role renders CSS_BASE_URL.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Defines what a services.mcp block may declare and enforces it: classification,
transport, auth, implementation and the provenance fields that make an upstream
claim checkable. The lint refuses an unknown key anywhere in the block and
refuses a provider whose allowed consumers cannot speak its transport or its
auth scheme.
Discovery resolves each provider's credential and endpoint per consumer and
per deployment, rejecting a server with a named reason. Transport, auth and
endpoint faults are fatal; a missing credential is not, because it also occurs
when the provider is simply not provisioned yet in the same play.
The filters render the Authorization header from the declared scheme, build
Open WebUI's tool-server connections with their group grants, and derive the
members of those groups from the RBAC assignments. Role scoping gains
application-scoped roles, so an mcp role only counts when it is granted on a
specific application.
roles_with_service gains a scope and a direction filter, and drops the canonical
domain requirement for direction-filtered entries: an MCP URL is built from the
service key and the internal port, so a provider without a public vhost stays
valid.
One audited implementation, instantiated once per provider application rather
than deployed as a single process holding every application's credential. Each
instance joins only its provider's network, carries only that provider's
least-privileged credential and serves a distinct endpoint.
Every decision lives in the policy layer, which validates the contract at
startup rather than at first call, so an instance missing a limit, an
allowlist or the schema hash never reaches a client. The server itself only
moves bytes over three JSON-RPC methods, with no stream state to get wrong.
Its upstream header is configurable because only half the providers accept an
RFC 6750 bearer.
The contract probe runs at deploy time rather than in Playwright, because it
needs the provider's real credential and a Playwright trace would capture it.
It proves, over both transports, that an unauthenticated caller is refused,
that a wrongly authenticated one is refused, that the handshake completes,
that the served tool list is exactly the contracted one, and that one
deterministic read answers without an error. Steps one and two are what
separate a guarded endpoint from one that merely happens to be unreachable.
The isolation probe counts the sidecar's network interfaces from inside it.
Reachability cannot be used here: an unknown name does not fail to resolve in
this environment, it falls through to a wildcard resolver that answers every
name with one reachable address.
The role is inert until a provider includes it; those inclusions follow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both probes are driven with retries and an until condition that reads the
marker in stderr, so writing REJECTED ends the loop on the first attempt no
matter how many retries remain. Both wrote it for every failure, which turned
a provider that had not finished starting into a final judgement.
The same defect cost a full run elsewhere in this family: a 503 from a
starting service ended its task at attempts 1 of 30 and failed the deploy
after an hour.
Transient statuses and connection errors now report PENDING so the caller
keeps waiting, while 4xx and contract violations stay REJECTED. A stream that
carries no endpoint event is a contract violation, not a slow start, and is
kept terminal deliberately. Both probes also catch OSError now, which used to
surface as a traceback with no marker at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The local exec helpers covered the development container and a throwaway
sidecar next to it, but nothing reached into an application the stack had
already deployed. Swarm had swarm-app-exec for that; compose had no
counterpart, so the only remaining route was a nested docker exec, which the
contributor rules forbid.
The helper goes through container.sh's positional form rather than its cmd
environment variable. container.sh runs that variable through `sh -lc`, so an
already-composed command would be split a second time; passing argv through
untouched leaves exactly one shell level, inside the target container.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The pretix image derives its worker count as `2 * $(nproc)`, which counts the
host's cores rather than the container's share. On a twenty-core host that is
forty gunicorn workers inside a 2g memory limit and a 1024 pid limit, and the
container is OOM-killed before it serves a request: docker reports exit 137
with OOMKilled true, and the log shows workers taken by SIGKILL.
NUM_WORKERS now applies upstream's own formula to the CPUs the service
declares, with a floor of two so a fractional allocation cannot round to none.
PRETIX_WEB_CONCURRENCY and PRETIX_WORKER_THREADS were already set but the
launcher reads neither for the worker count; they are pretix application
settings, so the role was configuring something that never influenced the
process count.
After the change pretix comes up healthy and stays up across three subsequent
full runs, and its MCP contract probe passes in two of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Mailu meters each sender login at an undeclared 200 messages per day. The
platform's no-reply identity carries both the deploy-time msmtp health check
and every service notification, so a day of deploys exhausts the quota and
from then on every further deploy fails on that health check rather than on
anything being wrong. Counted across eight runs: six failures each in the four
runs before the exemption, none in the four after. The limit was still active
an hour after the last attempt and the unit's timer only fires daily, so it
was not going to clear on its own.
The rejection comes from Mailu's sender-rate endpoint, not from its auth rate
limit. Postfix wraps the reply as "SASL login name rejected", which reads like
an authentication failure and is not one. MESSAGE_RATELIMIT_EXEMPTION is
checked before the counter is touched, so an exempt sender consumes nothing,
and it is Mailu's own mechanism for exactly this case. Raising the limit
instead would move the boundary for real user mail as well.
The value is lowercased because Mailu lowercases the configured set but
compares it against the raw sender from postfix, so a deployment whose domain
carries uppercase would silently miss the exemption.
The health check also misread the rejection. Every 4yz reply is transient per
RFC 5321, but the retry branch only matched 454 and the temporary
authentication wording, so a 450 aborted the deploy on its first attempt. It
now retries on any 4yz, which costs a longer wait when a 4xx condition is
persistent and keeps 5xx immediately terminal.
Its pacing is widened from twelve attempts twelve seconds apart to ten thirty
seconds apart. That was originally reasoned against AUTH_RATELIMIT and is not
what fixed the failure; it stands because a health check has no reason to
authenticate five times a minute.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role could never serve. 01_setup.yml copied nginx.conf.sample inside the
bootstrap shell, but magento/project-community-edition 2.4.8's dist archive is
2583 bytes and holds exactly one file, composer.json - nginx.conf.sample ships
in magento2-base and only lands once composer install runs. Executed, the
bootstrap's own shell says: cp: cannot stat 'nginx.conf.sample'.
set -e made that fatal, and the block's creates: guard made it permanent: it
points at html/composer.json, which create-project had already written, so
every rerun skipped the block, nginx.conf never appeared, the glob include
matched nothing and / returned 403 forever.
This is a regression from the create-project --no-install split: the cp stayed
behind in a step that no longer downloads the package that carries the file.
It now runs after the install with its own creates: on html/nginx.conf, which
also heals volumes the previous code already wedged.
The rest of the topology was proven sound in the same investigation - real
images, real nginx.conf.sample, HTTP/1.1 200 OK through php:9000, and a clean
nginx -t on an empty volume - so nothing else about the vhost changes.
Also here, all found while proving the above:
- placement: manager. 01_setup.yml bootstraps through container exec gated on
IS_STACK_HOST, but the role was not manager-pinned, so in swarm the manager
bootstraps while php and nginx run on a worker roughly two times in three:
volume inspect finds no volume and resolve-container-id returns an ID that
lives on another node. 30 roles already pin this; magento was the outlier.
- --base-url without the trailing slash. lookup('tls', …, 'url.base') already
ends in one (tls.py:89), so every stored link carried // and the session
cookie path became //.
- composer install --no-dev. The run was locking 622 packages including
codeception, allure and phpmd.
- Both nginx restarts now gate on the vhost task actually having changed;
changed_when: rc == 0 made them fire on every single deploy.
- fastcgi buffers on the server block. Magento's sample sets them only on the
main entry point, so /setup/ and /update/ fell back to nginx's 4k default.
- Nine of the ten variants-static-override nochecks were inert - the base
values they claim to override are Jinja, not literals - and their pasted
justification named backup shared on services that have nothing to do with
backup. Only container_backup.shared is load-bearing.
make test green across all four targets. test-lint caught one defect while
this was being written: the new task first used ansible.builtin.command, which
passes the container_address $(...) through literally instead of evaluating
it - the same swarm failure mode the placement fix addresses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A whitelist-scoped update preloaded and fully executed every provider role
the host is a member of, because the gate only asked whether the role was in
group_names. On this inventory that meant a single-app update built compose
contexts, rendered configs and vhosts for 17 unrelated applications.
reachable_roles() is lifted out of applications_if_group_and_all_deps() so the
same traversal can start from an arbitrary seed set instead of group_names.
The old function now calls it and filters as before, so every existing caller
keeps its result. A new application_closure lookup exposes it to Ansible, and
sys-service-loader gates each preload on the whitelist's closure. An empty
whitelist yields an empty closure, which the gate reads as the full inventory
scope, so a no-apps deploy is unchanged.
Measured on mode=update apps=web-app-snipe-it: 42:49 before, 17:18 after, with
ok tasks dropping from 4903 to 2030. Nothing outside the closure ran, and
snipe-it's real dependencies (mariadb, redis, openldap, postgres, keycloak,
mailu, prometheus, seaweedfs, cdn, logout) all still did.
The closure also gates the required branch, deliberately. `required` reads as
a global property but plugins/lookup/service.py derives it from group_names,
so it means "some provisioned app needs this" and admits nearly every service.
Exempting it was measured at 42:39 against 17:18 for the same deploy, which is
the entire saving. A service whose provider lies outside the closure is by
definition required by an application outside the deploy scope.
Out of scope: seeding applications_current_play from the closure. That shrinks
CURRENT_PLAY_DOMAINS_ALL, which cert SANs, openresty vhosts, DNS records and
the webserver health check all consume at full inventory scope.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
make quality failed on three ruff errors, all in the CI deploy-sequencing
work: RUF007 twice in test_ci_deploy_sequencing.py and TID252 in
tests/utils/ci/job_names.py. autoformat is chained ahead of test, so the whole
test stage never ran while they stood.
zip(CHAIN, CHAIN[1:], strict=False) becomes pairwise(CHAIN). Same two pairs
over the three-element chain, but strict=False was switching off a length
check that this call genuinely needs -- CHAIN[1:] is shorter by construction,
so strict=True would raise. pairwise has no second sequence and cannot have
the problem, so the suppression goes away with its reason. Both assertIn calls
fit on one line again as a result; that reflow is ruff format's, not a
hand-made edit.
job_names.py was the only file reaching PROJECT_ROOT through a relative parent
import; test_ci_deploy_sequencing.py:20 already spells it absolutely.
Verified rather than assumed: make quality now green end to end (ruff in the
ran list, all four targets green, 4030 + 299 passed). The sequencing file on
its own reports the same 24 subtests as before the change, so pairwise yields
the identical pair set and not merely a passing one. All four importers of
job_names run green (60 passed), which is what rules out the absolute import
resolving differently under another sys.path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
web-app-matrix 0,1,2 failed with no assertion at all - JUnit reports
errors=1, failures=0. Chromium crashed: dmesg holds exactly two traps in the
whole 04:34-05:55 window, second-exact to both attempts and at the identical
binary offset, so one fatal CHECK() twice. int3 is Chromium aborting itself,
not a kill from outside.
The same job proves the budget is what decided the outcome:
tests/test-seaweedfs.js hit the identical error class twice, ~5s in each
time, and passed on attempt 3 because it inherits retries: 2 from
playwright.config.js. The DM test opted down to 1 and died.
Ruled out with evidence rather than by elimination: not the application
(zero 502s in the whole openresty log, Synapse 200 throughout), not the auth
chain (both OIDC logins succeeded on attempt 1), not the test (no assertion
reached), not the timeout (effective budget 1200s, attempts took 170s and
164s), not OOM (dmesg-oom.txt empty, 9G available).
The comment goes with it. It attributed the crashes to processes being
'OOM-killed', which is exactly what the artifacts refute, and it would have
sent the next investigator after memory pressure that was never there.
Worst case is 8.5 min instead of 5.7 against a 59-minute suite budget.
Not addressed here: which CHECK fires at that offset is unknowable from
these artifacts because Playwright discards the browser's stderr.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
web-app-openwebui 0,1,2 was killed by the 350-minute action timeout with no
failed task anywhere in its 1.33M log lines. The sweep admitted fedora at
remaining=10372s after debian took 10028s, then never looked again: the
budget was only checked between distros. Its internal deadline passed 594
seconds before GitHub killed the job - the run knew it was over time for ten
minutes and had no way to act. Worst-case runtime was deadline plus the full
duration of the last admitted distro, unbounded.
The trigger was a single stalled task (803s for a no-op that takes 1.3s
warm, and a fully cold transaction 8 minutes later cost 24s), but that only
decided which job hit the wall. The wall itself is structural.
remaining is already computed at the top of the loop, so wrapping the call
in timeout adds no new state. rc 124 becomes 'skipped' rather than 'failed',
because a job that stops at its budget is the documented intent, not a
defect - test_distro_budget_fits_timeout.py already asserts this invariant,
which until now was simply untrue.
Two conditions narrow it, and both matter:
dur >= remaining - 5 keeps an inner timeout that fires early as a failure.
Verified: timeout returns 124 only for its own kill and passes inner exit
codes through unchanged (0 for success, 3 for an inner exit 3).
passed > 0 is the one that protects the test result. Without it, a job whose
very first distro overruns would report 'skipped' and exit 0 - green while
having verified nothing. That would have been a silent false pass across all
243 jobs of a run. The guard was executed over the full truth table,
including passed=0.
-k 60 is deliberate: one.sh execs into dedicated.sh and its ansible and
docker children can outlive SIGTERM.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
web-app-penpot 2 failed with 'Port 2049 never opened'. ganesha had not
crashed: the unit was active, 5 threads, 18ms CPU, and the journal stopped
after the export CLIENT warnings without ever reaching NFS SERVER
INITIALIZED. All five threads sat on one futex word (FUTEX_WAIT_PRIVATE,
val 2) with no live thread holding it - an orphaned mutex in liburcu-bp,
entered from ReadExports -> fsal_cfg_commit -> urcu_bp_register. Bind_sockets()
is strictly downstream of ReadExports(), so nothing was ever bound.
This is not the Fedora 43 / ganesha 7.2 SEGV: that one reached listen state
and died on accept, 100% reproducible. This one never listens, alive and
idle, and hits roughly 1-3% of fedora legs - lam 2 started the same build
twice in the same run.
The recovery that already existed could not fire. Type=forking plus an
ExecStart parent that exited 0 means systemd considers the unit started, so
Restart=on-failure never saw a failure. ExecStartPost runs before the start
job completes, so a failing probe there is the event it has been waiting for.
Verified rather than assumed, with a throwaway transient unit carrying the
same properties: a Type=forking unit whose ExecStartPost fails ends in
'failed' with the same message this class produces in CI, Restart=on-failure
then schedules restarts 1, 2, 3, and StartLimitBurst stops it with 'Start
request repeated too quickly'. StartLimitIntervalSec was 0, i.e. unbounded -
without the burst cap this fix would trade a rare wedge for a silent loop.
The 20s budget is measured, not guessed: a healthy ganesha binds 2049 in
0.32-0.38s across eight successful gates in lam 2, so the probe sits ~57x
above normal. /usr/bin/timeout and /bin/bash were checked on all four node
images before hardcoding them.
TimeoutStopSec=10 is required, not cosmetic: the wedged process blocks
SIGTERM in main() and its sigwait consumer is created downstream of the
wedge, so every recovery cycle would otherwise burn ~45s to SIGABRT.
The Ansible gate goes 60s -> 120s because one recovery cycle is ~32s and 60
expires before the first one finishes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
