Two callers polled the same stack_ready.sh with different budgets: the
handler read swarm_stack_converge_retries while post_deploy hardcoded
150. Eleven roles had set the override and were unprotected at the
second caller anyway. Both now read one SPOT in group_vars.
The gate also had no way to give up. A crash-looping espocrm burned 600
polls (20 min) before failing, while flowise was killed 45 s short of a
409 s image pull. has_fatal_task now exits 2 - and both callers stop at
once via 'until: rc in [0, 2]' - when a slot's newest attempt is
Rejected or Pending, or when a slot burned three attempts after the
service churned past swarm_stack_fatal_grace. Patience only grows where
nothing is broken.
report_tasks loses its 'awk !/error=$/' filter, which printed only rows
that HAVE an error and so stayed silent in exactly the no-error case
that occurred.
Measured on a live single-node swarm: a service failing 3x and
converging on the 4th returned rc 2 at t+15s before this change and rc 0
at t+32s after; a 3-replica stop-first rolling update gave 68/68 samples
rc 0; a permanent crash loop still reaches rc 2. The deploy epoch comes
from 'docker service inspect .UpdatedAt.Unix' rather than a /tmp marker,
which was reproduced contaminating a later deploy.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The hourly sweep fought the rest of the CI control plane instead of
helping it: it preempted itself (17 of the last 29 runs concluded
cancelled 1-2s after their successor was created, for 8s of work behind
a 74-80min queue wait) and shipped MAX_ATTEMPTS=0, which the script
reads as no ceiling while its own README promised the opposite. The 24h
age cutoff could not bound it either, because every rerun refreshes the
run's updatedAt and so keeps it inside the window.
Removes the workflow, the two scripts it was the sole consumer of, its
unit test, and the CI_RERUN_* documentation that would otherwise point
at a deleted file. The workflow is disabled in both origin and fork so
no sweep fires before this lands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A retrigger carried the workspace and instructions toggles over from the
run title, so a suite that had already gone green was dispatched again
and spent runners on work with a known answer.
Derive each toggle from the source run's job record instead: a suite is
turned off only when every one of its jobs reports success. A suite that
was skipped, cancelled or never dispatched has not passed, so it is still
asked for; absence of a failure is not evidence of a success. One failing
shard keeps the whole suite on.
Match against the job path rather than the run title by dropping the
workflow-name segment, and fail open when nothing matches, so an unknown
job layout re-runs the suite rather than silently skipping it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The provisioner probed a stored token first, so a token that outlived a
password change kept the run green while the deployed password no longer
authenticated. Authenticate with the inventory password first and consult
the stored token only once that succeeds; when it fails, converge the
password onto the existing user.
The write targets /Users/Password?userId=<id>: omitting the query
parameter retargets the caller's own account, and an administrator acting
on another user is why no current password is required. Accept 200 as
well as 204, matching the sibling calls; demanding 204 alone would have
failed a successful convergence.
The repair branch is only reached when authentication failed and always
writes, so it now reports CHANGED. It previously printed UNCHANGED
whenever the user already existed, which told Ansible nothing had changed
while a credential had just been rotated.
configure-auth.sh no longer continues past an administrator it cannot
authenticate. Both call sites logged a warning and carried on, finishing
green with SSO and LDAP silently unconfigured.
Adds the role's first unit tests, covering token reuse, token staleness,
the convergence write and its rejection path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A paused update never completes on its own: the updater returns early on that
state, before the branch that would finish it, and refuses to start again while
an update status is set. The restart policy brings the replicas back but does
not touch the status, so the service can sit fully replicated while the gate
still refuses it.
The probe polled that for six hundred attempts and reported a timeout after
twenty minutes. It now separates a state worth another poll from one that is
final, and the wait distinguishes the two exit codes instead of retrying every
refusal.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Driven by an until loop of 600 polls, the probe exited 1 without a word on
every refusal, so a service that never converged produced a twenty minute wait
and then a fatal carrying an empty stdout and an empty stderr. The reason had
to be reconstructed from a rescue artifact, and the update-status branch is not
recorded there at all.
Each refusal now names itself on stderr. Only the branch with task rows to show
prints them, filtered to the rows carrying an error and capped, which is the
shape the neighbouring stack_ready.sh already follows; the update-in-progress
branch stays a single line because it is the normal state of a healthy rollout
and fires on every poll.
A timed out service ps used to abort the script with its own code and no
message, reproducing the very symptom this addresses. It is now named too.
Adds the unit tests the script never had, mirroring the sibling's, including
the assertion that error-free task rows stay out of the output.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The msmtp_curl probe joined its mail branch to the curl probe with '&&', so a
relay that refuses the startup message short-circuited the whole command: curl
never ran, the container stayed unhealthy forever, and the swarm converge gate
burned all 150 retries before aborting the deploy. Friendica 0 and Fediwall 0
of run 31340965955 died that way, reporting only 'not converged:
friendica_friendica' with the real cause buried in a rescue artifact.
That contradicted the probe's own docstring, which promises an unrelated SMTP
outage cannot flap the container into unhealthy -- it only delivered that for
email_enabled=False. sys-ctl-hlth-msmtp/files/script.sh:23 has always tolerated
exactly these exit codes; the container probe was the outlier.
Liveness is now curl's verdict alone. The '&&' before touch stays, so a refused
mail leaves no marker and the next probe retries it rather than turning the
smoke test one-shot.
Detection does not disappear with the coupling: web-app-friendica now asserts
the delivery marker after convergence and fails with the relay's reason instead
of a timeout three layers away. sys-ctl-hlth-msmtp is a systemd timer and the
Mailu Playwright roundtrip submits through Roundcube, so neither covers a
container relaying over port 25 -- this assertion is what does.
The marker path moves into utils.docker.healthcheck.MAIL_MARKER and reaches
Ansible through lookup('container_mail_marker'), so the probe and the assertion
cannot drift apart.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A lookup plugin receives group_vars through variables.get() untemplated, and
group_vars/all/18_swarm.yml:2 defines DEPLOYMENT_MODE as a Jinja expression.
The comparison against 'swarm' could therefore never match, so every swarm
deploy fell through to _COMPOSE_HOST and baked
socks5h://host.docker.internal:9050 into Nextcloud's proxy system value.
svc-net-tor publishes SOCKS with mode: host, so that listener exists on the
manager alone. With three Nextcloud replicas against one Tor node the failure
is node-determined, not intermittent: the 104 'cURL error 7 ... over proxy
host.docker.internal' lines in run 31340965955 split 56 on wrk-02, 48 on
wrk-01 and 0 on mgr-01, which is what made the OIDC callback 500 on 2 of 3
replicas and read as a 39% flake.
Templating first yields socks5h://tor:9050 in swarm, which every replica
reaches over the overlay it already joins through the onion_sso consumer rule.
Compose is unchanged: a single-node inventory renders the expression to
'compose', and a suppressed template error falls through to the same branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
scripts/tests/environment/ becomes scripts/tests/workspace/ with one
subdirectory per track. base/ prepares the workspace and is a prerequisite;
compose/ and swarm/ are independent of each other, so CI can run them as
separate jobs with a budget each. 00_orchestrator.sh runs the tracks named
by INFINITO_WORKSPACE_TRACKS and aborts on an unknown one.
The job count stays at five: cli.meta.ci.dev_runtime matrix assigns exactly
one track per distro and flips the split with the run number, so two
consecutive runs cover both tracks on every distro without doubling the
matrix. The workspace_track input (rotate|compose|swarm) pins it from manual
steer through the orchestrator for a targeted re-run.
Job names carry the track glyph from utils/symbol_glossary and the image
shortened to its last two segments, so quay.io/centos/centos:latest reads
as centos:latest.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
utils/tls_common.py:275-283 returns enabled=False for any onion primary domain,
so lookup('tls', <app>, 'protocols.web') is "http" there and "https" everywhere
else. Every OIDC consumer inherits that through group_vars/all/11_oidc.yml, which
is where PeerTube's OAUTH_HTTP_REQUEST_FORBIDDEN comes from — 52 roles hang off
this one derivation and nothing covered it.
test_app_override_enabled_false already asserts protocols.web == "http", but it
gets there through an explicit server.tls.enabled: false, not through the onion
rule. These two cases exercise the rule itself, in both directions.
The mid-code comment above the sys.modules shim goes because the comment lint
flags it once the file is touched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
keycloak-js dropped the UMD build in 26.0.0. The package now ships only
lib/keycloak.js with "type": "module" and a default export, so the CDN
path dist/keycloak.js resolves to a file that no longer exists, and a
classic script tag could not parse the file that does. The adapter never
loaded, window.__oidcLoginReady was never set, and every app job that
co-deploys the dashboard failed its Playwright run on the resulting
console error.
The loader now uses a dynamic import and takes the constructor from the
module's default export. That also retires the window.Keycloak check,
which had nothing left to check once the global stopped existing.
Module fetches always run in CORS mode, unlike classic script loads. The
internal CDN emitted no Access-Control-Allow-Origin at all, so it could
not have served an ES module to the dashboard's origin. jsdelivr sends
"*", which is why the external flavor would have hidden the gap.
Every init option the template passes carries the same default in 24.0.5
and 26.2.4, and the silent-SSO postMessage contract is unchanged, so the
adapter's behaviour is untouched beyond the packaging.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A branch with no tracking remote fell straight through to origin, so on a fork checkout every run lookup asked the upstream repository and answered HTTP 404. i8cifail could not reach a single run of this branch.
Consult remote.pushDefault between the tracking remote and origin. That is git's own answer to where the branch is pushed, hence where its runs live, and the i8puma alias already reads exactly this key — the two resolution paths now agree instead of contradicting each other.
The fallback chain had no test at all: every existing case mocks resolve_repo away. Four cases now pin it, including a pushDefault naming a remote that no longer exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 123794c7f4)
The hourly rerun bot looked only at the newest run's conclusion. When that run was cancelled while an older one was still pending, reviving it added a second waiter to the ref's global-ci concurrency group — and cancel-in-progress evicts only in_progress runs, so neither could start and every later dispatch queued behind them.
Query the branch once with status included and leave it alone while anything on it is still active. The selection itself is unchanged; it now reads the same response instead of issuing a second one.
Scope, measured rather than assumed: this does NOT explain today's deadlock. The blocking revival happened at 14:51 on an idle branch, which the guard permits, and the later revival at 15:51 never produced a second attempt (run 31405424413 is still attempt 1). The guard covers a narrower case than the incident that motivated it.
scripts/github/rerun had no tests at all. The harness added here runs the script against a stubbed gh, following the pattern of tests/unit/scripts/tests/deploy/utils/test_rescue_index.py.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A branch with no tracking remote fell straight through to origin, so on a fork checkout every run lookup asked the upstream repository and answered HTTP 404. i8cifail could not reach a single run of this branch.
Consult remote.pushDefault between the tracking remote and origin. That is git's own answer to where the branch is pushed, hence where its runs live, and the i8puma alias already reads exactly this key — the two resolution paths now agree instead of contradicting each other.
The fallback chain had no test at all: every existing case mocks resolve_repo away. Four cases now pin it, including a pushDefault naming a remote that no longer exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`docker service ps` renders CurrentState as prose — "Preparing 3 minutes ago" —
which cannot distinguish an image still extracting from a task idling on
something else. That gap cost a whole investigation: Nextcloud 2's flowise task
spent ~412s somewhere between pull and running, and the artifact could only
bound the window, never attribute it.
report_task_states adds the machine-readable half:
zqimhnngo3my state=preparing since=2026-08-10T12:53:40Z msg=preparing
Verified against a real swarm, read-only, not just the stub: `--type task` is a
valid type (the daemon answers "task ... not found", not a type rejection), all
three field paths resolve on live tasks, and ten inspects cost 0.07s against the
15s timeout. The type pin matters here because this repo names a role's overlay
network after the same entity as its main container, so an untyped inspect can
resolve the wrong object — tests/lint/ansible/shell/container caught that.
The test file also corrects a stale contract. 3ebd6e950 dropped the
`awk '!/error=$/'` filter precisely so a healthy-but-slow task stays visible, but
test_error_free_task_rows_are_not_echoed still asserted the old behaviour and had
been failing since. It now asserts the row IS echoed, and a second case pins that
an empty inspect result does not print a bare indented line.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The loop condition is the usage of the whole filesystem, which the script cannot move once every remaining version is the last one of its application: is_directory_deletable always skips those. It then repeats an identical, empty pass. The journal of a real unit shows 132192 iterations in two minutes, and systemd kills the unit on its start timeout long before the script's own hour-long limit, so the failure surfaces as 'Active: failed (Result: timeout)'. The sys-service handler turns that into a failed play, nine minutes before the first application role runs.
b694f9335 already fixed this busy-loop for the shape where no backup versions exist at all, by breaking when average_version_directories reaches zero. With one version per application the average is one, so that guard never fires. Key the exit on the outcome instead: a pass that deleted nothing ends the loop. While anything is deletable the behaviour is unchanged, and the last version of an application stays untouched as before.
The regression test needs a running clock, because the shared helper freezes time.time() at zero and a regression would therefore hang rather than fail, plus a Path.is_dir mock that delegates outside the backup tree so runpy keeps working.
Touching the file pulled it into the comment lint, which flagged five pre-existing comments. Four were narration and are deleted; the fifth justified the noqa S602 marker and moves onto that marker's line, where the security lint expects it, so the file leaves that lint's baseline.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
4693a8195 put `# nocheck: cache-read` on a line that exceeded the length
limit, so ruff format wrapped the call and left the marker sitting on the
closing paren. The lint matches per call line, so the exemption stopped
applying and test_no_uncached_file_reads failed again -- the commit undid
its own fix the moment the formatter ran.
Bind the read to a local first: short enough that no wrap is possible, and
the marker stays on the call it exempts. Verified formatter-stable with
`ruff format --diff` reporting no change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both are fallout from earlier commits in this branch.
4b10d6b93 moved redirect_uris.py from the keycloak role's filter_plugins to
plugins/lookup, but the requirements doc still linked the old path, which
test_relative_links flagged.
test_primitives.py reads back a file that download_file just wrote into the
test's own TemporaryDirectory. test_no_uncached_file_reads asks for
utils.cache.files.read_text, but its own message names this case as the
opt-out -- "tempdirs, synthetic fixtures written inside the same test" -- and
read_text is wrong here anyway: the payload is bytes, and a session-shared
cache would defeat the point of asserting what was just written. Same marker
as tests/integration/meta/workflows/test_subset_roles.py:40.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
08_users/realm.yml seeded the realm from `lookup('users') | non_reserved_users`
while 08_rbac_groups_realm.yml built group memberships from the raw
`lookup('users')`. The membership builder has its own guards, but both are dead
against real data: across all 269 default users, none carries a `reserved` key
and none has an empty password. So membership was effectively unfiltered while
seeding filtered on `accounts`.
helpdesk and no-reply (accounts: [mailbox], roles: [bot]) were therefore
demanded as members of roles/web-app-mailu/bot but never created, and
ensure_group_path_members.sh exits 1 on the first miss. Four jobs -- Mattermost 1,
Friendica 1, Matrix 1, OpenProject 1 -- each burned five cross-node recovery
attempts on a data failure no reschedule could fix.
Rather than adding the filter at the site that forgot it, remove the ability to
forget: lookup('users_non_reserved') resolves users and applies the predicate in
one place, and all four consumers now read it -- the two Keycloak user tasks, the
RBAC group builder, and OPENLDAP_USERS, which had it right all along. The filter
stays as the pure predicate the lookup calls.
Reproduced offline against the real user model, byte-identical to the CI member
list: before, bot demanded helpdesk and no-reply; after, members are exactly the
four seeded accounts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PYI034. The earlier attempt did not land -- the string replace missed
because the formatter had already rewritten the line, leaving an unused
Self import that ruff then removed while the finding stayed open.
The import belongs under TYPE_CHECKING: the module has
`from __future__ import annotations`, so the annotation is a string and is
never evaluated at runtime.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A single anonymous GET for the pinned hadolint binary completed its TLS
handshake and its request, then got a clean FIN with no status line after
3.6s. download_file was a bare urlopen with no retry, and make install-lint
is a prerequisite of autoformat, lint and every lint-* target -- so one shed
socket took down the whole quality gate. The same asset downloaded fine five
times elsewhere in that run, 15s after the failure on another distro.
Retry four times with exponential backoff, catching (OSError, HTTPException).
Two names, not the obvious four: ssl.SSLError is an OSError but NOT a
URLError, so the narrow tuple would have missed it, while OSError subsumes
URLError, ConnectionError, TimeoutError and gaierror in one token.
HTTPException is needed separately because IncompleteRead is not an OSError.
HTTPError is caught FIRST and re-raised immediately for any status outside
{408,429,500,502,503,504}. That ordering is load-bearing -- HTTPError is
itself an OSError, so a broader catch first would swallow it -- and it is
what keeps this robustness rather than masking: a 404 is exactly what a
stale version pin looks like after roles/sys-lint is bumped, and it still
fails on the first attempt with the URL. The final attempt re-raises the
original with its traceback, and write_bytes stays in the else branch so a
full disk is never retried.
apt was the other uncached, unretried input in the same target (its default
Acquire::Retries is 0), so give both invocations -o Acquire::Retries=3,
matching what eight Dockerfiles in this repo already do.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
KEYCLOAK_REDIRECT_URIS derived the scheme from Keycloak's OWN TLS state and
stamped that single scalar onto every SSO consumer. With svc-net-tor
deployed Keycloak is onion-primary, so resolve_enabled returns False and
lookup('tls', ...) yields "http" -- which then went onto clearnet apps too.
Keycloak registered http://matrix.infinito.example/* while Synapse sends
https://.../_synapse/client/oidc/callback, and its redirect-URI match is
scheme-exact: 90 invalid_redirect_uri events, an error page instead of a
login form, and a 60s selector timeout that looked like a Playwright bug.
jitsi hit the same wall on https://meet.infinito.example/oauth2/callback.
Resolve per consumer instead, via resolve_enabled: an .onion is always
plaintext, a clearnet app follows its own server.tls.enabled, and only
then the global TLS_ENABLED. That override existed and was silently
discarded -- passing a pre-computed scheme string threw it away, which is
the same "what I am" instead of "what the consumer is" mistake one layer
up. resolve_enabled handles the onion case itself, so the filter's own
is_onion_domain branch is gone rather than duplicated.
Move it from a filter to a lookup while at it: the thing needs domains,
applications and TLS_ENABLED anyway, so resolving them inside collapses
the call site to lookup('redirect_uris') and drops
KEYCLOAK_REDIRECT_FEATURES, which listed the same path twice. A lookup
under plugins/lookup/ is also plainly importable, so the test loses the
sys.modules stubbing apparatus a role-local filter needed -- 60 lines net
smaller.
KEYCLOAK_WEB_ORIGINS is regex-derived from these URIs and inherits the
fix. Verified against the real applications view: 69 URIs, all https with
TLS_ENABLED true and all http with it false -- the same 69 the failing run
registered as plaintext.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two violations that tests/lint cannot see, because ruff runs under the
autoformat target rather than the lint suite:
TC003 test_db_tor_transport_gate.py imported pathlib.Path at runtime
for an annotation only. The module has `from __future__ import
annotations` and never constructs a Path -- the values come out
of ROLES_DIR.glob() -- so the import moves into TYPE_CHECKING.
RUF012 test_render.py's ENTRY is a mutable class attribute; annotate it
ClassVar[dict].
Plus three reflows from ruff format. No runtime behaviour changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
libcurl refuses a .onion host by name (RFC 7686, error 6) before it
consults /etc/hosts, extra_hosts or DNS, so the platform's DNS+iptables
onion egress cannot serve a server-side backchannel at all. Measured:
curl --noproxy '*' -> exit 6
curl --resolve auth.<onion>:80:127.0.0.1 … -> exit 6
--resolve fills curl's own cache strictly earlier than getaddrinfo and
is still refused, which makes sys-svc-container's onion_oidc_socks
template -- extra_hosts, despite its name -- a no-op for every libcurl
client. Only a configured proxy makes curl skip the check.
Since 9d91fbea8 emptied proxy.config.php.j2 there is no such proxy, while
svc-net-tor's exclusive/primary defaults replace Keycloak's clearnet
domain and KC_HOSTNAME pins the issuer to the onion. Nextcloud's
oidc_login discovery call therefore dies inside PHP, /apps/oidc_login/oidc
returns 500, and oidc_login_auto_redirect sends /login straight there.
Restoring the old socks5h://host.docker.internal:9050 alone would make it
worse: tor runs as a single manager-pinned replica publishing SOCKS with
mode: host, so only the replica on that node could connect and a
deterministic failure would become a 2-in-3 flake. In swarm, though, tor
is NOT host-networked -- that branch is compose-only -- and it already
binds SocksPort 0.0.0.0, so the shared overlay is a usable path from
every node.
Give svc-net-tor a shared_net overlay with alias 'tor' and dial it by
service name. Attachment is gated by a new consumer kind, onion_sso: the
consumer's services.sso.enabled AND the provider's services.tor.enabled.
Both are group-membership derived, so the conjunction is exactly "the
issuer is an onion" and stays false in clearnet deployments -- which
matters because the service registry is not filtered to deployed roles.
The existing kinds could not express it: keying on sso alone would also
hold without tor, and keying on tor would miss Nextcloud's variant 2,
which disables its own onion yet still needs the client path.
tor_socks_proxy keeps the swarm/compose split in one place so the six
roles sharing this defect need not restate it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The three svc-db-* credential tests each hand-rolled the same transport
gate in their test.env.j2:
(lookup('config', application_id, 'services.tor.enabled') | bool
and lookup('config', application_id, 'services.<entity>.exposed') | bool)
Three copies of one predicate, each naming its own entity key, and each
free to drift from the onion mapping that actually decides reachability.
Extract it into plugins/lookup/tor_reachable.py, split the same way as
its sibling tor_ports.py: a plain is_reachable() for callers holding the
merged applications view, plus a thin LookupModule resolving it through
lookup('applications'). Because collect_exposed_ports() scans
services.values(), the entity name disappears from the templates
entirely -- they collapse to
TOR_ENABLED={{ lookup('tor_reachable', application_id) | lower }}
Rework the integration guard accordingly: it now asserts that every
svc-db-* template delegates to the shared lookup, and drives its
base/variant agreement table through the real is_reachable() instead of
re-implementing the predicate. Add unit coverage for the new lookup.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The provisioner fell back to the declared connections only while the
instance had none. Once it had one, the declared set was ignored
entirely, so a newly declared provider was dropped without a word and
the grant set never grew. Declared and existing entries are merged now,
and url, path, key and auth_type are refreshed on an entry already
present, so a provider that moves or rotates its secret is updated
rather than frozen.
The final check compared counts, which a swap would have passed. It
compares identities and names whatever the write lost.
The reconciler hard-coded "sse" while meta/mcp.yml declared the same
fact, so the declaration decided which providers may admit the role and
the constant decided which ones it accepts. They disagreed, and only a
runtime error settled it. The transport now arrives as an environment
value and its absence aborts, because without it every provider reads as
incompatible and the registry converges to empty without complaining.
Registration now carries its response into login, so an instance that
refused to create the account says so instead of failing later as a bare
"User Not Found". The mcp group joins the SSO gate's allowed groups.
An mcp grant lets its holder drive clients that reach admitted providers
with the credentials the deployment issued, not the caller's own, so the
role is declared per application and never spans them.
The unit test pins that boundary directly: a scoped grant reaches only
the named application, an unscoped one reaches nothing, and an ordinary
role still spans everything. The Keycloak realm builder, the LDAP entry
builder and the Open WebUI reconciler all resolve membership through the
one module this covers.
An inventory re-initialization mints a new virtual key while the proxy
database keeps the previous one under the same alias. Aliases are unique,
so creating the new key was refused and the constructor aborted after ten
identical retries. The alias is now released before the key is created,
and the release tolerates finding nothing.
The script moves out of the task body into files/ensure_keys.py, with the
port passed as an environment variable instead of templated into the
source, so it is plain Python and covered by unit tests.
Being an MCP client was enough to join a provider's overlay, so a single
admission anywhere reached every provider that opened one. Attachment now
requires both that the role speaks as a client and that the provider
lists it in mcp.allowed_consumers.
consumer.kind accepts a list so one overlay can carry both its
service-flag consumers and its MCP clients, which is what the vector
store needs.
A client rendered an endpoint for every provider the inventory knew,
including providers this run never deploys, so it carried connections
nothing was serving. Discovery now intersects with
application_closure(deployment.whitelist), the same set the service
loader preloads from. An unresolvable closure filters nothing, because
dropping every provider would look exactly like a deployment without
any.
credential_missing stays non-fatal in the app pass, where a provider may
still be waiting to write its secret, and becomes fatal at the
reconciliation stage, where every provider has already run.
Every MCP role keeps its task tree under tasks/utils/ without numeric
prefixes, so a caller can reach any of them through include_role
tasks_from. utils/roles/mapping.py holds that path as the single source.
Ansible resolves a relative include against the including file's own
directory when that file was reached by path, but against tasks/ when it
was reached through tasks_from. The same file therefore loaded from its
own role and failed from a caller using tasks_from. Every nested include
now names {{ role_path }}/tasks/utils/..., which holds under both bases,
and a lint walks each tasks_from entrypoint transitively to keep it so.
The exec-gating lint learns the same form, otherwise it loses the include
chain and reports inherited IS_STACK_HOST gates as missing.
The expression resolving a provider's stored token appeared eighteen
times. plugins/lookup/mcp_credential.py replaces it, and mcp_probe_owner
disappears from eight roles: it was only ever a step towards the bearer
and no probe read it.
The Hermes and OpenClaw probes stop aborting the play while it is still
running. Roles deploy in dependency order, so a client probing during its
own role can run before a provider exists at all, and failing there
prevents that provider from ever deploying. The verdict moves to the
reconciliation pass, gated on MCP_RECONCILE_STRICT, which the reconciler
sets and group_vars declares so the readers need no default.
The sys-svc-mcp-reconcile role is added here; wiring it into a stage is a
separate change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every lint-* target depends on install-lint, which bootstraps the toolchain on
a bare interpreter. Reading the pins through utils.cache.yaml pulled PyYAML
into that path, so the whole lint stage died with 'ModuleNotFoundError: No
module named yaml' before a single linter ran.
pinned.py now reads the file through utils.cache.files.read_text and parses
its two-level shape itself; that import chain is stdlib-only. One test blocks
PyYAML via a meta_path finder in a subprocess to reproduce the bootstrap
condition, a second cross-checks the hand parser against PyYAML so the two
readings cannot drift apart.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 286dd32f30)
Every lint-* target depends on install-lint, which bootstraps the toolchain on
a bare interpreter. Reading the pins through utils.cache.yaml pulled PyYAML
into that path, so the whole lint stage died with 'ModuleNotFoundError: No
module named yaml' before a single linter ran.
pinned.py now reads the file through utils.cache.files.read_text and parses
its two-level shape itself; that import chain is stdlib-only. One test blocks
PyYAML via a meta_path finder in a subprocess to reproduce the bootstrap
condition, a second cross-checks the hand parser against PyYAML so the two
readings cannot drift apart.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
make install-lint resolved each tool's latest tag at install time, so a DNS
hiccup failed the target outright and two runs of the same commit could
install different linter versions.
roles/sys-lint/meta/services.yml now records the repository and ref of
actionlint, hadolint and shfmt. The pairs use the shape
utils/update/repository.py already scans, so update-repository-refs bumps
them by pull request. utils/install/lint/pinned.py reads the pin and applies
the <TOOL>_VERSION override; the three installers lost their duplicated
resolver along with resolve_latest_tag.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The swarm drill provisions a fresh inventory each round: _provision runs
the services disabler, which writes enabled: false, shared: false for
every service in `disable=`. _force_shared_db then ran four seconds
later and flipped shared back to true, because it only ever looked at
shared and never at enabled.
Its key set comes from every flagged top-level key of the svc-db-* roles,
so it covers svc-db-mariadb's own consumer entries tor and
container_backup, both of which sit on the disable list. The result was
enabled=False, shared=True, and the deploy aborted on the consistency
guard in services_disabler.
Guard the flip on enabled instead of narrowing the key set: the key set
would only drop today's two keys, while postgres, redis, memcached,
qdrant, typesense and ldap are all equally on the live disable list.
`is not False` keeps flipping entries that carry no enabled key at all.
derive_includes mirrors force_shared_db by contract, so it gets the same
condition; otherwise the dependency walk sees a different topology than
the deploy.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
GitHub cuts display_title at 512 UTF-8 bytes and closes it with a
literal '...'. The run-name template renders priority second-to-last
and whitelist last, and on the --failed path the whitelist is empty, so
the cut always lands inside the priority list: 21 of 69 roles survived
the source run's title, the 21st a half-token.
untriggered_priority parsed that title and passed undecodable names
through verbatim, so the fragment reached the dispatched priority
input, became the orchestrator's whitelist, and hit the complexity
filter's lexer, whose word class has no rule for CJK. Sequencing died
with argparse exit 2 and took the run with it. The retrigger that is
supposed to repair a red run was breaking its successor.
Read the priority from the ##[group] Inputs block a called job's log
records instead: it carries all 69 roles. Depth-1 jobs only -- a deeper
one echoes its own workflow's inputs, which have no priority.
Drop the decode passthrough now that the source is lossless. The loud
failure is only affordable once a whole value exists; before, it would
have aborted every large retrigger.
dispatched_priority cross-checks against the title: it still proves a
priority line existed even when truncation mangled it, so an empty log
read is a broken reader -- expired logs, a moved block -- and not a run
without priority. Without that check the carry-over would vanish
silently, which is what the rest of this change exists to prevent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Removing the `# --- section ---` banners in 2ab8ec6af left the blank line
that had sat above each one, so three method boundaries in
test_rbac_group_path.py and two in test_sso_proxy_consumers.py carried two
blank lines inside a class body where PEP 8 wants one.
Whitespace only: five deletions, no additions, no code touched.
Verified: ruff format --check reports both files already formatted, and
pytest over the two => 23 passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
make install-lint resolved each tool's latest tag at install time, so a DNS
hiccup failed the target outright and two runs of the same commit could
install different linter versions.
roles/sys-lint/meta/services.yml now records the repository and ref of
actionlint, hadolint and shfmt. The pairs use the shape
utils/update/repository.py already scans, so update-repository-refs bumps
them by pull request. utils/install/lint/pinned.py reads the pin and applies
the <TOOL>_VERSION override; the three installers lost their duplicated
resolver along with resolve_latest_tag.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
74 comments restated the assertion below them, narrated a step the test
name already carries, banner-separated sections, or cross-referenced
another file. All deleted; the only touched code lines are ones that
carried a trailing comment, so the test count is unchanged.
Six of the 74 were invisible to the repo's own comment lint.
find_invalid_comments() exempts any comment block whose next code line is
a def/class (block_def, test_comments_valid.py:385), so section banners
sitting above a test method pass, and real docstrings are never inspected
at all. That let five `# --- section ---` banners through, plus a class
docstring in test_rbac_group_path.py that cited commit 10cb02461 as the
reason the guard exists -- provenance, which belongs in a commit message
and not in the source. A green comment lint is therefore not evidence
that a file complies.
Module docstrings that state what a file covers are kept: describing
behaviour is what the header slot is for.
Verified: checker reports 0 offenders across all twelve; pytest over the
same twelve => 180 passed, 4 subtests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
40 comments in test_database.py and test_sso_oidc_plugin.py restated the
assertion directly below them ("id should be present", "enabled/shared
surfaced") or narrated a step the test name already carries. The repo's
comment lint flagged all of them once the files entered its scope.
The sys.modules eviction rationale above _load_module moves into that
function's docstring, where behaviour documentation belongs; everything
else is deleted outright. Trailing comments go too: Python is linted via
tokenize, so unlike the line-based languages its inline comments are in
scope.
Verified: pytest over both files => 20 passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
59 test files imported `unittest` and then `from unittest import mock`,
giving the same names two spellings so a call site no longer says which
one it follows. CodeQL flags this as py/import-and-import-from; at 59
occurrences it is the single largest item in the repo's code-scanning
backlog.
Keep `import unittest` and reach the submodule as `import unittest.mock
as mock`. Both are ast.Import, which is what the rule asks for, and
`mock.<attr>` call sites stay untouched — the diff is exactly one line
per file and nothing else.
The opposite direction would have been wrong here: PLR0402 rewrites
`import a.b as c` into `from a import b`, i.e. straight back into the
pattern CodeQL flags. It is deliberately absent from the ruff select
list in pyproject.toml, so nothing in the toolchain will undo this.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
GitHub cuts display_title at 512 UTF-8 bytes and closes it with a
literal '...'. The run-name template renders priority second-to-last
and whitelist last, and on the --failed path the whitelist is empty, so
the cut always lands inside the priority list: 21 of 69 roles survived
the source run's title, the 21st a half-token.
untriggered_priority parsed that title and passed undecodable names
through verbatim, so the fragment reached the dispatched priority
input, became the orchestrator's whitelist, and hit the complexity
filter's lexer, whose word class has no rule for CJK. Sequencing died
with argparse exit 2 and took the run with it. The retrigger that is
supposed to repair a red run was breaking its successor.
Read the priority from the ##[group] Inputs block a called job's log
records instead: it carries all 69 roles. Depth-1 jobs only -- a deeper
one echoes its own workflow's inputs, which have no priority.
Drop the decode passthrough now that the source is lossless. The loud
failure is only affordable once a whole value exists; before, it would
have aborted every large retrigger.
dispatched_priority cross-checks against the title: it still proves a
priority line existed even when truncation mangled it, so an empty log
read is a broken reader -- expired logs, a moved block -- and not a run
without priority. Without that check the carry-over would vanish
silently, which is what the rest of this change exists to prevent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both docker_root_free_bytes tests compared the result against a second
live disk_usage() call, so any concurrent write between the two samples
failed them — reliably under the parallel `make test` targets, never in
isolation. Mock the call and assert which path was measured, which is
what the tests are about.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
