ruff B905 flagged zip(entries, anchors) without strict=. The lines directly above reset anchors to len(entries) whenever the counts disagree, so strict=True can never fire and records that pairing; a future edit that drops the alignment now fails loudly instead of silently truncating and skipping the very offenders this rule exists to catch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role sets up the .onion split-DNS and points /etc/resolv.conf at dnsmasq,
but resolv.conf only feeds the glibc `dns` module, and the image ships
hosts: files myhostname resolve [!UNAVAIL=return] dns
`[!UNAVAIL=return]` returns on every status except UNAVAIL, so once
systemd-resolved refuses a name, glibc never reaches dnsmasq. systemd-resolved
refuses .onion as a special-use domain, which makes the role's own split-DNS
unreachable to every NSS consumer on the host.
Measured in run 33785198875 job 100756597086: dnsmasq kept the same pid from
20:30 to the capture, logged nothing, and answered `dig @127.0.0.1` correctly
at 22:05:05, while getaddrinfo failed for 28 minutes. Fourteen .onion domains
returned EAI_NONAME in 4-5ms each, an authoritative answer rather than a
timeout, and the one clearnet domain in the same probe was fine. msmtp failed
26 times on the same suffix, a second consumer of the same path. 11c98825aa
addressed the trigger, a bind-dynamic listener flap that recovered in 30
seconds; this addresses why the failure outlived it by 28 minutes.
Only NOTFOUND changes. `resolve` keeps its place ahead of `dns`, so
systemd-resolved still answers everything it knows and SUCCESS still returns;
TRYAGAIN keeps returning, so a timeout does not buy a second query on a host
whose upstream is already struggling. Stripping the qualifier outright, which
was the first shape of this fix, would have changed three statuses to buy one.
The two halves now sit in one block under a single TOR_DNSMASQ_OWNS_LISTENER
guard: pointing resolv.conf at dnsmasq and making NSS able to use it are one
concern, and splitting them is what left the role half finished.
Residual worth naming: if systemd-resolved answers UNAVAIL rather than
NOTFOUND for .onion, the old qualifier already fell through and the cause lies
elsewhere. The 4-5ms replies argue for NOTFOUND but do not settle it. The
in-zone probe cannot settle it either: 56b8f1716b taught it to find the zone
in the drop-in directory, but it still asks through `getent`, i.e. through the
very layer in question. A direct per-nameserver dig would decide it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
hermes_hermes.1 ran on a worker while run_one.yml ran its script on the loop
host. resolve_container_id.sh finds nothing node-locally on the manager, falls
through to the cluster-wide branch and returns the task's ContainerID, which
only the owning node's daemon knows, so `container exec` answered
`No such container: 65801ca259a1` twenty times over 116 seconds. Run
33785198875, job 100756596935.
Resolve the owning node and run there. meta/tests.yml gains
container_service_key for the two roles that exec into their own container; a
role without the key keeps the loop host, and role_tests returns the default
for a missing path, so absence is a polarity rather than an error.
The reset at run_one.yml:27-30 is unconditional and precedes every
delegate_to. Without it the first loop iteration dies with
'CLI_HOST_NODE' is undefined, because _cli_apps starts with a role that has no
key; the same task closes the fact leak that include_tasks leaves open between
iterations. The resolve include is gated on an explicit length check rather
than truthiness, or a keyless role would hang in resolve_host_cid's converge
wait, and it caps that wait at 30 x 2s instead of inheriting 600 x 2s: the
deploy is long finished by the time the CLI stage runs, so failing fast beats
waiting twenty minutes.
Staging, render and run all delegate. Delegating only the run would turn
`No such container` into `No such file or directory`, since the assets would
stay on the loop host. Both templates now carry CLI_LOCAL_CID: with the exec
on the owning node the id is node-local, so container_address's deferred
`$( ... )` is not needed, and it never survived `set -a && . test.env` anyway,
which evaluates it once at source time.
The stage directory is removed on the delegate too. tasks/main.yml clears it
without delegation, i.e. only on the loop host, and the task's own name calls
its contents decodable DB secrets.
Measured locally rather than argued: self-delegation leaves file, copy with
content, copy with src and shell byte-identical, including mode; the loop
reset gives keyless roles an empty container and keeps CLI_HOST_NODE on the
loop host across five iterations; and container_service_key resolves to
hermes, openclaw and empty strings through the same walk cli.timeout already
uses in production.
Not fixed, and named so it is not forgotten: web-app-joomla, web-app-yourls
and web-app-penpot exec from an IS_STACK_HOST gate without resolve_host_cid,
so a task of theirs landing on a worker hits the same error from the same
source.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
roles/<role>/tasks/main.yml gates 0N_core.yml behind
`run_once_<role> is not defined`. Entering that body from a different role
with `include_role: {name: <role>, tasks_from: 0N_core.yml}` bypasses the
gate, because the guard lives in main.yml and tasks_from never reads it, so
the body runs once per calling task instead of once per play.
For sys-svc-compose that means the shared compose pull-lock directory is wiped
again, and files/python/pull.py skips a pull only while its lock file exists,
so every stack deployed so far pulls again; under MODE_RESET it also reaches
01_reset.yml, which brings every project in DIR_COMPOSITIONS down and deletes
the directory holding every application's instance, env and volume paths. For
sys-svc-container it drains whatever topic is pending through the
unconditional meta: flush_handlers at 00_core.yml:53.
svc-runner carried the only two unguarded entries in the tree. This is not a
new convention: sys-svc-container is guarded this way at seven existing call
sites, in svc-opt-ssd-hdd, sys-ctl-hlth-csp, svc-swarm-node, desk-docker, and
twice inside sys-svc-compose itself.
The guard belongs at the call site rather than inside the core file.
sys-svc-compose/tasks/00_core.yml sets its flag as its first task on purpose:
it includes sys-svc-container, and utils/network/routine.yml leads back into
sys-svc-compose, so setting the flag first is what breaks the recursion. The
first-task lint records that rationale. Moving the flag to the end to let the
body self-guard would trade one defect for another.
The rule parses the task structures rather than matching lines: it walks
block, rescue and always, reads include_role and import_role including the
ansible.builtin spellings, and accepts a guard whether `when` is a scalar or a
list. A line anchor is still used for the report and the nocheck, not for the
verdict. A scratch control covers seven shapes; two of them, an inverted key
order and an entry nested in a block, escape a line-based rule entirely.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
svc-ai-mcp-adapter/tasks/rebuild.yml notifies compose-build and compose-up.
00_core.yml then flushed two tasks later, while the stack it was about to act
on had not been rendered yet: qdrant reaches sys-svc-compose only at the end,
through utils/network/routine.yml, not at the start through sys-stk-full.
svc-ai-mcp-adapter/tasks/instance.yml:12-16 already names this hazard in prose
and works around it with five explicit `notify: []` entries, because a
notification raised before the render "fires a compose up against a project
that has no compose.yml yet"; it records that rebuild.yml raises it once both
the context and the compose file exist. The flush in between broke that.
Drop it. The notification now survives to the flush inside
utils/network/routine.yml:13-17, which enters sys-svc-compose with
compose_handlers_flush: true, i.e. after 02_stack_host_prep has rendered and
under qdrant's own application_id.
That alone would move the notification into the window where routine.yml:2-6
pulls in sys-svc-container, whose 00_core.yml:53 is an unconditional
meta: flush_handlers, so a play in which qdrant is the first compose consumer
would drain it before the wrapper exists. The guarded bootstrap ahead of the
notify absorbs that flush against an empty queue, the same shape 252b3e58b2
applies in sys-svc-rdbms.
Only qdrant is affected. The commit that added the sidecars gave ten roles
this shape; the three others that pair rebuild with a flush render first and
are correct: web-svc-libretranslate (sys-stk-full :9, rebuild :13, flush :17),
web-app-prometheus (sys-stk-backend :11, rebuild :56, flush :60) and
web-app-zammad (sys-stk-full :11, rebuild :16, flush :24). qdrant is the only
svc-db role in that set, and svc-db roles load their stack last.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The suite covered only the roles that serve an MCP endpoint. The roles that
consume one render the same test.env interface through the same path and were
covered by nothing. `_render` also pinned compose_mode and DEPLOYMENT_MODE to
`compose`, so the swarm branch of lookup('container_address') was never
entered anywhere under tests/ -- and that branch is the one that emits the
deferred `$( ... )` form the CLI harness then freezes at source time.
`_providers` becomes a call to `_roles_with_direction` with the same direction
set, `_clients` its counterpart, filtered to roles that also ship
templates/test.env.j2 so a declared client without a CLI test is skipped
rather than crashing setUpClass. `_render` gains a mode parameter defaulting
to compose, so every existing call is unchanged.
Client templates need the variable layers run_one.yml:44-70 loads before it
renders: web-app-hermes resolves HERMES_MCP_SERVERS from its own vars, which
resolves MCP_DISCOVERED_SERVERS from group_vars/all. Both layers are mirrored
under the explicit fixture vars, with trust_as_template applied recursively
because ansible-core does not template a loaded value otherwise.
One interpreter is enough for both modes: plugins/lookup/container_address.py
:128 reads DEPLOYMENT_MODE straight from the templar variables, so the
per-process config cache that limits the provider fixture to one polarity of
MCP_TEST_ENABLED does not reach the mode split.
The four new assertions are mode-independent on purpose. Asserting that
MCP_CLIENT_CONTAINER carries no `$(` belongs with the change that delegates
the exec to the node holding the task; it fails today and would make this
commit red.
No coverage is lost. The provider renders were compared with and without the
new layers across all 20 of them and are byte-identical, and the suite goes
from nine test methods to thirteen.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
roles/sys-svc-container/templates/deploy.yml.j2 appends `node.role == manager`
at :11-13 for any role carrying `placement: manager`, and
`node.labels.<SANDBOX_NODE_LABEL> == true` at :18-20 for any role whose
`services.kata.enabled` is truthy. Both land in the same `_constraints` list,
which swarm evaluates as a conjunction. roles/svc-virt-kata/tasks/00_core.yml
:62-70 adds that label only `when: not (IS_STACK_HOST | bool)`, so the manager
never carries it and the two constraints have an empty intersection.
The pin therefore does not move the workload to the manager, it makes it
unschedulable, and the failure surfaces as a converge timeout rather than as a
placement error. That is why web-app-hermes was proposed for such a pin during
the GitLab#1 investigation and why the proposal was rejected.
The existing default-placement-manager rule cannot catch this: it only checks
that a nocheck marker is present, never that the justification holds, so a pin
excused on general grounds still ships. This rule demands its own marker for
the one case that is arithmetically impossible.
It reads YAML rather than matching lines, because `kata` is a top-level
sibling service in meta/services.yml and not a key inside the pinned service
block, so the correlation spans the file. File scope is right: both
constraints are role-scoped, not service-scoped. The line number for the
report and for the nocheck still comes from a match; only the verdict comes
from the parsed document. A templated `enabled` is not statically decidable
and is passed over rather than guessed at.
Green across all roles today, and a scratch control confirms it fires on a
pin plus kata, stays silent on either half alone, on a template, and on the
nocheck.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The CLI test discarded the exec status with `|| true` and judged the run on
its output alone: non-empty, and not carrying a `failed to start server`
line. `Error response from daemon: No such container: <id>` satisfies both,
so the script echoed the daemon's error, printed ALL CHECKS PASSED and
exited 0. CI reported green for a container it never reached.
That path was live in run 33785198875 job 100756596935, where hermes_hermes.1
sat on a worker while roles/test-e2e-cli ran the exec on the manager. The
harness half of that failure is a separate change; this one is about the
predicate, which would have reported success for any unreachable container in
any mode.
Capture the status and require it. The output guard stays as it is: no
positive success marker is introduced, because nothing in the repository
records what `mcp probe` prints when it succeeds, and a guessed marker can
break a working test. web-app-hermes keys on `Tools discovered:` for the same
reason the other way round, but that string is its own.
The condition gains a conjunct, so no case that was checked before is
unchecked now. The residual runs the other way: should the probe ever exit
non-zero while still producing usable output, a run that passed will now
fail.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two branch-only commits each armed a defect in the compose-up handler
topic, and both failed in run 33785198875.
63f7f962bf registered the topic owner at the top of sys-svc-rdbms'
dedicated.yml so the notify two tasks below stopped aborting with
ERROR_ON_MISSING_HANDLER. The notification then survived to the
unconditional meta: flush_handlers at sys-svc-container/tasks/00_core.yml:53,
reached from sys-svc-compose's own 00_core before that role installs
BIN_COMPOSE, and long before 02_stack_host_prep installs BIN_CA_INJECT.
Matrix#1 died there on ca-inject with rc 2 after seven retries. That play
carries exactly one compose notification (job 100756596907, log line
11046), so closing this window is sufficient, not merely necessary.
dedicated.yml now enters 00_core.yml under the role's run-once flag
before it notifies, so the flush drains an empty queue and the flag then
makes main.yml skip the body on the real entry. The registration itself
stays ungated: gating it would skip the owner from the second app on.
95186163aa hoisted the sys-svc-compose include to sys-stk-backend
position 1. The consumer therefore rendered and notified compose-up
before sys-svc-objstore lazily loaded the provider app role, whose own
unconditional flush consumed the notification and ran the single
registered handler in the provider's chdir. mastodon and peertube never
got a stack, in compose and in swarm alike; the victim's own guard task
appears zero times in all four logs. Restoring the include below rdbms,
objstore and engine puts the notify after every foreign crossing. Two
pre-hoist runs in exactly that order are green on record.
The test keeps its cases and gains one. It selects the render include by
the absence of tasks_from instead of by first occurrence, so an early
registration-only include stays legal while a hoisted render still fails.
The new case pins the bootstrap: entry point, position before the first
notify, and the run-once guard whose removal would let 01_reset.yml tear
down DIR_COMPOSITIONS once per dedicated-DB app.
Same class, pre-existing, not fixed here: svc-db-qdrant notifies and
flushes three tasks before its first compose entry; web-app-magento pairs
a dedicated database with a shared object store; web-app-seaweedfs still
flushes a topic it does not own; svc-runner enters 00_core.yml unguarded.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The hub serves 94 tools across 23 categories, write tools included, and enabling the ExApp switches every category on. That surface was published on the public vhost and every discovered client was handed an app password for it. svc-ai-mcp-adapter now sits in front in mcp_passthrough mode: clients present credentials.mcp_bearer to contextagentmcp and reach the nine read tools of calendar, contacts and files, which files/mcp/tools.json names and the sidecar enforces per call. Those three categories are the ones whose upstream is_available is unconditional, so the served set does not vary with which optional apps a deployment installs.
Enforcing it in the sidecar rather than in each client is what makes it hold: web-app-openwebui renders no per-tool filter at all, so a client-side include list would have bounded two consumers out of three.
06c13e7f3e withdrew the declared path from the edge for exactly this reason - "that surface is what the adapter exists to narrow". Nextcloud answers the same route under a second spelling, without the /index.php front controller, which try_files rewrites back onto the route after the edge has already declined to block it. The withdrawal becomes a regex covering both, and only where stripping the front controller changes the path, so the four adapter providers without one render unchanged.
The app password is minted after the stack renders, so it reaches the sidecar through upstream.env and rebuild.yml rather than the compose environment: compose up resolves env_file while creating the container, a restart does not. The sidecar is the last service in the template because a Jinja set has no block scope and its env_file would otherwise displace the one talk, whiteboard and talk_recording each carry.
Two suites cover what nothing measured: the vhost template rendered without a container, asserting both spellings are withdrawn, and every shipped tools.json loaded through the adapter own loader, so a bad contract fails in a second instead of as a probe timing out an hour into a deploy. The guest spec probed mcp.endpoint.path, which this change makes the proxy answer 404 for its own reasons; it now probes all three public paths and requires no MCP body.
Not yet deployed: the sidecar build, the upstream.env delivery and the hub answering exactly nine tools are unverified against a running stack.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
supported_version and minimum_version sat in 24 roles and were read by no template, no task and no lookup. Their only consumer was the audit report version column, which turned false the moment an automated image bump ran: qdrant reported a verified version nobody had verified, two releases long. Both leave MCP_KEYS, so a role that declares them again fails the schema lint instead of carrying a field with no effect.
Requirement 025 mandated minimum_version as a MUST and 035 carried it in an example; both are corrected here, otherwise the repository would contradict its own code.
The gate then surfaced five dead docs.mattermost.com links. Mattermost moved to /administration-guide/ and /deployment-guide/, and every replacement was probed for 200 rather than guessed. The MCP page is gone with no successor, so mattermost now points at mcpserver/README.md in the Agents repository, the source rather than a rendered mirror.
Both concerns share three files - mattermost's meta/mcp.yml, requirement 025, and the generated audit.md, which derives from a source of each - so splitting them would commit an artifact that does not match its sources.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The rule lived only in suite_passed's body, and it inverts easily on a
casual read: ask for the suite again when it failed. Under that reading
the retrigger aliases would drop the suite in their most common case,
because they force-cancel the source run before reading it, so a shard
that was still running comes back cancelled rather than red.
Only an all-green suite is retired. Failed, cancelled, skipped, timed
out, still running and never dispatched all keep it on the retrigger,
and each of those now has a test. The two docstrings enumerate them, and
so does the i8ciall description, which is where an operator reads it.
The name added in 63f7f962b was 112 characters and failed
tests/lint/repository/test_name_emoji_and_length.py.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
dedicated.yml notifies compose-up but registers no handler for it, so a
caller that reaches sys-svc-rdbms before any compose consumer has loaded
the sys-svc-compose block dies with "The requested handler 'compose-up'
was not found" and no PLAY RECAP. 95186163aa guarded only the
sys-stk-backend caller; web-app-matrix's ansible flavor includes
sys-svc-rdbms directly in 03_database.yml and still hit it as the first
compose consumer of its play (run 33751742070, job 100643400267).
Register the owner as the first task of the notifying file. In the
sys-stk-backend path this lands before the pre-flush rebind, so the app's
copy stays the last registration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
65c72fc374 settled the convention: import unittest, and reach the submodule as import unittest.mock as mock, so a call site has one spelling of the name. test_onion.py and test_tor_socks_proxy.py still carried from unittest import mock; CodeQL reports them as py/import-and-import-from, the last two in the repository.
Same one-line change per file the sweep applied; mock.<attr> call sites untouched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ansible resolves a notification to the last handler block registered
under that name and templates application_id in that block's context.
Since 95186163aa sys-svc-compose is loaded first, so the provider roles
included after it (svc-db-postgres via sys-svc-rdbms, objstore, engine)
leave their own copy as the last one, and the trailing flush deployed
the dedicated postgres instead of the app. Run 33751742070: six jobs
waited on a keycloak or mattermost container that was never created.
A dynamic include_role directly before the flush makes the app's copy
the last registration again. import_role would not do: it expands when
main.yml is parsed, before the providers run. The first include stays,
so the missing-handler error stays fixed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
assertTrue(x > 0) fails as "False is not true" and hides the value that failed; assertGreater(x, 0) reports the actual count. Same predicate, so pass and fail are unchanged on every input. CodeQL py/imprecise-assert on both generator tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four module-level constants were defined and never read: _OPERATORS in the complexity filter, superseded by the op group of _TOKEN_RE two lines below; _VALID_SWARM_CONDITIONS, which swarm_restart_condition never checks because it maps to one of three literals by construction; _MISSING in services/config.py, where resolution already goes through get(strict=False, default={}); and _DEFAULT_FLAG_VALUES_THAT_ARE_NOT_NAMES in the exec-resolver lint, referenced by no rule in it. The nfs_fstype lookup rebound its variables parameter and returned without reading it.
Every name was private to its module and a repo-wide search over every file type finds no reader. CodeQL flagged them as unused; ruff cannot, because pyflakes has no rule for module globals and treats a rebound parameter as used.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CodeQL reports cmd and sort_spec as possibly uninitialised after the except branch that calls parser.error, because it does not model the argparse error method as NoReturn. It is: usage is printed and the process exits with status 2. An explicit return 2 on that branch states the exit status the call already produces, so the analyser and a reader see the same path.
The retry loop in the registry client counted two attempts in a variable the body never read; the underscore prefix was the convention for that, but CodeQL only honours the bare name.
No path changes behaviour; the three alerts stop pointing at correct code so the scanner stays readable for the ones that are not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The validation job checks out the pull request head and runs make install from it, then setup-python saves the pip cache in its post-step, after that code has run with write access to the cache directory. The workflow is reachable through workflow_dispatch, where the cache key resolves in the default-branch scope, so a poisoned entry written there is restored by every later trusted run. CodeQL flags each run step after the checkout as poisonable.
Removing the cache removes the sink. The checkout stays as it is: read-only token, no persisted credentials, excluded under pull_request_target. Nine other setup-python steps keep their cache; this is the one job that executes untrusted code, and the comment says so where the next tidy-up would put the cache back.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Seven workflows declared actions: write or packages: write at the top level, so every job in the file inherited it, including any job added later by someone not thinking about the token. Scorecard flags each as TokenPermissionsID.
Each write now sits on the single job that uses it and the top level is contents: read throughout. The read scopes those jobs also need moved down with the write, so no job is narrower than before; each file has one job, so the effective token per job is unchanged and only the default for a future job changes.
call-images-mirror-missing.yml is reusable; both callers already grant packages: write on their own job, so the caller-side cap is the same as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
mutating_tools_enabled: false was a claim with nothing behind it. The agent include list is built by subtracting tools.mutating from tools.allowlist, and the role named no mutating tool, so the subtraction removed nothing and every client was offered HassTurnOn, HassTurnOff, HassBroadcast, HassCancelAllTimers and the three list writers while the declaration said mutations were off.
Naming the seven turns the flag into the control it claims to be: a client now sees GetDateTime, GetLiveContext and todo_get_items, and setting the flag to true restores the full surface. The adapter already refuses a contract whose tool carries no mutating verdict; this brings the one native provider that serves actuating tools under the same rule. Entity exposure still bounds what the read tools reach.
The README described entity exposure as the only control and said the write tools stay listed; it now describes both layers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Hermes and OpenClaw wrapped the tool allowlist in a guard that dropped the key when the callable set was empty. Both upstreams read an absent filter as "register every tool the server serves", so a provider resolving to no tools got the widest surface instead of none. Nextcloud keeps its contract in tools.categories with no allowlist and admits both clients, so it was the live case.
The templates now render the filter unconditionally, the way the n8n client node already pins include to selected; the two tests that asserted the omission are inverted. mcp_servers additionally rejects such a provider with the fatal code tools_unenforceable: a client filters by tool name, so a runtime-composed surface reaches it as no surface at all, and a server the agent can never call is worse wired in than refused with a reason.
A deployment pairing Nextcloud with either agent now aborts naming the provider, where it previously handed the agent every Context Agent tool unfiltered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The regular line restarted at the head on every retrigger. resume_offset
compared the source run's deploy tokens against a freshly computed
ranking, but every axis rotates with the sweep number and a retrigger is
a new run with a new one: measured against the current 277-row ranking,
2 of 277 tokens matched. The walk stopped on its first row every time,
and the 66 rows past the job budget never deployed at all.
Membership now compares role#variant, the identity the rotation cannot
move, and a row counts as proven only when every job it had was green.
A priority row deploys its whole cross-product, so one role#variant can
carry several verdicts, and a green compose job must not carry a red
swarm one past the line. The offset also inherits the source run's own
offset as its floor rather than falling back to the head.
The priority line drops --include-unrun with it: only the hard failures
and the priority entries the run never deployed belong there. The rows
it never reached are the regular line's turn now that the offset walks.
sys-svc-rdbms notifies compose-up from dedicated.yml, but sys-svc-compose, the only role that owns that topic, was included after it. With ERROR_ON_MISSING_HANDLER on, the strategy raises before the task result is emitted, so the play dies with '[ERROR]: The requested handler compose-up was not found' and no PLAY RECAP. Rounds that happened to deploy another compose consumer first masked it; a round that deploys the app alone does not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
d14354e892 added both probes reading only /etc/dnsmasq.conf, but svc-net-tor writes its onion zone to /etc/dnsmasq.d/tor-onion.conf and leaves the main file with nothing but a conf-dir pointer. Every Tor host of run 33698844864 therefore captured resolve-inzone.txt as '[no output, exit 2]' and all 471 sampler lines as inzone=unconfigured: during the resolver outage that ended Gitea#0, nobody ever asked dnsmasq for an onion name, and two investigations went without the one measurement that would have settled them.
Both probes now follow the conf-dir pointer, main file first so a lab node with its zone in the main config answers as before. Each is exercised by a test that runs it against real files rather than comparing its text, which the rescue constant had no test for at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
9ae0c7eff6 replaced bind-interfaces with bind-dynamic when it taught the role to bind the node address for swarm, an address that appears after dnsmasq starts. Without a bridge address there is nothing to wait for, and the rebuild bind-dynamic performs on every netlink event can then only drop the listener: on the DiD host of run 33698844864 job 100479883964 the sampler lost dnsmasq's 127.0.0.1:53 for four consecutive samples one second after a veth unregistered.
interface=lo is deliberately not added. dnsmasq unions interface= and listen-address= rather than intersecting them, measured on 2.93: naming lo additionally seizes 127.0.0.53 and ::1, and against a distro stub dnsmasq exits 2, which any_errors_fatal turns into a failed deploy.
The resolv.conf keeps both nameservers. bb0e098a9f added the second one from a measured total outage where dnsmasq held no socket at all, and that failure mode is untouched by this change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The OIDC admin wiring is gated on SSO alone, and SSO is 'web-app-keycloak' in group_names. Round 1 takes keycloak variant 1, which pins ldap off, so svc-db-openldap is not in the closure; the block still resolved its swarm service and the waiter spent 600 attempts on a service that does not exist (run 33698844864, Mattermost#1, 25 minutes). In compose the same block would have died on a missing container, so the defect is mode-independent.
The number it fetched is already in the registry: svc-db-openldap/tasks/04_users.yml:118 writes uidNumber from the same lookup('users'), and an earlier run shows inventory uid 1001 against ldapsearch 1001. Reading it directly removes the cross-stack exec entirely.
Keycloak's mattermost-id mapper reads the uidNumber attribute, which until now only the LDAP federation wrote, so the same round would have produced no id claim to match. ensure_realm_user.sh now mirrors the uid into that attribute whenever it is passed one, leaving callers without a uid unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
headers/buffers.conf.j2 lifts large_client_header_buffers to 32k for exactly one reason, stated in its own first line: Keycloak's tokens. It was included only inside each vhost server block, but nginx picks the buffer from the server it has selected while parsing, and that is the catch-all default server until the Host header arrives. Ruby's OIDC clients send Authorization first, so the /userinfo call carrying the access token hit nginx's built-in 8k instead.
Run 33698844864 shows the consequence twelve times per Decidim job: 'client sent too long header line: "Authorization: Bearer ..."', a 400 answered by nginx itself (UpstreamTime: -), omniauth reporting API Access Failed and redirecting to the app's own sign-in page, which is the URL the assertion reports. The same shape ends the mastodon and gitlab logout specs.
Including the partial at http level covers the catch-all, the 443 reject block and every server that forgets the include; the vhost include stays and sets identical values.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
utils/roles/applications/config.py turns a missing key with a None default into False, so the provider-side probe in attachments.py read an omitted mcp_consumer flag as an explicit refusal. No MCP client ever joined a provider's shared_net: open-webui rendered no gitlab network, http://gitlabmcp:8080/mcp did not resolve from its namespace, and Open WebUI's verify call answered 400 in both GitLab#1 jobs of run 33698844864, clearnet and tor alike. Measured on that run's own inventory: without the change open-webui does not join the gitlab overlay, with it it does, and a sentinel probe shows the provider carries no mcp_consumer key at all.
utils/roles/applications/mcp.py:295 already states the rule this restores, that an omitted flag inherits the client's own declaration and only an explicit false excludes.
The stub in the render tests answered an unset path with the caller's default, which is why the regression passed them; it now coerces the way the accessor does, and the new case fails without the fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
docker stack deploy --detach=false converges on a running replica. web-svc-cdn's build container and every migration job end Shutdown/Complete by design, so the CLI waited for a state those stacks cannot reach: nine jobs of run 33698844864 burned the runner's 20400s budget inside this one handler, six of them on web-svc-cdn. The gate that follows already knows better, is_completed_oneshot accepts exactly that task shape.
The timeout keyword could not bound it either. The stack host is reached over ansible_connection: docker, whose plugin runs the child inside a Popen context manager, and __exit__ waits the child out before the timeout exception escapes; a probe measured 90s of wall clock against a 6s timeout. The bound now sits in the shell, where it works.
Handing the wait back to the gate would reopen the race 87e4313e11 closed, so the deploy stamps its start and names the services it reported touching, and the gate treats their earlier completed update as still pending. An inspect format the daemon rejects falls back to the old one, which only disables that check.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two gate runs and a CI workspace job each reddened on a single opencollective funding link from package-lock.json, and all three URLs answered 200 on every later probe. A one-off 4xx from a third-party host is not a broken reference, so the suite now repeats such a probe once after a pause, the way _probe_canary already treats an unreachable canary. A status that repeats still fails, and a transport error is not retried because it is already only a warning.
The retry runs inside the worker that owns the URL, so the pool keeps its concurrency: the added wall-clock is one pause per wave of failures, and the runs measured 84s and 88s against 100s without it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
26 roles resolve a domain in their metadata, so lookup('tls') and lookup('domains') run inside the applications render. Under the re-entry guard those calls receive the unrendered tree, whose service flags are still Jinja strings, and _as_bool reads "{{ 'svc-net-tor' in group_names }}" as false. The resulting map skipped onion injection for every app that does not enable tor literally, and it was cached, so it answered every later read in that process.
Before the applications carrier each worker rendered and poisoned its own cache the same way, which is why the shared Keycloak client registered 5 onion origins then and 65 now. The render state joins the cache key, so the render's own call sites still share one map while a later read builds the finished one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The SSO gate handed X-Forwarded-Access-Token to every oauth2-fronted upstream, so each of the 28 gated roles received a full-scope bearer for the signed-in user while exactly one, n8n, decodes it. The token also grows with the realm's registered origins, and LAM's Apache rejects a header field above 8190 bytes (Moodle#0, run 33689701895).
Consumers now declare services.sso.oauth2.pass_access_token. Without it oauth2-proxy already answers the auth request without the token, and the gate blanks the upstream header rather than dropping the directive, because nginx inherits proxy_set_header as an array and an unset header would let a client-supplied one through. The new lint keeps declaration and use in step in both directions.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
lookup('redirect_uris') walked every app in the domains map whose services.sso.enabled resolves truthy, which every role spells as "'web-app-keycloak' in group_names". A round deploying 21 apps therefore registered 69 redirect URIs and web origins on the shared client, 65 of them onion, and an app that is not running has no reason to be a redirect target. group_names is now required and scopes the result, matching KEYCLOAK_SAML_APPS which already selects on it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The web-origins scope mapper listed every registered origin in each access token. openresty forwards that token to SSO-gated upstreams, so on a Tor deployment the claim reached 5457 of the token's 8650 characters and LAM's Apache refused the request with 400 (Moodle#0, run 33689701895).
The realm template carries the change, so a fresh import and every CI round get it directly. The reconcile task exists because 04_realm_import.yml only imports when the realm is missing, which would leave the old mapper in place on running installations.
Keycloak resolves CORS from client.webOrigins in DefaultCors.checkAllowedOrigins whenever the token carries no origins, so enforcement is unchanged; that fallback exists since 26.7 and the role pins 26.7.3. Introspection keeps the claim, and nothing in this repository reads it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
