mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-06 02:54:26 +00:00
kevinveenbirkenbachandClaude Opus 5
791eee7c9f
The contract check ran as an Ansible task in the middle of every provider deploy, and the two client checks ran the same way. A test that is part of the deploy cannot be run without deploying, cannot be re-run on its own, and reports its verdict as a deploy failure. Twenty-seven task files lose 491 lines to this.
It is now what it always was. Each provider ships files/test/test.sh, templates/test.env.j2 and meta/tests.yml, and test-e2e-cli stages one shared harness beside it: run.sh drives contract.sh in a throwaway container on the provider's own network, then isolation.sh for adapter-fronted providers. The read probe each provider offers moved from the calling task into meta/mcp.yml next to the allowlist it must satisfy, and the schema now rejects a read probe naming a tool outside that allowlist.
The bearer the test presents is the one a client resolves through mcp.credential, via the new mcp_client_bearer lookup over resolve_credential, which moved to utils.roles.applications.mcp so it is the same function mcp_servers renders every client connection with. The probe presented whatever the calling role passed, which for the fifteen adapter-fronted providers is a different secret than any client holds.
Sidecar isolation is read from the engine rather than measured inside the container, so it no longer needs a delegation to the node holding the task: attachment is compared by network name, which drops the gwbridge special case, and a swarm worker says it cannot measure instead of skipping silently.
Retrying is bounded by wall clock against MCP_DEADLINE_SECONDS, which scales through the same lookup('timeout') as the harness budget it has to stay under. A flag that is neither true nor false aborts rather than reading as off.
mcp-provider-probe becomes mcp-provider-cli-test: an enabled provider must ship the test rather than include the task. mcp-probe-bearer becomes mcp-adapter-bearer, tying ADAPTER_BEARER to the declared credential key now that no probe include carries it. A new integration test renders the shared env template for every provider through Ansible's own templar, so a later change to it breaks visibly instead of poisoning twenty deploys quietly.
Hermes and OpenClaw keep their client-side proof as CLI tests of their own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
57 lines
1.7 KiB
Bash
57 lines
1.7 KiB
Bash
#!/usr/bin/env bash
|
|
# Run the MCP test suite of whichever role staged this harness.
|
|
#
|
|
# A connection error is retried and a REJECTED verdict is not: the endpoint may
|
|
# still be warming up when the round reaches its tests, but a served surface
|
|
# that disagrees with the declared contract will disagree just as much in five
|
|
# minutes.
|
|
#
|
|
# Retrying is bounded by wall clock rather than by a count, because the budget
|
|
# it has to stay under is one: meta/tests.yml states the timeout the harness
|
|
# kills the run at, and both scale through the same lookup('timeout'). A
|
|
# retry count would keep its own budget and drift out from under that one.
|
|
#
|
|
# Env (rendered into test.env from test-e2e-cli/templates/mcp/test.env.j2):
|
|
# MCP_TEST_ENABLED whether this round serves MCP for the role at all
|
|
# MCP_SIDECAR_SERVICE the sidecar, empty for non-adapter providers
|
|
# MCP_DEADLINE_SECONDS when to stop retrying, below the harness timeout
|
|
set -euo pipefail
|
|
|
|
: "${MCP_TEST_ENABLED:?}"
|
|
: "${MCP_SIDECAR_SERVICE?}"
|
|
: "${MCP_DEADLINE_SECONDS:?}"
|
|
|
|
SLEEP_SECONDS=10
|
|
|
|
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
|
|
case "${MCP_TEST_ENABLED}" in
|
|
true) ;;
|
|
false)
|
|
echo "SKIP: MCP is switched off in this round"
|
|
exit 0
|
|
;;
|
|
*)
|
|
echo "[FATAL] MCP_TEST_ENABLED is '${MCP_TEST_ENABLED}', neither true nor false" >&2
|
|
exit 1
|
|
;;
|
|
esac
|
|
|
|
while :; do
|
|
if out="$(bash "${here}/contract.sh" 2>&1)"; then
|
|
echo "${out}"
|
|
break
|
|
fi
|
|
if [[ "${out}" == *REJECTED* || ${SECONDS} -ge ${MCP_DEADLINE_SECONDS} ]]; then
|
|
echo "${out}" >&2
|
|
exit 1
|
|
fi
|
|
sleep "${SLEEP_SECONDS}"
|
|
done
|
|
|
|
if [[ -n "${MCP_SIDECAR_SERVICE}" ]]; then
|
|
bash "${here}/isolation.sh"
|
|
fi
|
|
|
|
echo "ALL CHECKS PASSED"
|