mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-06 10:59:59 +00:00
kevinveenbirkenbachandClaude Opus 5
791eee7c9f
The contract check ran as an Ansible task in the middle of every provider deploy, and the two client checks ran the same way. A test that is part of the deploy cannot be run without deploying, cannot be re-run on its own, and reports its verdict as a deploy failure. Twenty-seven task files lose 491 lines to this.
It is now what it always was. Each provider ships files/test/test.sh, templates/test.env.j2 and meta/tests.yml, and test-e2e-cli stages one shared harness beside it: run.sh drives contract.sh in a throwaway container on the provider's own network, then isolation.sh for adapter-fronted providers. The read probe each provider offers moved from the calling task into meta/mcp.yml next to the allowlist it must satisfy, and the schema now rejects a read probe naming a tool outside that allowlist.
The bearer the test presents is the one a client resolves through mcp.credential, via the new mcp_client_bearer lookup over resolve_credential, which moved to utils.roles.applications.mcp so it is the same function mcp_servers renders every client connection with. The probe presented whatever the calling role passed, which for the fifteen adapter-fronted providers is a different secret than any client holds.
Sidecar isolation is read from the engine rather than measured inside the container, so it no longer needs a delegation to the node holding the task: attachment is compared by network name, which drops the gwbridge special case, and a swarm worker says it cannot measure instead of skipping silently.
Retrying is bounded by wall clock against MCP_DEADLINE_SECONDS, which scales through the same lookup('timeout') as the harness budget it has to stay under. A flag that is neither true nor false aborts rather than reading as off.
mcp-provider-probe becomes mcp-provider-cli-test: an enabled provider must ship the test rather than include the task. mcp-probe-bearer becomes mcp-adapter-bearer, tying ADAPTER_BEARER to the declared credential key now that no probe include carries it. A new integration test renders the shared env template for every provider through Ansible's own templar, so a later change to it breaks visibly instead of poisoning twenty deploys quietly.
Hermes and OpenClaw keep their client-side proof as CLI tests of their own.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
55 lines
1.6 KiB
Bash
55 lines
1.6 KiB
Bash
#!/usr/bin/env bash
|
|
# CLI test for web-app-hermes: the agent reaches every MCP server it was
|
|
# configured against, and each one answers with a tool list.
|
|
#
|
|
# Env (rendered into test.env from templates/test.env.j2):
|
|
# MCP_TEST_ENABLED whether this round configures MCP for Hermes at all
|
|
# MCP_CLIENT_CONTAINER addressable Hermes container
|
|
# MCP_SERVER_IDS JSON array of the discovered server ids
|
|
set -euo pipefail
|
|
|
|
: "${MCP_TEST_ENABLED:?}"
|
|
: "${MCP_CLIENT_CONTAINER:?}"
|
|
: "${MCP_SERVER_IDS:?}"
|
|
|
|
RETRIES=20
|
|
SLEEP_SECONDS=6
|
|
|
|
if [[ "${MCP_TEST_ENABLED}" != "true" ]]; then
|
|
echo "SKIP: MCP is switched off in this round"
|
|
exit 0
|
|
fi
|
|
|
|
mapfile -t servers < <(printf '%s' "${MCP_SERVER_IDS}" |
|
|
python3 -c 'import json,sys; print("\n".join(json.load(sys.stdin)))')
|
|
|
|
if [[ ${#servers[@]} -eq 0 ]]; then
|
|
echo "[FATAL] MCP is on but no server was discovered; a client with an" \
|
|
"empty server list is indistinguishable from one that lost its" \
|
|
"providers" >&2
|
|
exit 1
|
|
fi
|
|
|
|
failed=0
|
|
for id in "${servers[@]}"; do
|
|
attempt=0
|
|
while :; do
|
|
out="$(container exec -i "${MCP_CLIENT_CONTAINER}" hermes mcp test "${id//-/_}" 2>&1 || true)"
|
|
if [[ "${out}" == *"Tools discovered:"* && "${out}" != *"Connection failed"* ]]; then
|
|
echo "[OK] ${id}"
|
|
break
|
|
fi
|
|
attempt=$((attempt + 1))
|
|
if [[ ${attempt} -ge ${RETRIES} ]]; then
|
|
echo "[FAIL] ${id}: ${out}" >&2
|
|
failed=1
|
|
break
|
|
fi
|
|
sleep "${SLEEP_SECONDS}"
|
|
done
|
|
done
|
|
|
|
[[ ${failed} -eq 0 ]] || exit 1
|
|
|
|
echo "ALL CHECKS PASSED"
|