Files
computer-playbook/scripts/tests/deploy/swarm/utils/_context.sh
kevinveenbirkenbachandClaude Opus 5 7162b7bf4c Merge fork/feature/svc-net-tor into feature/ai-agent-employees
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.

The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.

Resolutions that were not unions:

Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.

plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.

compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.

tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.

Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.

01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.

Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.

roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.

config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.

READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.

Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 16:32:30 +02:00

178 lines
5.8 KiB
Bash
Executable File

#!/usr/bin/env bash
set -euo pipefail
_REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../../.." && pwd)"
# shellcheck source=scripts/meta/env/load.sh
source "${_REPO_ROOT}/scripts/meta/env/load.sh"
if [ -s "${SWARM_DRILL_ENV:-}" ]; then
set -a
# shellcheck source=/dev/null
source "${SWARM_DRILL_ENV}"
set +a
fi
: "${APP_ID:?APP_ID required}"
ENTITY="$(PYTHONPATH="${_REPO_ROOT}" "${PYTHON}" -c "from utils.roles.entity.name import get_entity_name; print(get_entity_name('${APP_ID}'))")"
STACK_NAME="${ENTITY}"
CUSTOM_IMAGE_REPO="${ENTITY}_custom"
ROLE_DIR="${_REPO_ROOT}/roles/${APP_ID}"
PRIMARY_SERVICE_KEY="$(PYTHONPATH="${_REPO_ROOT}" "${PYTHON}" -c "
import os, yaml
key = '${ENTITY}'
svc = os.path.join('${ROLE_DIR}', 'meta', 'services.yml')
if os.path.exists(svc):
data = yaml.safe_load(open(svc)) or {}
entity = data.get('${ENTITY}') if isinstance(data, dict) else None
if isinstance(entity, dict):
k = (entity.get('proxy') or {}).get('service_key')
if isinstance(k, str) and k:
key = k
print(key)
")"
SERVICE_NAME="${STACK_NAME}_${PRIMARY_SERVICE_KEY}"
DB_DEP=none
if [ -f "${ROLE_DIR}/meta/services.yml" ]; then
if grep -qE '^mariadb:' "${ROLE_DIR}/meta/services.yml"; then
DB_DEP=mariadb
elif grep -qE '^postgres:' "${ROLE_DIR}/meta/services.yml"; then
DB_DEP=postgres
fi
fi
DEFAULT_PLACEMENT_MANAGER=false
if [ -f "${ROLE_DIR}/meta/services.yml" ] &&
grep -qE "placement:[[:space:]]*manager\b" "${ROLE_DIR}/meta/services.yml"; then
DEFAULT_PLACEMENT_MANAGER=true
fi
PROBE_PORT="$(PYTHONPATH="${_REPO_ROOT}" "${PYTHON}" -c "
import os, yaml
port = 80
svc = os.path.join('${ROLE_DIR}', 'meta', 'services.yml')
if os.path.exists(svc):
data = yaml.safe_load(open(svc)) or {}
entity = data.get('${ENTITY}') if isinstance(data, dict) else None
if isinstance(entity, dict):
try:
port = entity['ports']['internal']['http'] or 80
except Exception:
port = 80
print(port)
")"
NFS_VOLUMES=""
if [ -f "${ROLE_DIR}/meta/volumes.yml" ]; then
NFS_VOLUMES="$(python3 -c "
import sys, yaml
with open('${ROLE_DIR}/meta/volumes.yml') as f:
data = yaml.safe_load(f) or {}
if isinstance(data, list):
entries = [e for e in data if isinstance(e, dict)]
elif isinstance(data, dict):
entries = [v for v in data.values() if isinstance(v, dict)]
else:
entries = []
for v in entries:
if v.get('type') != 'volume':
continue
if v.get('nfs') is False:
continue
docker_name = v.get('docker_name') or v.get('name')
if isinstance(docker_name, str):
print(docker_name)
")"
fi
PRIMARY_NFS_VOLUME="${NFS_VOLUMES%%$'\n'*}"
NFS_CHECK_MOUNTPOINT="/mnt/nfs-check"
export APP_ID ENTITY STACK_NAME SERVICE_NAME PRIMARY_SERVICE_KEY CUSTOM_IMAGE_REPO DB_DEP NFS_VOLUMES PRIMARY_NFS_VOLUME DEFAULT_PLACEMENT_MANAGER PROBE_PORT NFS_CHECK_MOUNTPOINT
# Exits the calling chaos step (09/10/11) with success when the role is
# manager-pinned (no worker task exists to drain).
skip_chaos_if_manager_pinned() {
if [ "${DEFAULT_PLACEMENT_MANAGER}" = true ]; then
echo "SKIP: ${ENTITY} declares placement: manager (single-node on the swarm manager) — drain-a-worker reschedule chaos does not apply"
exit 0
fi
}
has_swarm_service() {
local _names=""
[ -n "${MGR:-}" ] || return 1
if ! _names="$(docker exec "${MGR}" docker stack services \
--format '{{.Name}}' "${STACK_NAME}" 2>/dev/null)"; then
_names=""
fi
printf '%s\n' "${_names}" | grep -qxF "${SERVICE_NAME}"
}
# Prints the swarm service carrying APP_ID's database. A role that pins its own
# engine runs it as <entity>_database inside its own stack, a central provider
# as <dep>_<dep> in its own (plugins/lookup/database.py).
resolve_db_service() {
local _names=""
[ "${DB_DEP}" != "none" ] || return 0
[ -n "${MGR:-}" ] || return 0
if ! _names="$(timeout 30 docker exec "${MGR}" docker service ls --format '{{.Name}}' 2>/dev/null)"; then
return 0
fi
if printf '%s\n' "${_names}" | grep -qxF "${STACK_NAME}_database"; then
printf '%s\n' "${STACK_NAME}_database"
elif printf '%s\n' "${_names}" | grep -qxF "${DB_DEP}_${DB_DEP}"; then
printf '%s\n' "${DB_DEP}_${DB_DEP}"
fi
}
# Exits the calling gate (03/05/06/07) with success when the role deploys no
# swarm service to converge or drain.
skip_if_no_swarm_service() {
if ! has_swarm_service; then
echo "SKIP: ${ENTITY} (${APP_ID}) deployed no swarm service — converge/chaos gate does not apply"
exit 0
fi
}
# A stack whose sidecar is named after its app (<app>mcp) makes every
# `--filter name=<app>` a two-hit match, so both helpers below match exactly.
# Param: $1 node container, $2 swarm service name
service_container_id() {
docker exec "$1" docker ps \
--filter "label=com.docker.swarm.service.name=$2" \
--format '{{.ID}}' | head -1
}
# Param: $1 manager container, $2 swarm service name
service_replicas() {
docker exec "$1" docker service ls \
--filter "name=$2" --format '{{.Name}} {{.Replicas}}' |
awk -v svc="$2" '$1 == svc { print $2 }'
}
# Param: $1 node container, $2 app container id, $3 port
probe_app_reachable() {
docker exec "$1" docker exec "$2" sh -c \
"curl -sS http://localhost:$3/ || wget -qO- http://localhost:$3/ || nc -z localhost $3" >/dev/null 2>&1 ||
docker exec "$1" docker exec "$2" bash -c "exec 3<>/dev/tcp/localhost/$3" >/dev/null 2>&1 ||
docker exec "$1" sh -c "docker inspect -f '{{.State.Health.Status}}' $2 | grep -qx healthy" >/dev/null 2>&1
}
if [ "${SWARM_NFS_PILOT_VERBOSE:-0}" = "1" ]; then
echo "==> APP_ID=${APP_ID}"
echo " ENTITY=${ENTITY}"
echo " STACK_NAME=${STACK_NAME}"
echo " SERVICE_NAME=${SERVICE_NAME}"
echo " CUSTOM_IMAGE_REPO=${CUSTOM_IMAGE_REPO}"
echo " DB_DEP=${DB_DEP}"
echo " PRIMARY_NFS_VOLUME=${PRIMARY_NFS_VOLUME}"
echo " DEFAULT_PLACEMENT_MANAGER=${DEFAULT_PLACEMENT_MANAGER}"
echo " PROBE_PORT=${PROBE_PORT}"
fi