mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-06 19:01:54 +00:00
kevinveenbirkenbachandClaude Opus 5
1e88eb9d1d
update_config never set failure_action, so swarmkit's default applied: pause, at a max_failure_ratio of zero. One failed task stops the rollout for good, and nothing in the deploy resumes it. That contradicts the restart_policy three lines below. The policy replaces the task and the service ends up healthy, while the update status stays latched and service_converged.sh reads that status before it looks at any task, so the deploy reports a failure over a service that is running. The rescue artifact shows the sequence: the old task shut down, its replacement tripped over a containerd object runsc could not reap, and the third task was Running while the job died on "update latched". Four swarm jobs failed that way, all in the update pass, never in the first. The convergence check is untouched and stays as strict as it was: a genuinely broken spec fails every task and still trips "tasks not running yet". What this does give up: the latch was also the only thing catching a service that keeps crash-looping. At ratio zero any failure latched, so a task dying and restarting forever was guaranteed to be reported. Now the rollout completes, and the check samples task state at one moment - a task on its fiftieth restart still reads as Running, and wait_converged's `until` needs a single lucky sample. Restoring that detection needs the check to look at task churn, which the service ps history already carries and the script prints only on failure today. Not in this commit. max_failure_ratio cannot substitute here: at replicas 1, the common case in this repository, one failure is always the whole service. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>