Files
computer-playbook/scripts
kevinveenbirkenbachandClaude Opus 5 4fbde3b188 test(swarm-drill): prove the volume leg instead of staging a shadow restore
Three steps of the DR drill, one theme: every step now proves what it claims.

[9] restored into the volume's mountpoint while the stack was down - a bare directory,
because docker mounts a volume declaring its own backing store only while a container
holds it. The payload landed on the node disk and the real mount shadowed it on the next
deploy: success reported, nothing delivered, and roughly a gigabyte added to the very
device the drill later runs out of. It now deletes the marker server-side, binds the
backing store at the mountpoint, runs the recover CLI through it, and asserts the marker
on the NFS server's own disk. That also exercises the positive branch of the guard from
773a25ab6 - its refusal path is unit-tested, its pass-through was tested nowhere. When
the manager exposes no backing store the branch skips with a reason: docker volume
inspect is node-local, and with replicas: 1 the volume object usually lives elsewhere.

[3b] replaces a proof that used to happen by accident. The double capture 2efc73d3b
removed also showed, incidentally, that the volume unit really writes - the drill found
its marker in the volume repo too. The new step checks it directly: the newest
backup-docker-to-local generation must be at least as new as DRILL_START_TS (taken on
the manager, the same clock that names generations) and hold a non-empty file under a
volume subtree. Its failure message names the suspicion it exists for: an exclusion that
swallowed every volume. With no volume unit deployed it skips, tracked by
VOLUME_TRIGGERED alongside the existing SECRETS_TRIGGERED.

[5] prints du -sm per machine and repo when the space precondition fails, so the next red
run names what fills the pulled tree instead of leaving it to arithmetic.

Four adversarial review rounds found five defects in this block and confirmed each fix:
an NFS mount reconstructed from driver options the codebase never emits (it is a bind),
the recover CLI's second positional passed a volume name where it takes a host, a missing
volume object aborting the drill instead of skipping, and an umount ordered before the
return-code check, which under set -e would have killed the drill on a busy mount with no
diagnostic and left the bind behind.

None of this has run in a swarm yet. Every failure path is loud and names its cause.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 17:58:47 +02:00
..

Script Collection

This directory contains automation scripts used across the project.

  • system/ contains host and system-facing helpers.

  • container/ contains helpers that run inside the local container.

  • github/ contains GitHub workflow helper scripts.

  • For the canonical Make target index, see make.md.

  • 🛠️ Organize operational helpers for development, CI, release, and maintenance

  • 🧭 Separate responsibilities into focused subdirectories with clear scope

  • 📚 Keep script-based workflows discoverable and easier to navigate

The scope of this folder is script organization at the repository level. Detailed responsibilities belong in the README files of the respective subdirectories.