Files
computer-playbook/docs/contributing/actions/testing/external.md
T
kevinveenbirkenbachandClaude Opus 4.7 98457f98fd refactor(update): unify version bumpers under utils/update + cli/contributing/update + scripts/github/update
Folds the parallel Docker-image and git-ref version-update plumbing
into a single subsystem so the auto-update workflow, the external
freshness tests, and the unit tests all share one module layout.

utils/
  update/
    base.py        # semver primitives shared by both backends
    docker.py      # OCI registry lookups + version: line rewriter
    repository.py  # git ls-remote walker + ref: line rewriter

cli/contributing/update/
  docker/           # python -m cli.contributing.update.docker
  repository/       # python -m cli.contributing.update.repository

scripts/github/update/
  image_versions.sh
  repository_refs.sh

Tests mirror the same shape under tests/{unit,external}/update/{docker,repository}/.

Also widens the Docker bump scope to all roles (previously gated on
`web-*`) so non-web roles like sys-svc-container get bumped too;
pins Bookwyrm's python image at 3.11-bookworm via
`# nocheck: docker-version` with a TODO until upstream supports
Python >= 3.12.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 20:12:07 +02:00

3.0 KiB

External Tests 🌐

External tests depend on live third-party systems such as Docker Hub, GHCR, or other external services. Run them with make test-external.

Goal 🎯

  • You MUST use external tests only when the behavior depends on a live third-party system and a hermetic unit, lint, or integration test would not verify the real condition.
  • You MUST keep external tests outside the default make test flow so normal validation stays reproducible without third-party network access.
  • You SHOULD keep external tests focused, advisory, and easy to run on demand.

Location 📁

  • External tests MUST live under tests/external/.
  • You SHOULD group them by dependency family such as tests/external/update/ or tests/external/github/ when more than one test exists.
  • Every external test directory MUST contain __init__.py so repository-wide test structure checks keep passing.

Requirements 📋

  • You MUST document in the test file why the check is external and what live dependency it contacts.
  • You MUST NOT move an external test into tests/lint/, tests/unit/, or tests/integration/ just for convenience when it still depends on live third-party state.
  • External tests MAY emit warnings instead of hard failures when the goal is advisory freshness reporting rather than merge-blocking correctness.
  • You SHOULD keep network traffic narrow and query each external dependency only as much as needed for a meaningful result.
  • test_urls_reachable.py scans git-tracked plus untracked-but-not-ignored text files for literal public http:// and https:// URLs.
  • The URL reachability check skips template placeholders and reserved local or example hosts such as localhost and *.example.
  • The URL reachability check treats HTTP 401, 403, and 405 as reachable (auth-gated or method-restricted servers, not dead links).
  • The URL reachability check emits warning annotations for HTTP 429, 500, and 503 (rate-limited or temporarily unavailable) via utils.annotations.message.
  • The URL reachability check treats read and connect timeouts as warnings.
  • All other 4xx/5xx codes and connection errors (DNS, SSL) fail the test.

Suppression Comments 🚫

Lint and external checks share a single per-item suppression syntax based on # noqa / # nocheck markers. The full grammar, position semantics, and the catalog of rule keys live at suppression.md.

Running 🏃

Use the dedicated make target:

make test-external

To scope the run to one file:

make test-external TEST_PATTERN=test_image_versions.py

CI Workflow 🤖

The dedicated workflow lives at .github/workflows/test-code-external.yml. It MAY be triggered manually and is intentionally separate from the default make test flow. It is also wired into the orchestrated code-testing gate in CI so the live check can run there without changing local default validation.