0 Fork 0

mirror of https://github.com/kevinveenbirkenbach/computer-playbook.git synced 2026-11-08 03:38:23 +00:00

Files

T

kevinveenbirkenbachandClaude Opus 4.8 5a223f407e refactor(ci): consolidate lint and test workflows into make-backed pairs

Replace the ten lint-*.yml workflows and the four test-code-*.yml
workflows with lint.yml (make lint fanning out every check, plus the
hadolint SARIF job) and test.yml (make test running unit, integration,
lint and external suites in parallel on the runner host). Drop the
lint-gate and test-code-gate jobs; code-quality-gate now needs lint,
test and security-codeql directly. Dependency diagram, workflows table
and pipeline docs follow.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-07-18 19:53:20 +02:00

3.0 KiB

Raw Blame History

External Tests 🌐

External tests depend on live third-party systems such as Docker Hub, GHCR, or other external services. Run them with make test-external.

Goal 🎯

You MUST use external tests only when the behavior depends on a live third-party system and a hermetic unit, lint, or integration test would not verify the real condition.
You MUST keep external tests outside the default make test flow so normal validation stays reproducible without third-party network access.
You SHOULD keep external tests focused, advisory, and easy to run on demand.

Location 📁

External tests MUST live under tests/external/.
You SHOULD group them by dependency family such as tests/external/update/ or tests/external/github/ when more than one test exists.
Every external test directory MUST contain __init__.py so repository-wide test structure checks keep passing.

Requirements 📋

You MUST document in the test file why the check is external and what live dependency it contacts.
You MUST NOT move an external test into tests/lint/, tests/unit/, or tests/integration/ just for convenience when it still depends on live third-party state.
External tests MAY emit warnings instead of hard failures when the goal is advisory freshness reporting rather than merge-blocking correctness.
You SHOULD keep network traffic narrow and query each external dependency only as much as needed for a meaningful result.
test_urls_reachable.py scans git-tracked plus untracked-but-not-ignored text files for literal public http:// and https:// URLs.
The URL reachability check skips template placeholders and reserved local or example hosts such as localhost and *.example.
The URL reachability check treats HTTP 401, 403, and 405 as reachable (auth-gated or method-restricted servers, not dead links).
The URL reachability check emits warning annotations for HTTP 429, 500, and 503 (rate-limited or temporarily unavailable) via utils.annotations.message.
The URL reachability check treats read and connect timeouts as warnings.
All other 4xx/5xx codes and connection errors (DNS, SSL) fail the test.

Suppression Comments 🚫

Lint and external checks share a single per-item suppression syntax based on # noqa / # nocheck markers. The full grammar, position semantics, and the catalog of rule keys live at suppression.md.

Running 🏃

Use the dedicated make target:

make test-external

To scope the run to one file:

make test-external INFINITO_TEST_PATTERN=test_image_versions.py

CI Workflow 🤖

External tests run inside .github/workflows/test.yml via make test. It MAY be triggered manually and is intentionally separate from the default make test flow. It is also wired into the orchestrated code-testing gate in CI so the live check can run there without changing local default validation.

3.0 KiB Raw Blame History