mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-06 02:54:26 +00:00
kevinveenbirkenbachandClaude Opus 5
71888d86af
The workflow directory mixed entry points, reusable stages and scheduled jobs into one flat list sorted by topic, so the first question about any file - what starts it - was the one the name did not answer. Every workflow now carries one of three prefixes, derived from its on: block: cron- for a schedule trigger, call- for one only reachable through workflow_call, entry- for everything a repository event or a human starts. Scheduled workflows additionally carry an alarm clock as the first character of their name. Two lints keep it that way. One derives the expected prefix from the triggers and compares it against the file name; it flagged all 17 offenders before the rename and none after. The other resolves every .github/workflows/<file> string in the repository against the directory, which is what a rename silently breaks in Markdown links, Python constants and shell API URLs alike. Renaming pulled 69 pre-existing comments into the comment lint's changed-file scope. The vendored CodeQL template got the nocheck marker its scorecard sibling already carried, two workflow headers moved above name: where they count as file headers, and the remaining banners were deleted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
73 lines
3.0 KiB
Markdown
73 lines
3.0 KiB
Markdown
# External Tests 🌐
|
|
|
|
External tests depend on live third-party systems such as Docker Hub,
|
|
GHCR, or other external services. Run them with `make test-external`.
|
|
|
|
## Goal 🎯
|
|
|
|
- You MUST use external tests only when the behavior depends on a live third-party
|
|
system and a hermetic unit, lint, or integration test would not verify the
|
|
real condition.
|
|
- You MUST keep external tests outside the default `make test` flow so normal
|
|
validation stays reproducible without third-party network access.
|
|
- You SHOULD keep external tests focused, advisory, and easy to run on demand.
|
|
|
|
## Location 📁
|
|
|
|
- External tests MUST live under `tests/external/`.
|
|
- You SHOULD group them by dependency family such as `tests/external/update/`
|
|
or `tests/external/github/` when more than one test exists.
|
|
- Every external test directory MUST contain `__init__.py` so repository-wide
|
|
test structure checks keep passing.
|
|
|
|
## Requirements 📋
|
|
|
|
- You MUST document in the test file why the check is external and what live
|
|
dependency it contacts.
|
|
- You MUST NOT move an external test into `tests/lint/`, `tests/unit/`, or
|
|
`tests/integration/` just for convenience when it still depends on live
|
|
third-party state.
|
|
- External tests MAY emit warnings instead of hard failures when the goal is
|
|
advisory freshness reporting rather than merge-blocking correctness.
|
|
- You SHOULD keep network traffic narrow and query each external dependency only
|
|
as much as needed for a meaningful result.
|
|
- [test_urls_reachable.py](../../../../tests/external/repository/test_urls_reachable.py)
|
|
scans git-tracked plus untracked-but-not-ignored text files for literal
|
|
public `http://` and `https://` URLs.
|
|
- The URL reachability check skips template placeholders and reserved local or
|
|
example hosts such as `localhost` and `*.example`.
|
|
- The URL reachability check treats HTTP `401`, `403`, and `405` as reachable
|
|
(auth-gated or method-restricted servers, not dead links).
|
|
- The URL reachability check emits warning annotations for HTTP `429`, `500`,
|
|
and `503` (rate-limited or temporarily unavailable) via
|
|
[`utils.annotations.message`](../../../../utils/annotations/message.py).
|
|
- The URL reachability check treats read and connect timeouts as warnings.
|
|
- All other `4xx`/`5xx` codes and connection errors (DNS, SSL) fail the test.
|
|
|
|
## Suppression Comments 🚫
|
|
|
|
Lint and external checks share a single per-item suppression syntax based on
|
|
`# noqa` / `# nocheck` markers. The full grammar, position semantics, and the
|
|
catalog of rule keys live at [suppression.md](suppression.md).
|
|
|
|
## Running 🏃
|
|
|
|
Use the dedicated make target:
|
|
|
|
```bash
|
|
make test-external
|
|
```
|
|
|
|
To scope the run to one file:
|
|
|
|
```bash
|
|
make test-external INFINITO_TEST_PATTERN=test_image_versions.py
|
|
```
|
|
|
|
## CI Workflow 🤖
|
|
|
|
External tests run inside `.github/workflows/call-test.yml` via `make test`.
|
|
It MAY be triggered manually and is intentionally separate from the default
|
|
`make test` flow. It is also wired into the orchestrated code-testing gate in
|
|
CI so the live check can run there without changing local default validation.
|