Compare commits
5 Commits
cd32841847
...
c0f813b792
| Author | SHA1 | Date | |
|---|---|---|---|
| c0f813b792 | |||
| 8f231d49e7 | |||
| 545c213f88 | |||
| 04f24bb16d | |||
| 26585a86fc |
66
skills/plan/SKILL.md
Normal file
66
skills/plan/SKILL.md
Normal file
@@ -0,0 +1,66 @@
|
||||
---
|
||||
name: plan
|
||||
description: >
|
||||
Turn a request into an objective-oriented plan and hand it to the robot skill
|
||||
for autonomous execution. Trigger on /plan or when the operator asks for a
|
||||
plan, a breakdown, or a strategy for a task before any work starts. Portable
|
||||
across projects.
|
||||
---
|
||||
|
||||
Produce a plan whose every item is an outcome with a check, then execute it
|
||||
autonomously. A plan that lists activities instead of objectives cannot be
|
||||
verified, and an unverifiable plan cannot be handed to a robot.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. **Interview first.** Open every plan with the `active-listening` skill: ask
|
||||
until scope, constraints, success criteria, and the operator-only facts are
|
||||
all pinned, then reflect the understanding back. This is the only point in
|
||||
the run where questions are allowed, so leave nothing open here.
|
||||
2. **Pin the objective.** State the end state in one sentence, as a condition
|
||||
that is either true or false, never as an activity. Name the command or
|
||||
observation that proves it.
|
||||
3. **Inspect before decomposing.** Read the code, config, tests, and history the
|
||||
objective touches. Every plan item must rest on something you have seen, not
|
||||
on an assumption about how the project works.
|
||||
4. **Decompose into sub-objectives.** Each item gets: the outcome it reaches,
|
||||
the verification that proves it, and the items it depends on. Split an item
|
||||
whenever it needs more than one verification. Order by dependency and mark
|
||||
the items that are independent, so they can run in parallel.
|
||||
5. **Resolve the open decisions now.** Any root cause, design choice, or
|
||||
trade-off the plan rests on gets settled before execution, escalating to the
|
||||
`dialectic` skill where being wrong is expensive. The robot does not ask, so
|
||||
an unresolved decision becomes a guess at runtime.
|
||||
6. **Record the plan as a todo list.** Write the sub-objectives into the
|
||||
harness's own todo tracking, one entry per item, phrased as the outcome. Keep
|
||||
a plan file only when the operator asks for one.
|
||||
7. **Present it and wait.** Show the objective, the ordered sub-objectives with
|
||||
their verifications, what is deliberately out of scope, and the assumptions
|
||||
the plan rests on. Stop here: the plan is the deliverable of this step.
|
||||
8. **Implement it statically, in full.** On the operator's go, write every code,
|
||||
config, test, and doc change the plan calls for, across all sub-objectives,
|
||||
before running anything that needs live infrastructure. Verify statically:
|
||||
read the diff, run the linters, the unit tests, and whatever dry-run or
|
||||
syntax check the project offers. This step ends only when the whole plan
|
||||
exists on disk, with no sub-objective left unwritten.
|
||||
9. **Then iterate under `robot`.** With the static implementation complete,
|
||||
invoke the `robot` skill with the objective as its goal for the dynamic part:
|
||||
run, deploy, observe, fix, repeat until every verification passes. It drives
|
||||
the loop without check-ins, verifies each outcome instead of assuming it, and
|
||||
reports the items that fall outside its clearance for the operator to run.
|
||||
|
||||
## Rules
|
||||
|
||||
- One objective per item, in outcome form: "endpoint returns 200 for an expired
|
||||
token", not "fix the auth middleware".
|
||||
- No item without a verification. If you cannot name what proves it, the item is
|
||||
still a wish, not a plan.
|
||||
- Plan only what the request covers. Speculative future-proofing, refactors
|
||||
nobody asked for, and abstractions with one caller belong in the out-of-scope
|
||||
list, not in the plan.
|
||||
- Static before dynamic: never start the robot loop on a half-written plan. A
|
||||
deploy that fails on code you had not written yet costs a full cycle and
|
||||
proves nothing.
|
||||
- Re-plan on contradicted evidence: when execution disproves an assumption the
|
||||
plan rests on, revise the plan instead of forcing the original steps through.
|
||||
- Cite `file:line` for every claim about the current state of the code.
|
||||
95
skills/test-fix/SKILL.md
Normal file
95
skills/test-fix/SKILL.md
Normal file
@@ -0,0 +1,95 @@
|
||||
---
|
||||
name: test-fix
|
||||
description: >
|
||||
Run the project's Makefile test targets in a loop - run, diagnose, fix, re-run -
|
||||
until every selected target is green or the loop is provably stuck. Trigger on
|
||||
/test-fix or when the operator asks to make the tests pass, fix the failing
|
||||
tests, or get the suite green. Portable across projects.
|
||||
---
|
||||
|
||||
Same target discovery and selection as the `test` skill - invoke it for step 1
|
||||
and step 2 rather than reimplementing the grep and the selection list. This
|
||||
skill owns what happens after the first red result: the fix loop.
|
||||
|
||||
The contract is narrow: the code becomes correct, the tests stay honest.
|
||||
|
||||
## The loop
|
||||
|
||||
One iteration is: run -> read the failure -> one root cause -> one fix ->
|
||||
re-run. Never batch several speculative fixes into one iteration; a green run
|
||||
after three simultaneous changes proves nothing about which one mattered.
|
||||
|
||||
### Run
|
||||
|
||||
Run the failing target alone, not the whole selection - the fastest command
|
||||
that reproduces the failure. Widen back to the full selection only when that
|
||||
target is green.
|
||||
|
||||
### Diagnose
|
||||
|
||||
Read the actual error output before touching a file. Apply the `triage` skill
|
||||
to reach a verified root cause; for a failure whose cause is genuinely unclear
|
||||
after one look, apply `dialectic` rather than guessing twice.
|
||||
|
||||
Decide explicitly which side is wrong, and say so in one line before editing:
|
||||
|
||||
- **Code is wrong**: the test states the intended behaviour. Fix the code.
|
||||
- **Test is wrong**: the test encodes an outdated or incorrect expectation.
|
||||
Fixing it is allowed *only* with a stated reason for why the old expectation
|
||||
was wrong - never because the code disagrees with it.
|
||||
- **Environment is wrong**: missing dependency, stale container, absent env var,
|
||||
unbuilt artefact. Fix the environment or report it; do not patch code around
|
||||
a broken environment.
|
||||
|
||||
### Fix
|
||||
|
||||
Smallest change that addresses the root cause. Follow the project's existing
|
||||
idioms and the `no-defaults` and `comments-clean` rules.
|
||||
|
||||
Forbidden, in every iteration, without explicit operator approval:
|
||||
|
||||
- deleting, skipping, `xfail`-ing, or commenting out a failing test
|
||||
- loosening an assertion to whatever the code currently produces
|
||||
- catching or swallowing the exception the test was written to surface
|
||||
- adding retries, sleeps, or reruns to paper over a flaky failure
|
||||
- disabling a linter rule, a type check, or a test target from the selection
|
||||
|
||||
Any of these is a report item, not a fix. A suite that is green because the
|
||||
failing test no longer runs is a regression disguised as success.
|
||||
|
||||
### Re-run
|
||||
|
||||
Re-run the same target. Then, once it passes, re-run every target that was
|
||||
already green - a fix that breaks a neighbouring suite is not a fix. Only
|
||||
after the full selection is green is the loop done.
|
||||
|
||||
## Stopping
|
||||
|
||||
Stop and report, without asking for permission to stop:
|
||||
|
||||
- **Green**: full selection passes. Report and stop.
|
||||
- **No progress**: the same failure survives 3 iterations, or the failure count
|
||||
stops falling across 3 iterations. Report the root cause reached so far, what
|
||||
was tried, and why each attempt failed.
|
||||
- **Fix exceeds the mandate**: the real fix is an API change, a dependency bump,
|
||||
a schema migration, or a redesign. Name it, show the minimal diff it would
|
||||
need, and let the operator decide.
|
||||
- **Oscillation**: fixing A re-breaks B and vice versa. Report both with the
|
||||
conflict between them - that is a design problem, not a test problem.
|
||||
- **Flaky**: a target passes and fails without any change between runs. Report
|
||||
it as flaky with both outputs; never "fix" it by rerunning until green.
|
||||
|
||||
Never loop silently. Emit one line per iteration: target, failure, hypothesis,
|
||||
change made.
|
||||
|
||||
## Report
|
||||
|
||||
- Per iteration: what failed, the root cause, the fix, the result.
|
||||
- Final state of every selected target, pass or fail, with the verbatim output
|
||||
of anything still failing.
|
||||
- Everything deliberately not done: tests judged wrong but left alone,
|
||||
environment issues, out-of-mandate fixes.
|
||||
- The exact command reproducing the final state.
|
||||
|
||||
State calibrated confidence in the fixes per the `confidence` skill. Do not
|
||||
commit unless the operator asked for a commit.
|
||||
77
skills/test/SKILL.md
Normal file
77
skills/test/SKILL.md
Normal file
@@ -0,0 +1,77 @@
|
||||
---
|
||||
name: test
|
||||
description: >
|
||||
Discover the test targets of the project's Makefile, let the operator pick
|
||||
which ones to run, then run them and report. Trigger on /test or when the
|
||||
operator asks to run the tests, the test suite, or a specific test target
|
||||
without naming the exact command. Portable across projects.
|
||||
---
|
||||
|
||||
The Makefile is the source of truth for how this project runs tests. Never
|
||||
invent a command (`pytest`, `npm test`, `go test`) while a Makefile target
|
||||
exists - the target carries the project's env vars, build dependencies and
|
||||
container setup.
|
||||
|
||||
## Step 1: find the targets
|
||||
|
||||
From the repository root:
|
||||
|
||||
```bash
|
||||
grep -nE '^test[A-Za-z0-9_.-]*:' Makefile
|
||||
```
|
||||
|
||||
Read the matched rules plus their prerequisites, so the selection list can say
|
||||
what each one actually does (delegated script, container build, sub-targets).
|
||||
|
||||
Edge cases, handled explicitly rather than guessed around:
|
||||
|
||||
- **No Makefile**: say so in one line, name the test runner the project does
|
||||
use (from `pyproject.toml`, `package.json`, `tox.ini`, CI workflow), and ask
|
||||
before running anything.
|
||||
- **No `test*` target**: report the targets that do exist and stop.
|
||||
- **Included makefiles** (`include foo.mk`): grep those too.
|
||||
- **Aggregate targets**: a target whose recipe is only other test targets (e.g.
|
||||
`test: test-unit test-integration`) is the "run everything" entry - mark it as
|
||||
such in the list, do not expand it into its parts silently.
|
||||
|
||||
## Step 2: offer the selection
|
||||
|
||||
Ask with `AskUserQuestion`, `multiSelect: true`, one question. Options are the
|
||||
discovered targets, each with a one-line description of what it runs. Order the
|
||||
aggregate/full target first and label it as the complete suite.
|
||||
|
||||
The tool takes at most 4 options. With more targets than that:
|
||||
|
||||
- print the **complete** discovered list as text first, one line per target, so
|
||||
nothing is hidden, then
|
||||
- offer the 4 most useful entries (aggregate first, then the ones matching the
|
||||
operator's stated intent or the files currently uncommitted), and note that
|
||||
"Other" accepts any target name from the printed list.
|
||||
|
||||
Never silently truncate. If the operator already named a target in their
|
||||
request, skip the question and run it.
|
||||
|
||||
## Step 3: run
|
||||
|
||||
Run the selected targets in the order the operator listed them, one `make`
|
||||
invocation per target, each as its own command so a failure is attributable:
|
||||
|
||||
```bash
|
||||
make <target>
|
||||
```
|
||||
|
||||
Do not add `-k`, do not reorder, do not substitute a faster equivalent. If a
|
||||
target needs a long timeout (container builds, e2e), set it on the Bash call
|
||||
rather than backgrounding blindly.
|
||||
|
||||
Stop after the first failing target unless the operator asked for all of them -
|
||||
a later suite running against a broken build produces noise, not information.
|
||||
|
||||
## Step 4: report
|
||||
|
||||
Per target: pass/fail plus the failing test names and the exact error output,
|
||||
quoted verbatim. Never paraphrase a failure. Then one line with the exact
|
||||
command to reproduce the failure alone.
|
||||
|
||||
Do not fix what failed unless the operator asks - report first. If the operator
|
||||
does ask for a fix, apply the `triage` skill to it.
|
||||
@@ -17,20 +17,33 @@ root cause is proven.
|
||||
whose conclusion is failure, cancelled, or timed-out. A downstream aggregate
|
||||
job that fails only because an upstream job did is not a separate root cause —
|
||||
note it and move on.
|
||||
2. **Dialectic per failing job.** For EACH failing job, invoke the `dialectic`
|
||||
skill: form a thesis about the root cause from evidence (the job log, its
|
||||
artifacts, the code at the run's commit, git history), attack it with
|
||||
independent skeptics, and iterate to a ~99% thesis. The jobs are independent,
|
||||
so run their investigations in parallel where the tooling allows.
|
||||
3. **Distinguish shared vs hidden causes.** When several jobs share one root
|
||||
cause, fix it once. When a job hides a second failure behind the first, keep
|
||||
going until the job is actually green, not just past the first error.
|
||||
4. **Fix at the root.** Apply the real fix in the repository for each proven root
|
||||
2. **Cluster the failures by error message.** Group the failing jobs by their
|
||||
actual failing line, not by job name. Jobs whose messages differ in anything
|
||||
but the interpolated identifiers (host, app, port, path) belong in separate
|
||||
clusters.
|
||||
3. **Download artifacts per cluster.** For each cluster, fetch every artifact of
|
||||
one representative job: reports, rescue diagnostics, inventories, container
|
||||
logs. Pull a second member's artifacts only when the representative's
|
||||
evidence does not explain the whole cluster. The job log usually names an
|
||||
artifact path and nothing more; the assertion text, the server-side
|
||||
exception and the state dumps are inside the artifact. Do this even when the
|
||||
log looks conclusive, because a log that explains the symptom rarely
|
||||
explains the cause.
|
||||
4. **Dialectic per cluster.** For EACH cluster, invoke the `dialectic` skill:
|
||||
form a thesis about the root cause from evidence (the job log, its artifacts,
|
||||
the code at the run's commit, git history), attack it with independent
|
||||
skeptics, and iterate to a ~99% thesis. The clusters are independent, so run
|
||||
their investigations in parallel where the tooling allows.
|
||||
5. **Distinguish shared vs hidden causes.** A cluster is a hypothesis, not a
|
||||
proof: if the dialectic shows one cluster splitting into two causes, split it.
|
||||
When a job hides a second failure behind the first, keep going until the job
|
||||
is actually green, not just past the first error.
|
||||
6. **Fix at the root.** Apply the real fix in the repository for each proven root
|
||||
cause. Never mask a failure — no retry-until-pass, no disabling the check, no
|
||||
soft-skip. If a failure is genuinely external (upstream outage, flaky infra),
|
||||
confirm that with evidence and surface it honestly instead of fixing around
|
||||
it.
|
||||
5. **Follow the run.** While the run is still in progress, re-check it
|
||||
7. **Follow the run.** While the run is still in progress, re-check it
|
||||
periodically and triage each newly failed job as it appears. The run is done
|
||||
only when it has finished and every failure has a verified fix.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user