Compare commits
5 Commits
cd32841847
...
c0f813b792
| Author | SHA1 | Date | |
|---|---|---|---|
| c0f813b792 | |||
| 8f231d49e7 | |||
| 545c213f88 | |||
| 04f24bb16d | |||
| 26585a86fc |
66
skills/plan/SKILL.md
Normal file
66
skills/plan/SKILL.md
Normal file
@@ -0,0 +1,66 @@
|
|||||||
|
---
|
||||||
|
name: plan
|
||||||
|
description: >
|
||||||
|
Turn a request into an objective-oriented plan and hand it to the robot skill
|
||||||
|
for autonomous execution. Trigger on /plan or when the operator asks for a
|
||||||
|
plan, a breakdown, or a strategy for a task before any work starts. Portable
|
||||||
|
across projects.
|
||||||
|
---
|
||||||
|
|
||||||
|
Produce a plan whose every item is an outcome with a check, then execute it
|
||||||
|
autonomously. A plan that lists activities instead of objectives cannot be
|
||||||
|
verified, and an unverifiable plan cannot be handed to a robot.
|
||||||
|
|
||||||
|
## Procedure
|
||||||
|
|
||||||
|
1. **Interview first.** Open every plan with the `active-listening` skill: ask
|
||||||
|
until scope, constraints, success criteria, and the operator-only facts are
|
||||||
|
all pinned, then reflect the understanding back. This is the only point in
|
||||||
|
the run where questions are allowed, so leave nothing open here.
|
||||||
|
2. **Pin the objective.** State the end state in one sentence, as a condition
|
||||||
|
that is either true or false, never as an activity. Name the command or
|
||||||
|
observation that proves it.
|
||||||
|
3. **Inspect before decomposing.** Read the code, config, tests, and history the
|
||||||
|
objective touches. Every plan item must rest on something you have seen, not
|
||||||
|
on an assumption about how the project works.
|
||||||
|
4. **Decompose into sub-objectives.** Each item gets: the outcome it reaches,
|
||||||
|
the verification that proves it, and the items it depends on. Split an item
|
||||||
|
whenever it needs more than one verification. Order by dependency and mark
|
||||||
|
the items that are independent, so they can run in parallel.
|
||||||
|
5. **Resolve the open decisions now.** Any root cause, design choice, or
|
||||||
|
trade-off the plan rests on gets settled before execution, escalating to the
|
||||||
|
`dialectic` skill where being wrong is expensive. The robot does not ask, so
|
||||||
|
an unresolved decision becomes a guess at runtime.
|
||||||
|
6. **Record the plan as a todo list.** Write the sub-objectives into the
|
||||||
|
harness's own todo tracking, one entry per item, phrased as the outcome. Keep
|
||||||
|
a plan file only when the operator asks for one.
|
||||||
|
7. **Present it and wait.** Show the objective, the ordered sub-objectives with
|
||||||
|
their verifications, what is deliberately out of scope, and the assumptions
|
||||||
|
the plan rests on. Stop here: the plan is the deliverable of this step.
|
||||||
|
8. **Implement it statically, in full.** On the operator's go, write every code,
|
||||||
|
config, test, and doc change the plan calls for, across all sub-objectives,
|
||||||
|
before running anything that needs live infrastructure. Verify statically:
|
||||||
|
read the diff, run the linters, the unit tests, and whatever dry-run or
|
||||||
|
syntax check the project offers. This step ends only when the whole plan
|
||||||
|
exists on disk, with no sub-objective left unwritten.
|
||||||
|
9. **Then iterate under `robot`.** With the static implementation complete,
|
||||||
|
invoke the `robot` skill with the objective as its goal for the dynamic part:
|
||||||
|
run, deploy, observe, fix, repeat until every verification passes. It drives
|
||||||
|
the loop without check-ins, verifies each outcome instead of assuming it, and
|
||||||
|
reports the items that fall outside its clearance for the operator to run.
|
||||||
|
|
||||||
|
## Rules
|
||||||
|
|
||||||
|
- One objective per item, in outcome form: "endpoint returns 200 for an expired
|
||||||
|
token", not "fix the auth middleware".
|
||||||
|
- No item without a verification. If you cannot name what proves it, the item is
|
||||||
|
still a wish, not a plan.
|
||||||
|
- Plan only what the request covers. Speculative future-proofing, refactors
|
||||||
|
nobody asked for, and abstractions with one caller belong in the out-of-scope
|
||||||
|
list, not in the plan.
|
||||||
|
- Static before dynamic: never start the robot loop on a half-written plan. A
|
||||||
|
deploy that fails on code you had not written yet costs a full cycle and
|
||||||
|
proves nothing.
|
||||||
|
- Re-plan on contradicted evidence: when execution disproves an assumption the
|
||||||
|
plan rests on, revise the plan instead of forcing the original steps through.
|
||||||
|
- Cite `file:line` for every claim about the current state of the code.
|
||||||
95
skills/test-fix/SKILL.md
Normal file
95
skills/test-fix/SKILL.md
Normal file
@@ -0,0 +1,95 @@
|
|||||||
|
---
|
||||||
|
name: test-fix
|
||||||
|
description: >
|
||||||
|
Run the project's Makefile test targets in a loop - run, diagnose, fix, re-run -
|
||||||
|
until every selected target is green or the loop is provably stuck. Trigger on
|
||||||
|
/test-fix or when the operator asks to make the tests pass, fix the failing
|
||||||
|
tests, or get the suite green. Portable across projects.
|
||||||
|
---
|
||||||
|
|
||||||
|
Same target discovery and selection as the `test` skill - invoke it for step 1
|
||||||
|
and step 2 rather than reimplementing the grep and the selection list. This
|
||||||
|
skill owns what happens after the first red result: the fix loop.
|
||||||
|
|
||||||
|
The contract is narrow: the code becomes correct, the tests stay honest.
|
||||||
|
|
||||||
|
## The loop
|
||||||
|
|
||||||
|
One iteration is: run -> read the failure -> one root cause -> one fix ->
|
||||||
|
re-run. Never batch several speculative fixes into one iteration; a green run
|
||||||
|
after three simultaneous changes proves nothing about which one mattered.
|
||||||
|
|
||||||
|
### Run
|
||||||
|
|
||||||
|
Run the failing target alone, not the whole selection - the fastest command
|
||||||
|
that reproduces the failure. Widen back to the full selection only when that
|
||||||
|
target is green.
|
||||||
|
|
||||||
|
### Diagnose
|
||||||
|
|
||||||
|
Read the actual error output before touching a file. Apply the `triage` skill
|
||||||
|
to reach a verified root cause; for a failure whose cause is genuinely unclear
|
||||||
|
after one look, apply `dialectic` rather than guessing twice.
|
||||||
|
|
||||||
|
Decide explicitly which side is wrong, and say so in one line before editing:
|
||||||
|
|
||||||
|
- **Code is wrong**: the test states the intended behaviour. Fix the code.
|
||||||
|
- **Test is wrong**: the test encodes an outdated or incorrect expectation.
|
||||||
|
Fixing it is allowed *only* with a stated reason for why the old expectation
|
||||||
|
was wrong - never because the code disagrees with it.
|
||||||
|
- **Environment is wrong**: missing dependency, stale container, absent env var,
|
||||||
|
unbuilt artefact. Fix the environment or report it; do not patch code around
|
||||||
|
a broken environment.
|
||||||
|
|
||||||
|
### Fix
|
||||||
|
|
||||||
|
Smallest change that addresses the root cause. Follow the project's existing
|
||||||
|
idioms and the `no-defaults` and `comments-clean` rules.
|
||||||
|
|
||||||
|
Forbidden, in every iteration, without explicit operator approval:
|
||||||
|
|
||||||
|
- deleting, skipping, `xfail`-ing, or commenting out a failing test
|
||||||
|
- loosening an assertion to whatever the code currently produces
|
||||||
|
- catching or swallowing the exception the test was written to surface
|
||||||
|
- adding retries, sleeps, or reruns to paper over a flaky failure
|
||||||
|
- disabling a linter rule, a type check, or a test target from the selection
|
||||||
|
|
||||||
|
Any of these is a report item, not a fix. A suite that is green because the
|
||||||
|
failing test no longer runs is a regression disguised as success.
|
||||||
|
|
||||||
|
### Re-run
|
||||||
|
|
||||||
|
Re-run the same target. Then, once it passes, re-run every target that was
|
||||||
|
already green - a fix that breaks a neighbouring suite is not a fix. Only
|
||||||
|
after the full selection is green is the loop done.
|
||||||
|
|
||||||
|
## Stopping
|
||||||
|
|
||||||
|
Stop and report, without asking for permission to stop:
|
||||||
|
|
||||||
|
- **Green**: full selection passes. Report and stop.
|
||||||
|
- **No progress**: the same failure survives 3 iterations, or the failure count
|
||||||
|
stops falling across 3 iterations. Report the root cause reached so far, what
|
||||||
|
was tried, and why each attempt failed.
|
||||||
|
- **Fix exceeds the mandate**: the real fix is an API change, a dependency bump,
|
||||||
|
a schema migration, or a redesign. Name it, show the minimal diff it would
|
||||||
|
need, and let the operator decide.
|
||||||
|
- **Oscillation**: fixing A re-breaks B and vice versa. Report both with the
|
||||||
|
conflict between them - that is a design problem, not a test problem.
|
||||||
|
- **Flaky**: a target passes and fails without any change between runs. Report
|
||||||
|
it as flaky with both outputs; never "fix" it by rerunning until green.
|
||||||
|
|
||||||
|
Never loop silently. Emit one line per iteration: target, failure, hypothesis,
|
||||||
|
change made.
|
||||||
|
|
||||||
|
## Report
|
||||||
|
|
||||||
|
- Per iteration: what failed, the root cause, the fix, the result.
|
||||||
|
- Final state of every selected target, pass or fail, with the verbatim output
|
||||||
|
of anything still failing.
|
||||||
|
- Everything deliberately not done: tests judged wrong but left alone,
|
||||||
|
environment issues, out-of-mandate fixes.
|
||||||
|
- The exact command reproducing the final state.
|
||||||
|
|
||||||
|
State calibrated confidence in the fixes per the `confidence` skill. Do not
|
||||||
|
commit unless the operator asked for a commit.
|
||||||
77
skills/test/SKILL.md
Normal file
77
skills/test/SKILL.md
Normal file
@@ -0,0 +1,77 @@
|
|||||||
|
---
|
||||||
|
name: test
|
||||||
|
description: >
|
||||||
|
Discover the test targets of the project's Makefile, let the operator pick
|
||||||
|
which ones to run, then run them and report. Trigger on /test or when the
|
||||||
|
operator asks to run the tests, the test suite, or a specific test target
|
||||||
|
without naming the exact command. Portable across projects.
|
||||||
|
---
|
||||||
|
|
||||||
|
The Makefile is the source of truth for how this project runs tests. Never
|
||||||
|
invent a command (`pytest`, `npm test`, `go test`) while a Makefile target
|
||||||
|
exists - the target carries the project's env vars, build dependencies and
|
||||||
|
container setup.
|
||||||
|
|
||||||
|
## Step 1: find the targets
|
||||||
|
|
||||||
|
From the repository root:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
grep -nE '^test[A-Za-z0-9_.-]*:' Makefile
|
||||||
|
```
|
||||||
|
|
||||||
|
Read the matched rules plus their prerequisites, so the selection list can say
|
||||||
|
what each one actually does (delegated script, container build, sub-targets).
|
||||||
|
|
||||||
|
Edge cases, handled explicitly rather than guessed around:
|
||||||
|
|
||||||
|
- **No Makefile**: say so in one line, name the test runner the project does
|
||||||
|
use (from `pyproject.toml`, `package.json`, `tox.ini`, CI workflow), and ask
|
||||||
|
before running anything.
|
||||||
|
- **No `test*` target**: report the targets that do exist and stop.
|
||||||
|
- **Included makefiles** (`include foo.mk`): grep those too.
|
||||||
|
- **Aggregate targets**: a target whose recipe is only other test targets (e.g.
|
||||||
|
`test: test-unit test-integration`) is the "run everything" entry - mark it as
|
||||||
|
such in the list, do not expand it into its parts silently.
|
||||||
|
|
||||||
|
## Step 2: offer the selection
|
||||||
|
|
||||||
|
Ask with `AskUserQuestion`, `multiSelect: true`, one question. Options are the
|
||||||
|
discovered targets, each with a one-line description of what it runs. Order the
|
||||||
|
aggregate/full target first and label it as the complete suite.
|
||||||
|
|
||||||
|
The tool takes at most 4 options. With more targets than that:
|
||||||
|
|
||||||
|
- print the **complete** discovered list as text first, one line per target, so
|
||||||
|
nothing is hidden, then
|
||||||
|
- offer the 4 most useful entries (aggregate first, then the ones matching the
|
||||||
|
operator's stated intent or the files currently uncommitted), and note that
|
||||||
|
"Other" accepts any target name from the printed list.
|
||||||
|
|
||||||
|
Never silently truncate. If the operator already named a target in their
|
||||||
|
request, skip the question and run it.
|
||||||
|
|
||||||
|
## Step 3: run
|
||||||
|
|
||||||
|
Run the selected targets in the order the operator listed them, one `make`
|
||||||
|
invocation per target, each as its own command so a failure is attributable:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make <target>
|
||||||
|
```
|
||||||
|
|
||||||
|
Do not add `-k`, do not reorder, do not substitute a faster equivalent. If a
|
||||||
|
target needs a long timeout (container builds, e2e), set it on the Bash call
|
||||||
|
rather than backgrounding blindly.
|
||||||
|
|
||||||
|
Stop after the first failing target unless the operator asked for all of them -
|
||||||
|
a later suite running against a broken build produces noise, not information.
|
||||||
|
|
||||||
|
## Step 4: report
|
||||||
|
|
||||||
|
Per target: pass/fail plus the failing test names and the exact error output,
|
||||||
|
quoted verbatim. Never paraphrase a failure. Then one line with the exact
|
||||||
|
command to reproduce the failure alone.
|
||||||
|
|
||||||
|
Do not fix what failed unless the operator asks - report first. If the operator
|
||||||
|
does ask for a fix, apply the `triage` skill to it.
|
||||||
@@ -17,20 +17,33 @@ root cause is proven.
|
|||||||
whose conclusion is failure, cancelled, or timed-out. A downstream aggregate
|
whose conclusion is failure, cancelled, or timed-out. A downstream aggregate
|
||||||
job that fails only because an upstream job did is not a separate root cause —
|
job that fails only because an upstream job did is not a separate root cause —
|
||||||
note it and move on.
|
note it and move on.
|
||||||
2. **Dialectic per failing job.** For EACH failing job, invoke the `dialectic`
|
2. **Cluster the failures by error message.** Group the failing jobs by their
|
||||||
skill: form a thesis about the root cause from evidence (the job log, its
|
actual failing line, not by job name. Jobs whose messages differ in anything
|
||||||
artifacts, the code at the run's commit, git history), attack it with
|
but the interpolated identifiers (host, app, port, path) belong in separate
|
||||||
independent skeptics, and iterate to a ~99% thesis. The jobs are independent,
|
clusters.
|
||||||
so run their investigations in parallel where the tooling allows.
|
3. **Download artifacts per cluster.** For each cluster, fetch every artifact of
|
||||||
3. **Distinguish shared vs hidden causes.** When several jobs share one root
|
one representative job: reports, rescue diagnostics, inventories, container
|
||||||
cause, fix it once. When a job hides a second failure behind the first, keep
|
logs. Pull a second member's artifacts only when the representative's
|
||||||
going until the job is actually green, not just past the first error.
|
evidence does not explain the whole cluster. The job log usually names an
|
||||||
4. **Fix at the root.** Apply the real fix in the repository for each proven root
|
artifact path and nothing more; the assertion text, the server-side
|
||||||
|
exception and the state dumps are inside the artifact. Do this even when the
|
||||||
|
log looks conclusive, because a log that explains the symptom rarely
|
||||||
|
explains the cause.
|
||||||
|
4. **Dialectic per cluster.** For EACH cluster, invoke the `dialectic` skill:
|
||||||
|
form a thesis about the root cause from evidence (the job log, its artifacts,
|
||||||
|
the code at the run's commit, git history), attack it with independent
|
||||||
|
skeptics, and iterate to a ~99% thesis. The clusters are independent, so run
|
||||||
|
their investigations in parallel where the tooling allows.
|
||||||
|
5. **Distinguish shared vs hidden causes.** A cluster is a hypothesis, not a
|
||||||
|
proof: if the dialectic shows one cluster splitting into two causes, split it.
|
||||||
|
When a job hides a second failure behind the first, keep going until the job
|
||||||
|
is actually green, not just past the first error.
|
||||||
|
6. **Fix at the root.** Apply the real fix in the repository for each proven root
|
||||||
cause. Never mask a failure — no retry-until-pass, no disabling the check, no
|
cause. Never mask a failure — no retry-until-pass, no disabling the check, no
|
||||||
soft-skip. If a failure is genuinely external (upstream outage, flaky infra),
|
soft-skip. If a failure is genuinely external (upstream outage, flaky infra),
|
||||||
confirm that with evidence and surface it honestly instead of fixing around
|
confirm that with evidence and surface it honestly instead of fixing around
|
||||||
it.
|
it.
|
||||||
5. **Follow the run.** While the run is still in progress, re-check it
|
7. **Follow the run.** While the run is still in progress, re-check it
|
||||||
periodically and triage each newly failed job as it appears. The run is done
|
periodically and triage each newly failed job as it appears. The run is done
|
||||||
only when it has finished and every failure has a verified fix.
|
only when it has finished and every failure has a verified fix.
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user