---
name: jpvarbed/goal-spec
source: https://app.decimal.ai/s/jpvarbed-goal-spec@1/SKILL.md
source_sha256: 68afcb039291
---

# goal-spec — compile a verifiable /goal brief

**Verifiability is the #1 determinant of whether a dispatched agent finishes.** A `/goal`
launch only goes as well as the spec + rubric you hand it. This skill turns a rough ask into
a launch-ready brief so the agent can run to completion unattended — and so you stop
hand-assembling context every time.

> Working model: every turn that *isn't* a `/goal` should be building the brief for the next
> one. This skill is that build. The output is a paste-ready brief, not the work itself.

## Step 1 — Pass the verifiability gate (do this first)
State the done-condition as a **binary check** a human or script could evaluate with no
judgment call.
- Can't state it cleanly → the unit is too big or too fuzzy. **Decompose** until each unit
  has a binary done-check, then spec each unit separately.
- "Looks good / is better / handles edge cases" are not checks. "Beats the 90% holdout
  baseline", "exit 0 + artifact at PATH", "all N rows reconcile" are.

## Step 1b — Blindspot pass (map vs territory)
The brief is a **map**; the agent works in the **territory** (real codebase, live constraints,
the human's actual intent). The gap between them is unknowns — where a dispatched agent guesses.
Surface all three non-obvious quadrants BEFORE emitting the brief, one move each:
- **Known unknowns** → turn into interview questions, one at a time, prioritizing the ones whose
  answer would change the architecture; write the answers INTO the brief (references over prose).
- **Unknown knowns** (obvious-to-the-human but unstated) → show 3–4 candidate directions to react
  to (`grilling` / a throwaway `prototype`) instead of asking in the abstract.
- **Unknown unknowns** → a dedicated blindspot list: "what would make this run wrong that neither
  of us has said out loud" — likely edge cases, adjacent systems, unstated invariants.
More work per run ⇒ more unknowns ⇒ more places to guess wrong: a long blindspot list is a
decomposition signal (back to Step 1). Also tell the dispatched agent to keep a **deviations
log** — an unknown hit mid-build may mean the problem should be solved a different way; route
it back to the plan, don't bury it.

## Step 2 — Context access (give the keys, not the data)
List what the agent must reach AND the tool to reach it. Prefer live access over pasted dumps.
1. Reference material: markdown of prior experiments/decisions/numbers; the relevant docs.
2. Tools/CLIs it will drive, each with its auth source (e.g. training CLI, `langsmith-cli`
   for traces, `linear`/`gh`/`bws`).
3. Output location (repo / branch / dir) and any fixtures (the train + holdout sets).

Bad: paste three logs. Good: "traces live in LangSmith — read with `langsmith-cli runs …`".

## Step 3 — Verification plan (how it knows it's *working*, not just *done*)
1. **Primary signal** — the metric that moves ("number go up") and how to measure it.
2. **Hill-climb set** — the pre-split train + holdout it iterates against (never tune on holdout).
3. **Number to beat** — an explicit baseline (e.g. "break 90%, the GLM-5.2 baseline").
4. **Reading intermediate signal** — how to inspect rollouts/traces to choose the next move.
5. **Human hints** — known good moves + failure modes (length penalty if turns balloon,
   prompt-eng first, and what reward-hacking looks like for *this* task).

## Step 4 — Binary rubric (the done-checklist the agent self-checks before returning)
Every item PASS/FAIL, no vibes. Cover three axes: **outcome**, **integrity**, **reproducibility**.
Example (fine-tuning run):
- [ ] Beat the 90% holdout baseline.
- [ ] Checked traces for reward hacking, and reported what was found.
- [ ] Experiment logged in LangSmith; reproducible with a single command + a write-up.

## Step 5 — Adversarial pre-launch pass (don't skip)
Red-team the brief with `adversarial-review` before dispatch:
- **Can the agent pass the rubric without doing the work?** (reward-hacking the checks) → tighten,
  or add an integrity check it can't spoof.
- Is every rubric item binary AND verifiable with the context you provided?
- Is anything in the verification plan unmeasurable with the tools listed? → add the tool or cut the check.

## Step 6 — Emit the brief
Output one paste-ready block, then hand it to `/goal`:
```
GOAL: <one verifiable outcome>
CONTEXT: <access list — tools (+auth) + refs + output location + fixtures>
EFFORT: <high | medium | low — high for design/interface/data-model or irreversible calls; low for
  mechanical/verified-downstream work. Set the dispatched agent's reasoning tier by leverage, not
  size. Orchestrators launch high (Fable xhigh). See docs/effort-policy.md.>
VERIFY: <primary signal · hill-climb set · number-to-beat · how to read signal · hints>
RESOLVED (do not reopen): <decisions already settled — tech choices, scope cuts. A reviewer/agent
  that reopens one is out of scope. Pass this to `review-council --resolved` so gates don't re-argue it.>
RUBRIC (binary):
  - [ ] <outcome check>
  - [ ] <integrity check>
  - [ ] <reproducibility check>
DONE = all rubric checks PASS.
```

## Errors / failure modes
| Issue | Fix |
|---|---|
| Done-condition isn't binary | Decompose until each unit has a script/metric-checkable done-check. |
| No baseline to beat | Establish one first (measure current / the holdout) — a number-to-beat is required, not optional. |
| Rubric is game-able (reward-hackable) | Add an integrity check (inspect traces/artifacts) or make the metric harder to spoof. |
| Context pasted, not accessible | Replace dumps with the tool + location the agent can query live. |
| Verification tool/CLI missing or unauthed | Provide it (or its `bws` key) before launch — if the agent can't verify, don't dispatch. |
| Task too large for one /goal | Split into units, each its own brief; sequence or dispatch in parallel. |