Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →harness-engineering
This skill should be used when designing autonomous agent harnesses: research loops, evaluation scaffolds, locked and editable surfaces, durable logs, novelty gates, pruning, rollback, PR preparation, and human approval boundaries.
Passed 6 of 22 test cases with the skill, and 0 of 22 without it.
and 7 more in the full report
6 fixed0 passed either way16 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 6 test cases the skill fixed. Same model, same prompt, with and without the skill.
An engineer has finished writing a new automated experiment harness. What verification step should be executed before deploying it to production tasks?
| Checked | Without skill | With skill |
|---|---|---|
| Validate the harness against one known good artifact and one known bad artifact. | Missed | Passed |
The instructions your model reads when the skill loads.
Harness engineering designs the control system around an agent: what it may edit, how it receives feedback, where it writes state, how failures recover, and who can approve irreversible actions. The harness is the difference between a helpful agent session and an autonomous loop that can run for days without corrupting its objective.
Activate this skill when:
Do not activate this skill for adjacent work owned by other skills:
evaluation.tool-design.project-development.hosted-agents.Separate the agent from the environment it operates inside. The agent proposes actions; the harness defines allowed surfaces, feedback, persistence, and promotion rules.
Use four surface classes:
| Surface | Examples | Rule |
|---|---|---|
| Locked | Eval metric, rubric, validation script, merge policy | Agent may read and propose changes, but cannot score itself with modified rules |
| Editable | Skill draft, experiment file, prompt, config under test | Agent may mutate during the loop |
| Append-only | Results log, research thread, rejected ideas | Agent may append, not rewrite |
| Human-controlled | Merge, production deploy, credentials, destructive operations | Requires explicit human approval |
Autonomy works when feedback is fast, unambiguous, and hard to game. Karpathy's autoresearch is the minimal pattern: one editable file, one locked evaluation file, fixed wall-clock budget, one scalar metric, git rollback, and a durable results log. The lesson is not that every harness needs one metric; it is that ambiguous feedback creates ambiguous autonomy.
test cases passed, vs 0 / 22 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.