Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →tencent/skillhone-evaluation
Run and interpret skill evaluations.
Passed 16 of 18 test cases with the skill, and 7 of 18 without it.
and 10 more in the full report
11 fixed8 passed either way2 still failing1 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 10 test cases the skill fixed. Same model, same prompt, with and without the skill.
I am ready to evaluate a skill baseline during active iteration. I have my skill directory at /tmp/my-skill and eval repo at /tmp/my-eval. Show me the exact command line invocation to run a fast iteration probe and write the output into the standard data folder.
| Checked | Without skill | With skill |
|---|---|---|
| Includes `--split probe` in the evaluation command | Missed | Passed |
| Specifies `--output _data/probe_result.json` as the output path | Missed | Passed |
The instructions your model reads when the skill loads.
Run evaluations and diagnose results to inform improvement decisions.
The evaluator is not just a scorer. It is a harness that runs the skill against private tasks and leaves a structured evidence trail:
trajectory.jsonl, produced artifacts, and stderr-like
signals that explain failures the score cannot explain.This separation matters. A low score may come from weak skill instructions, but it may also come from missing files, tool crashes, invalid compiled artifacts, over-strict verifier rules, or infrastructure errors. Evaluation work is about mapping the failure to the correct harness layer.
On Forgejo-backed repos, status.py is a read-only context check before PR
validation or merge decisions:
python3 ~/.skillhone/skills/skillhone/scripts/status.py
# Probe — fast iteration signal
python3 ~/.skillhone/skills/skillhone/scripts/eval.py \
--skill-dir /path/to/skill --eval-dir /path/to/eval-repo \
--split probe --output _data/probe_result.json
# Test — final benchmark (NEVER during iteration)
test cases passed, vs 7 / 18 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.