Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →tencent/skillhone-synthesis
Use this skill to synthesize closed-form, automatically verifiable benchmark Q/A by exploring a tool environment, building a reusable exploration graph, and mining multiple hard questions from that graph. Use for: building a benchmark, writing eval items, generating evaluation data, closed-form QA, verifiable-answer datasets, synthesising eval data. Applies to any domain with callable tools.
Passed 17 of 19 test cases with the skill, and 7 of 19 without it.
and 9 more in the full report
12 fixed8 passed either way2 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 10 test cases the skill fixed. Same model, same prompt, with and without the skill.
Where in the project directory should seed exploration evidence and reasoning paths be stored during dataset synthesis?
| Checked | Without skill | With skill |
|---|---|---|
| Designates graph artifacts to be stored under the directory path path graph/<seed_id>/ | Missed | Passed |
The instructions your model reads when the skill loads.
This skill builds a generic closed-form Q/A dataset from real tool use.
Closed-form means each question has one intended answer, and the answer can be checked by code: exact match, regex, numeric tolerance, enum, or JSON field checks. The final benchmark should not require a human or an LLM judge for basic grading.
The central idea is simple:
The skill is domain-agnostic. Concrete task files, tool wrappers, and domain examples live outside this skill.
The task description is the source of truth for both the solver prompt and the verifier. When a task spec uses words such as "must", "must not", "required", "forbidden", "output format", "validation", "quality", or "acceptance", treat those clauses as benchmark requirements.
For every requirement in the task spec that can be checked mechanically, the
generated verifier should include a corresponding scores key. This includes
artifact constraints such as file names, raw-vs-wrapped output, syntax validity,
compile/render success, required fields, banned strings, shape/count limits,
style tokens, fixed palettes, local-only dependencies, and any other observable
property of the submitted answer.
Do not reduce the verifier to only the gold answer. If the task says the output must have a particular structure or quality, that structure or quality should be scored directly whenever it is observable from the answer or from local helper tools. Subjective requirements may be approximated by deterministic proxies, but clearly subjective-only preferences should not be silently converted into a pass.
Ask for these if they are missing:
test cases passed, vs 7 / 19 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.