Use when the user has nothing — no traces, no labels, no eval set — and needs to build a v0 evaluation from scratch. Also use when the user says "I need to start evaluating my app but don't know where to begin," "I want to set up eval for a new product," or has just identified failure modes and needs to turn them into principles. Outputs a v0 grader in 30 minutes using OpenJudge SimpleRubricsGenerator, plus a roadmap to reach calibrated evaluation.
SkillScore100Maintained
Automatic
◆AI Engineer pack
Last verified 8/14/2026
What it changes
Passed 22 of 22 test cases with the skill, and 5 of 22 without it.
Without skill23%
With skill100%
Checks it passed that the model alone missed
Asks for examples of perfect outputs
Asks for system purpose or target audience
Specifies collecting 50 human-labeled examples
and 30 more in the full report
17 fixed5 passed either way0 still failing0 broke
Pass-rate lift
+77 pp
Tokens
+24%
Agent turns
0%
Response time
-38%
Model gemini-3.6-flashMeasured 8/14/2026v122 of 22 cases passed with skill
Tokens, turns and time describe resource use, not a dollar price.
See it in action
One of 17 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 17
Prompt
Create a python script that synthesizes 30 test inputs for an AI financial advice summarizer cold-start evaluation using OpenJudge schema.
What was checked, without and with the skill
Checked
Without skill
With skill
Allocates exactly 18 inputs to typical or common scenarios
Missed
Passed
Allocates exactly 9 inputs to boundary or ambiguous scenarios
Missed
Passed
Allocates exactly 3 inputs to edge cases or unusual scenarios
Missed
Passed
case-05 · gemini-3.6-flash · 8/14/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md212 linessingle file
Bootstrap
Cold-start an evaluation system when you have nothing. In 30 minutes you get a working
v0 grader and a clear path to a calibrated, trustworthy evaluation.
Requires OpenJudge (pip install py-openjudge) for the grader generators
(SimpleRubricsGenerator / IterativeRubricsGenerator). The interview, stratification,
and calibration-roadmap methodology is SDK-independent.
Checklist
You MUST create a task for each item and complete them in order:
Understand the product — one-shot interview, not question-by-question
Generate v0 grader — use OpenJudge SimpleRubricsGenerator
Synthesize eval inputs — 30 inputs with 60/30/10 stratification
Run v0 evaluation — GradingRunner with the generated grader
Output roadmap — exactly how to reach 50 labels → calibrate
Step 1: Product Interview (One Shot)
Ask the user to describe their system in one go:
To bootstrap your evaluation, I need to understand what you're building.
Please describe (all at once):
- What does your system do? Who uses it?
- What are 3 examples of perfect outputs?
- What are 3 things the system must never do?
- What failures worry you most?
Don't drip-feed these questions. One prompt, one answer. If the user provides a spec