Apply ONLY the spec clauses literally — no aggregation, dedup, or cleanup transforms; each input record maps to its own output line.
NewMaintained
Automatic
◆Coding pack
Last verified 10/6/2026
What it changes
Passed 21 of 24 test cases with the skill, and 21 of 24 without it.
Without skill88%
With skill88%
Checks it passed that the model alone missed
The response does not contain "keyboard"
The response does not contain "sensor_A"
The response contains two separate lines with "GET /api/v1/users -> 500"
3 fixed19 passed either way0 still failing3 broke
Pass-rate lift
0 pp
Tokens
+51%
Agent turns
0%
Response time
+45%
Model gemini-3.6-flashMeasured 10/6/2026v122 of 25 cases passed with skill · 24 comparable
Tokens, turns and time describe resource use, not a dollar price.
See it in action
One of 3 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 3
Prompt
Process `batch_2024_01.json` containing the following records:
`[{"product": "keyboard", "quantity": 7}, {"product": "keyboard", "quantity": 8}, {"product": "mouse", "quantity": 12}]`
Specification requirement: Filter input records to those where quantity is at least 10. Format each passing record as `product: quantity`.
What was checked, without and with the skill
Checked
Without skill
With skill
The response does not contain "keyboard"
Missed
Passed
The response contains "mouse: 12"
Passed
Passed
case-01 · gemini-3.6-flash · 10/6/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md29 linessingle file
Spec Literal Execution — no unstated transforms
Problem
An exact-match grader rejected output.txt (score 0.0) although the agent
believed it followed every spec clause and even re-read the file.
Root cause
Spec says "one line per product with quantity > 0" and never mentions
aggregation, yet the agent inferred "product = unique product name" and
summed duplicate rows (cherry 20+2 → 22, elderberry 11+6 → 17). The grader
treats each JSON object as its own product row: CHERRY;20 and CHERRY;2
must be separate lines. The added transformation — not the arithmetic or
tooling — caused the failure.
Fix
Apply ONLY the clauses literally present in the spec. Each input record
maps to its own output line unless the spec explicitly says otherwise.
Never add aggregation, dedup, rounding, or case changes "for cleanliness".
If a spec filter (e.g. qty >= 10) is the only thing removing duplicates,
apply it per row — do not aggregate first.
Evidence
FAIL: spec-format2-1 (0.0). PASS: spec-format1-1, spec-format3-1 (1.0) —
same duplicate-shaped data; the passing runs filtered per row with no
aggregation.