Generate bounded independent candidates, score them against a frozen rubric, and verify the selected result with a proof log.
SkillScore91Maintained
On-demandOn-demand: invoked explicitly — never occupies your agent's context until called.
◆AI Engineer pack
Last verified 10/10/2026
What it changes
Passed 20 of 22 test cases with the skill, and 10 of 22 without it.
Without skill45%
With skill91%
Checks it passed that the model alone missed
Records UNKNOWN as the verification status.
Returns needs_contract before generating candidates.
Halts candidate generation and outputs needs_contract.
and 12 more in the full report
10 fixed10 passed either way2 still failing0 broke1 errored
Pass-rate lift
+45 pp
Tokens
-11%
Agent turns
0%
Response time
-58%
Model gemini-3.6-flashMeasured 10/10/2026v220 of 23 cases passed with skill · 22 comparable
Tokens, turns and time describe resource use, not a dollar price.
See it in action
One of 10 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 10
Prompt
We are refactoring our authentication token storage architecture across microservices. We have provided all contract details: repo git commit 8f1a2b, staging env, $50 compute budget, pass threshold 8.0/10, tie-breaker highest security score, and log path TESTING.md. How many alternative implementation candidates should be generated by default when no candidate count is specified in the prompt?
What was checked, without and with the skill
Checked
Without skill
With skill
Sets the default candidate count to 3.
Passed
Passed
Rejects generating 5 candidates without explicit cost justification.
Missed
Passed
case-07 · gemini-3.6-flash · 10/10/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md52 linessingle file
Best-of-N selection and proof
Use this skill when a task has several plausible solutions, a wrong choice is costly, and a deterministic check or independent reviewer can evaluate the selected result. Do not use it for mechanical work with one clear implementation and a direct acceptance test.
Read the full method at Best-of-N: Generate, Select, and Verify before running the protocol.
Inputs to collect before generation
Task scope, exclusions, repository revision, environment, permissions, and budget.
Acceptance criteria, mandatory failure conditions, and executable checks.
A rubric with weights, observable anchors, passing threshold, tie-breaker, candidate count or predeclared batch schedule, and stop rule.
The selected TESTING.md path. Start from the portable proof-log template.
If any item is missing, return needs_contract and list the missing fields. Do not generate candidates first and invent the rubric afterward.
Procedure
Freeze the contract in the proof log. Default to three candidates. Use five only when the expected improvement justifies the additional generation, scoring, and verification cost. If work runs in batches, declare every batch and the between-batch stop condition before generation.
Generate each candidate from the same frozen contract. Do not reveal candidate text, scores, or private reasoning across generators. Assign an opaque identifier to every generated candidate.
Preserve each candidate as a separate artifact. For code, use isolated diffs or worktrees from the same base revision. Add one proof-log line for every generated candidate, including every candidate in Best-of-5 and rejected candidates.
Blind provenance and presentation order for scoring when practical. Apply the fixed rubric to every candidate in the declared N, or to every candidate in the completed predeclared batch. Record criterion-level evidence and disqualify mandatory failures.
Select the highest passing candidate using the declared tie-breaker. Treat any combination of candidate fragments as a new synthesized candidate with its own ID, score, and verification.
Run the declared executable checks in the recorded environment. Capture commands, output location, exit status, artifact hash or revision, and uncovered scope.
If executable verification cannot decide the requirement, request a reviewer who did not generate the candidate and who receives a fresh task packet. Record shared model, context, tools, and repository access as correlation risks.
Finish the proof log with PASS, FAIL, or UNKNOWN. UNKNOWN blocks a claim that the requirement was verified.
Guardrails
Candidate generation is not selection. Selection is not synthesis. Majority vote is not evidence of correctness.
Never use a self-grading generator as the only acceptance gate.
Generate and score all candidates in the declared N before selection. A batched run may stop only after the complete predeclared batch is evaluated and its predeclared stop condition is met. Do not keep sampling until an answer feels persuasive.
Do not claim candidates are independent solely because they came from different calls. State the isolation controls and remaining shared context.
Preserve failed candidates and failed checks in the proof log. They bound what was actually tested.
Required output
Return this concise record and write the full details to TESTING.md: