Prompt engineering frameworks for building, testing, versioning, and evaluating prompts: chain-of-thought, few-shot, regression testing, and rubrics.
SkillScore74Maintained
Automatic
◆AI Engineer pack
Last verified 8/20/2026
What it changes
Passed 16 of 22 test cases with the skill, and 3 of 22 without it.
Without skill13%
With skill74%
Checks it passed that the model alone missed
The command includes the --json flag.
The command includes the --show-diff flag.
The command includes the --fail-under 0.80 flag.
and 16 more in the full report
14 fixed3 passed either way6 still failing0 broke
Pass-rate lift
+61 pp
Tokens
+33%
Agent turns
0%
Response time
-31%
Model gemini-3.6-flashMeasured 8/20/2026v117 of 23 cases passed with skill · 22 comparable
Tokens, turns and time describe resource use, not a dollar price.
See it in action
One of 14 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 14
Prompt
Which reference file path documents failure mode catalog entries and the troubleshooting matrix?
What was checked, without and with the skill
Checked
Without skill
With skill
The response specifies references/failure-modes-and-workflows.md.
Missed
Passed
case-13 · gemini-3.6-flash · 8/20/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md71 lines7 filesincludes scripts
Prompt Engineer Toolkit — Production Prompt Engineering
The complete lifecycle for production prompts: design patterns that work, testing frameworks that catch regressions, versioning systems that track changes, and evaluation rubrics that replace subjective "looks good" with measurable quality. This treats prompts as production code with the same rigor — not clever tricks.
Testing framework — test-case structure, suite composition (40/30/15/15), a 5-dimension automated scoring rubric with a weighted formula, and a regression-testing protocol.
Versioning — version-control layout, changelog format with rationale/baselines/rollback, and a prompt-diff risk checklist.
Failure-mode catalog — instruction override, format drift, sycophancy, verbosity, hallucination, anchoring, lost-in-the-middle, each with fixes.
Lifecycle workflows — design a prompt, debug a degraded prompt, migrate a prompt to a new model.
When to Use
Designing production prompts or building a prompt library.
Running A/B tests or regression tests on prompt variants.
Versioning prompts and gating changes on test scores.
Debugging a degraded prompt or migrating prompts across models.
Clarify First
Before designing or testing the prompt, confirm these inputs. If any is unknown or vague, ASK — do not assume:
Task & definition of "good" — what the prompt must produce and how success is judged (drives the 5-dimension evaluation rubric)
Target model — calibration (temperature, few-shot count) and migration paths differ by model
Lifecycle stage — design new / debug a degraded prompt / migrate to a new model (selects the workflow)
Stop rule: ask only the 2-3 that most change the output. If the user says "just draft it," proceed and list your assumptions at the top of the artifact.
Tools
Tool
Purpose
Command
eval_scorer.py
Score evaluation results from JSON test cases (exact/contains/regex)