Methodology for minimizing test count while maximizing behavioral coverage - behavior definition, anti-pattern catalog, consolidation patterns, stopping criterion, coverage-preserving validation
SkillScore83Maintained
On-demandOn-demand: invoked explicitly — never occupies your agent's context until called.
◆QA & Test Engineering pack
Last verified 8/16/2026
What it changes
Passed 17 of 21 test cases with the skill, and 5 of 21 without it.
Without skill26%
With skill83%
Checks it passed that the model alone missed
Declines to author test runner harness plugins.
Calculates the maximum test budget as 10 unit tests.
Collapses per-item test cases into a single set-difference test.
and 18 more in the full report
13 fixed6 passed either way4 still failing0 broke
Pass-rate lift
+57 pp
Tokens
+213%
Agent turns
0%
Response time
-41%
Model gemini-3.6-flashMeasured 8/16/2026v119 of 23 cases passed with skill · 21 comparable
Tokens, turns and time describe resource use, not a dollar price.
See it in action
One of 12 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 12
Prompt
A domain component exposes 3 driving ports with 5 distinct observable outcomes. The current test file contains 18 unit tests. What does the stopping criterion procedure indicate for this suite?
What was checked, without and with the skill
Checked
Without skill
With skill
Calculates the maximum test budget as 10 unit tests.
Missed
Passed
Triggers test consolidation because actual test count exceeds budget.
Missed
Passed
case-19 · gemini-3.6-flash · 8/16/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md414 linessingle file
Test Optimization Methodology
Mission
Minimize tests, maximize value, reduce feedback time, maintain quality.
(Ale, 2026-04-28: "Bisogna minimizzare i test, massimizzare il valore per ridurre il tempo di feedback, mantenendo la qualità.")
This skill operationalizes that mission. Apply during DELIVER COMMIT, scheduled audits, or /nw-optimize-tests invocations.
1. Behavior Definition (the loose joint, closed)
The phrase "distinct behavior" in the test budget formula max_unit_tests = 2 × distinct_behaviors is the loose joint that lets test counts inflate. Close it with these rules.
1.1 What IS a behavior
A behavior is an observable outcome via a port (driving or driven):
A return value from a driving-port call given specific inputs
A state change visible through a driving-port query after an action
A side effect at a driven-port boundary (call sequence + payload)
An exception raised from a driving port
A business invariant that holds across a class of inputs
1.2 What is NOT a behavior
Phrase presence in markdown — that is one document contract, not one behavior per phrase
AST shape (try/except wrapping, decorator presence, import order) — that is source structure, not runtime behavior
Type system facts (subclass relations, attribute presence) — Python language guarantees, not domain behaviors
Internal data shape (dataclass field assignment, dict structure) — language guarantees
Internal method calls (mock.assert_called_with) — implementation, not outcome
Source-shape compliance — runtime behavior on each supported environment IS the behavior, not the source it was compiled from
1.3 Counting rules — concrete cases
Surface
Behaviors
Markdown skill with required phrases
1 (the file conforms to the contract)
5 skill files × 30 required phrases each
5 (one per file), NOT 150
Function with N input variations, same assertion shape
1 (parametrize the variations)
Function with N error types, distinct messages and paths