When you can run it, run it. Prefer executing the real path — the one-shot command, the actual agent loop, the failing test now passing, the app in front of you — over inspecting the code that would run. "Looks right" and "the code clearly does X" are smells to verify, not conclusions.
SkillScore86Maintained
Automatic
◆Coding pack
Last verified 8/3/2026
What it changes
Passed 18 of 21 test cases with the skill, and 17 of 21 without it.
Without skill82%
With skill86%
Checks it passed that the model alone missed
Inspects output files in `dist/` to confirm artifact generation.
1 fixed18 passed either way3 still failing0 broke
Pass-rate lift
+5 pp
Tokens
-17%
Agent turns
0%
Response time
-37%
Model gemini-3.6-flashMeasured 8/3/2026v119 of 22 cases passed with skill · 21 comparable
Tokens, turns and time describe resource use, not a dollar price.
See it in action
A test case the skill fixed. Same model, same prompt, with and without the skill.
Prompt
Running `npm run build` returned exit code 0. Should we check the contents of `dist/` before finalizing the build task?
What was checked, without and with the skill
Checked
Without skill
With skill
Inspects output files in `dist/` to confirm artifact generation.
Missed
Passed
case-15 · gemini-3.6-flash · 8/3/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md27 linessingle file
Run the Real Path
Reading code tells you what should happen. Running it tells you what does. When both are available, running wins — and it's usually cheaper than the paragraph you'd write justifying the inspection.
The smell
These thoughts mean stop and run it:
"The code clearly does X."
"This is wired correctly."
"It should work now."
"Looks right."
Your 100% certainty in unfamiliar territory is a cue to check, not a green light. The "looks right" signal is exactly where inspection fails and execution catches it.
How to apply
Find the realest entry point you can run. The one-shot that exercises the whole user path beats any proxy: the agent loop over a mock, the CLI over a unit test, the deployed URL over localhost-only.
Example:vanta run "audit stripe.com" runs the real persona + tools + provider in one shot — that's the Done test, so run that, not the direct tool call.
Run it. Read the actual output. Not the exit code alone — the content. Did it return the graded report, or just not crash?
If you genuinely cannot run it (no env, no creds, destructive), say so explicitly and mark the claim ◐ code-path in the ledger ([[vd-claim-ledger]]) — never silently upgrade it to ✅.
Re-run after the last edit. A green from before your final change proves nothing about the code that ships.
Don't fake the run
A mock that returns success, a test that asserts nothing, a happy-path stub — these look like execution and prove less than honest inspection. If you stub it, the claim is ○ assumed, not ✅.
Governed by [[vd-done-is-executed]]. Pairs with [[vd-plumbing-vs-behavior]].