Set up an end-to-end test suite in any repo, following practices that make e2e a reliable per-PR gate: real flows over bypass, layered assertions, a reusable auth/session helper, video+trace evidence, and a compounding suite.
SkillScore95Maintained
Automatic
◆QA & Test Engineering pack
Last verified 8/15/2026
What it changes
Passed 21 of 22 test cases with the skill, and 17 of 22 without it.
Without skill77%
With skill95%
Checks it passed that the model alone missed
Third-party webhooks must be received via local forwarding rather than mocked stubs.
Drive authentic user paths through the app interface rather than database insertion bypasses.
The test suite must run against an already-running application stack without starting app servers itself.
and 2 more in the full report
4 fixed17 passed either way1 still failing0 broke
Pass-rate lift
+18 pp
Tokens
+22%
Agent turns
0%
Response time
-15%
Model gemini-3.6-flashMeasured 8/15/2026v121 of 22 cases passed with skill
Tokens, turns and time describe resource use, not a dollar price.
See it in action
One of 4 test cases the skill fixed. Same model, same prompt, with and without the skill.
1 of 4
Prompt
Our team needs to establish an automated browser and API testing gate for pull requests. Can you give us a comprehensive framework setup plan that outlines where tests should reside, how to manage test state and auth efficiently without slowing down runs, selector best practices, and how to safely handle third-party webhooks?
What was checked, without and with the skill
Checked
Without skill
With skill
System e2e tests must be placed in a dedicated top-level directory rather than within individual application subfolders.
Passed
Passed
Authentication state should be saved once or minted via service helper and loaded across specs rather than logging in on every test.
Passed
Passed
Third-party webhooks must be received via local forwarding rather than mocked stubs.
Missed
Passed
case-02 · gemini-3.6-flash · 8/15/2026 · answers are capped and redacted
What's inside
The instructions your model reads when the skill loads.
SKILL.md57 linessingle file
Set up an e2e test suite
E2e tests verify the whole running system through the app (browser/API), not one
module. They are the per-PR gate. Pairs with dev-local-setup (a reproducible
local stack) and verifier-setup (which scaffolds the repo's /verify skill —
the verify→ship loop this gate feeds into).
Where it lives
Unit/integration tests stay inside each app/package — they own one module.
System e2e is a dedicated top-level package (e.g. e2e/) — it spans all
apps, so it belongs to none. Add it to the workspace if a monorepo.
The recipe
Stand the app up reproducibly — see dev-local-setup. The e2e suite never
boots the app itself; it runs against the already-running stack. That stack can
be local (dev-local-setup) or an isolated cloud box (crabbox-setup) —
same specs, run against either. For parallel agents use the cloud box (one
laptop can't host concurrent stacks).
Pick the framework that fits (Playwright for browser; your HTTP client for API).
Turn on video + trace — the recording is the proof, and it's gitignored output.
Explore the flow live first (don't guess selectors), then crystallize it into
a committed spec.
Keep the gate small: a handful of critical journeys, deterministic. Each new
feature PR adds its spec — the suite compounds.
Practices that make e2e trustworthy
Real flow, not bypass. Drive the genuine path. For email codes / OTP, read the
real code from a local mail server (Mailpit / Inbucket / MailHog) — never hardcode
a fixed test code. That's what makes it a test, not a rehearsal.
Verify auth ITSELF once; bypass it everywhere else. A dedicated signup/login
spec proves auth works. Every other spec shouldn't re-pay the login tax — build
a session helper that mints an authed state once (real flow → saved storage
state, or a service-role/token mint) and load it.
Layered assertions: client → server → product. Don't stop at "the UI changed."
Confirm the server agrees (token validates / row/state is right) AND the
user-visible outcome (e.g. plan upgraded and credits granted).
Stable selectors. Prefer role/label/text; add a small data-testid in the
component when there's no good handle — never a brittle CSS path.
Fresh data per run. Unique emails/ids so reruns don't collide; mind rate
limits (auth email, etc.).