Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Reliably test your most difficult user flows
.claude/skills/testdriverai-testdriver-what-is-testdriver/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-02 | ✗→✓ | ▲ Improved | -28% | 0% |
| case-03 | ✗→✓ | ▲ Improved | -22% | 0% |
| case-05 | ✗→✓ | ▲ Improved | -33% | 0% |
| case-06 | ✗→✓ | ▲ Improved | -42% | 0% |
| case-07 | ✗→✓ | ▲ Improved | -18% | 0% |
<!-- Generated from what-is-testdriver.mdx. DO NOT EDIT. -->
Modern testing tools like Playwright are designed to test a single web application, running in a single browser tab using selectors.
However, selectors are often either unreliable or unavailable in complex scenarios, leading to brittle and flaky tests:
| Challenge | Problem | Examples | |-----------|---------|----------| | Fast moving teams | Frequently change UI structure, breaking CSS/XPath selectors | Agile teams, startups, vibe-coders | | Dynamic content | Cannot be targeted with selectors | AI chatbots, PDFs, images, videos | | Software you don't own | May lack proper accessibility attributes | Other websites, extensions, third-party applications | | Multi-application workflows | Cannot be tested with web-only tools | Desktop apps, browser extensions, IDEs | | Visual states | Impossible to verify with code-based selectors | Charts, graphs, videos, images, spelling errors, UI layout |
TestDriver is a complete testing platform built specifically for handling these scenarios. It consists of a Javascript SDK, hosted infrastructure, and debugging tools that make it easy to write, run, and maintain tests for your most difficult user flows.
Here is an example of a TestDriver test that installs a production Chrome extension from the Chrome Web Store and verifies that it appears in the extensions menu:
javascript Installing Loom from the Chrome Web Storeimport { describe, expect, it } from "vitest"; import { TestDriver } from "testdriverai/vitest/hooks"; describe("Chrome Extension Test", () => { const testdriver = TestDriver(context); // Launch Chrome with Loom loaded by its Chrome Web Store ID await testdriver.provision.chromeExtension({ extensionId: 'liecbddmkiiihnedobmlmillhodjkdmb' }); // Click on the extensions button (puzzle piece icon) in Chrome toolbar const extensionsButton = await testdriver.find("The puzzle-shaped icon in the Chrome toolbar."); await extensionsButton.click(); // Look for Loom in the extensions menu const loomExtension = await testdriver.find("Loom extension in the extensions dropdown"); expect(loomExtension.found()).toBeTruthy(); });
<Tip>vitest is the preferred test runner for TestDriver.</Tip>
,
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-01 | fail→fail | 17,157 | 25,000 | +46% | 1 | 1 | 0% | 2,562 | 2,300 | -10% | 0 | 0 | — |
case-02 | fail→pass | 12,530 | 5,874 | -53% | 1 | 1 | 0% | 2,056 | 1,486 | -28% | 0 | 0 | — |
case-03 | fail→pass | 17,438 | 8,862 | -49% | 1 | 1 | 0% | 2,622 | 2,035 | -22% | 0 | 0 | — |
case-04 | pass→pass | 13,947 | 2,083 | -85% | 1 | 1 | 0% | 2,111 | 855 | -59% | 0 | 0 | — |
case-05 | fail→pass | 11,432 | 3,154 | -72% | 1 | 1 | 0% | 1,630 | 1,090 | -33% | 0 | 0 | — |
case-06 | fail→pass | 7,480 | 1,545 | -79% | 1 | 1 | 0% | 1,283 | 742 | -42% | 0 | 0 | — |
case-07 | fail→pass | 7,357 | 2,144 | -71% | 1 | 1 | 0% | 1,098 | 903 | -18% | 0 | 0 | — |
case-08 | fail→pass | 12,845 | 2,328 | -82% | 1 | 1 | 0% | 2,109 | 956 | -55% | 0 | 0 | — |
case-09 | fail→pass | 18,955 | 3,092 | -84% | 1 | 1 | 0% | 2,469 | 930 | -62% | 0 | 0 | — |
case-10 | pass→pass | 17,200 | 8,160 | -53% | 1 | 1 | 0% | 2,368 | 1,828 | -23% | 0 | 0 | — |
case-11 | fail→pass | 12,020 | 4,095 | -66% | 1 | 1 | 0% | 1,703 | 1,154 | -32% | 0 | 0 | — |
case-12 | fail→pass | 16,067 | 4,267 | -73% | 1 | 1 | 0% | 2,299 | 1,126 | -51% | 0 | 0 | — |
case-13 | pass→pass | 17,664 | 2,836 | -84% | 1 | 1 | 0% | 2,565 | 1,040 | -59% | 0 | 0 | — |
case-14 | fail→pass | 19,387 | 5,987 | -69% | 1 | 1 | 0% | 3,357 | 1,541 | -54% | 0 | 0 | — |
case-15 | fail→pass | 12,439 | 1,898 | -85% | 1 | 1 | 0% | 2,098 | 835 | -60% | 0 | 0 | — |
case-16 | pass→pass | 15,959 | 2,476 | -84% | 1 | 1 | 0% | 2,093 | 954 | -54% | 0 | 0 | — |
case-17 | fail→pass | 7,616 | 2,804 | -63% | 1 | 1 | 0% | 1,417 | 840 | -41% | 0 | 0 | — |
case-18 | pass→pass | 7,996 | 1,839 | -77% | 1 | 1 | 0% | 1,323 | 846 | -36% | 0 | 0 | — |
case-19 | pass→pass | 6,839 | 4,078 | -40% | 1 | 1 | 0% | 1,167 | 1,080 | -7% | 0 | 0 | — |
case-20 | pass→pass | 6,919 | 5,158 | -25% | 1 | 1 | 0% | 1,340 | 1,491 | +11% | 0 | 0 | — |
case-21 | pass→pass | 9,456 | 4,675 | -51% | 1 | 1 | 0% | 1,823 | 1,399 | -23% | 0 | 0 | — |
case-22 | pass→pass | 7,956 | 4,887 | -39% | 1 | 1 | 0% | 1,421 | 1,255 | -12% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted. The headline lift of +55 percentage points is the difference between those two pass rates over the 22 comparable cases.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.