Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Use dynamic data and secure secrets in your tests
.claude/skills/testdriverai-testdriver-variables/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-09 | ✗→✓ | ▲ Improved | -3% | 0% |
| case-01 | ✗→✓ | ▲ Improved | -32% | 0% |
| case-02 | ✗→✓ | ▲ Improved | -10% | 0% |
| case-03 | ✗→✓ | ▲ Improved | 7% | 0% |
| case-07 | ✗→✓ | ▲ Improved | -16% | 0% |
<!-- Generated from variables.mdx. DO NOT EDIT. -->
Scale your testing with dynamic data and secure secrets management. Choose the right approach based on your testing needs.
Environment variables are ideal for configuration that changes between environments (dev, staging, production) or for secrets that shouldn't be committed to code. Use this approach when you need to run the same tests against different servers or with different credentials.
javascriptimport { test } from 'vitest'; import { chrome } from 'testdriverai/presets'; test('multi-environment testing', async (context) => { const env = process.env.TEST_ENV || 'staging'; const urls = { dev: 'https://dev.myapp.com', staging: 'https://staging.myapp.com', production: 'https://myapp.com' }; const { testdriver } = await chrome(context, { url: urls[env] }); await testdriver.assert('app is running'); });
bash# Run against different environments TEST_ENV=dev vitest run TEST_ENV=staging vitest run TEST_ENV=production vitest run
Test fixtures work best when you have structured, reusable test data that needs to be shared across multiple tests. Use fixtures when testing different user roles, product catalogs, or any scenario where you want to parameterize tests with a known set of data.
javascript test/fixtures/users.jsexport const testUsers = [ { email: 'admin@test.com', role: 'admin' }, { email: 'user@test.com', role: 'user' }, { email: 'guest@test.com', role: 'guest' } ]; export const products = [ { name: 'Laptop', price: 999 }, { name: 'Mouse', price: 29 }, { name: 'Keyboard', price: 89 } ];
javascript test/permissions.test.jsimport { test } from 'vitest'; import { chrome } from 'testdriverai/presets'; import { testUsers } from './fixtures/users.js'; test.each(testUsers)('$role can access dashboard', async ({ email, role }, context) => { const { testdriver } = await chrome(context, { url }); await testdriver.find('email input').type(email); await testdriver.find('password input').type('password123'); await testdriver.find('login button').click(); if (role === 'admin') { await testdriver.assert('admin panel is visible'); } else { await testdriver.assert('user dashboard is visible'); } });
Dynamic data generation is perfect for creating unique test data on each run, avoiding conflicts with existing records, and testing edge cases with realistic data. Use libraries like Faker when you need fresh emails, names, or other data that won't collide with previous test runs.
javascriptimport { test } from 'vitest'; import { chrome } from 'testdriverai/presets'; import { faker } from '@faker-js/faker'; test('user registration with dynamic data', async (context) => { const { testdriver } = await chrome(context, { url }); // Generate unique test data for each run const userData = { firstName: faker.person.firstName(), lastName: faker.person.lastName(), email: faker.internet.email(), password: faker.internet.password({ length: 12 }) }; await testdriver.find('first name input').type(userData.firstName); await testdriver.find('last name input').type(userData.lastName); await testdriver.find('email input').type(userData.email); await testdriver.find('password input').type(userData.password); await testdriver.find('register button').click(); await testdriver.assert('registration successful'); console.log('Registered user:', userData.email); });
bashnpm install --save-dev @faker-js/faker
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-09 | fail→pass | 13,887 | 10,192 | -27% | 1 | 1 | 0% | 2,604 | 2,519 | -3% | 0 | 0 | — |
case-01 | fail→pass | 14,904 | 7,704 | -48% | 1 | 1 | 0% | 2,705 | 1,829 | -32% | 0 | 0 | — |
case-02 | fail→pass | 13,800 | 6,888 | -50% | 1 | 1 | 0% | 2,572 | 2,325 | -10% | 0 | 0 | — |
case-08 | pass→pass | 9,477 | 3,742 | -61% | 1 | 1 | 0% | 1,444 | 1,698 | +18% | 0 | 0 | — |
case-03 | fail→pass | 10,658 | 5,991 | -44% | 1 | 1 | 0% | 1,953 | 2,082 | +7% | 0 | 0 | — |
case-04 | pass→pass | 10,054 | 4,781 | -52% | 1 | 1 | 0% | 1,851 | 1,828 | -1% | 0 | 0 | — |
case-05 | pass→pass | 14,268 | 7,366 | -48% | 1 | 1 | 0% | 2,061 | 2,511 | +22% | 0 | 0 | — |
case-06 | pass→pass | 8,775 | 7,427 | -15% | 1 | 1 | 0% | 1,607 | 2,092 | +30% | 0 | 0 | — |
case-07 | fail→pass | 13,466 | 5,785 | -57% | 1 | 1 | 0% | 2,512 | 2,115 | -16% | 0 | 0 | — |
case-10 | fail→pass | 13,204 | 6,844 | -48% | 1 | 1 | 0% | 2,391 | 2,306 | -4% | 0 | 0 | — |
case-11 | pass→pass | 12,596 | 6,664 | -47% | 1 | 1 | 0% | 1,816 | 2,290 | +26% | 0 | 0 | — |
case-12 | fail→pass | 5,277 | 1,964 | -63% | 1 | 1 | 0% | 902 | 1,294 | +43% | 0 | 0 | — |
case-13 | pass→pass | 11,037 | 5,720 | -48% | 1 | 1 | 0% | 1,991 | 2,100 | +5% | 0 | 0 | — |
case-14 | fail→pass | 12,650 | 5,080 | -60% | 1 | 1 | 0% | 1,672 | 1,678 | +0% | 0 | 0 | — |
case-15 | pass→pass | 9,336 | 6,833 | -27% | 1 | 1 | 0% | 1,659 | 2,060 | +24% | 0 | 0 | — |
case-16 | fail→pass | 9,634 | 4,031 | -58% | 1 | 1 | 0% | 1,373 | 1,558 | +13% | 0 | 0 | — |
case-17 | pass→pass | 5,605 | 2,875 | -49% | 1 | 1 | 0% | 973 | 1,504 | +55% | 0 | 0 | — |
case-18 | pass→pass | 13,087 | 5,229 | -60% | 1 | 1 | 0% | 1,831 | 1,731 | -5% | 0 | 0 | — |
case-19 | fail→fail | 9,277 | 2,540 | -73% | 1 | 1 | 0% | 1,345 | 1,266 | -6% | 0 | 0 | — |
case-20 | fail→pass | 11,485 | 4,541 | -60% | 1 | 1 | 0% | 1,808 | 1,796 | -1% | 0 | 0 | — |
case-21 | fail→pass | 12,993 | 7,506 | -42% | 1 | 1 | 0% | 2,131 | 2,099 | -2% | 0 | 0 | — |
case-22 | fail→pass | 10,397 | 3,688 | -65% | 1 | 1 | 0% | 1,618 | 1,622 | +0% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted. The headline lift of +55 percentage points is the difference between those two pass rates over the 22 comparable cases. 1 case got worse with the skill loaded, and it is included in that figure.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.