Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →remotion-dev/writing-tests
Rules for writing and reviewing tests in the Remotion repository.
Passed 13 of 17 test cases with the skill, and 10 of 17 without it.
and 3 more in the full report
4 fixed13 passed either way5 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
This benchmark measured an earlier version of the skill.
A newer attempt on gemini-3.6-flash on 10/2/2026 did not produce a publishable comparison (not comparable). The snapshot above is the previously measured run.
One of 4 test cases the skill fixed. Same model, same prompt, with and without the skill.
We are writing tests for the Remotion Lambda site deployment orchestration pipeline in `packages/lambda`. Should we mock the internal bundle preparation step, or only mock the remote AWS service boundary?
| Checked | Without skill | With skill |
|---|---|---|
| Only the remote AWS provider boundary is mocked. | Missed | Passed |
| Internal bundle preparation steps use real owned code. | Missed | Passed |
The instructions your model reads when the skill loads.
Apply this skill when the user explicitly requests new tests or when existing tests, fixtures, or snapshots need updates for feature work, bug fixes, refactors, or PR preparation. Do not add new tests, including new test cases in existing files, unless the user explicitly requests them. A request to review tests does not authorize changing them.
The purpose of a test is confidence that a workflow or contract works for a Studio user, CLI user, package consumer, or renderer consumer.
Test the feature at the widest stable boundary that gives useful confidence. Prefer one complete integration workflow; use focused unit tests for genuinely complex logic and E2E tests for critical cross-process workflows.
Test count, assertion count, file count, and coverage percentage are not quality metrics. One rendered frame, final artifact, or persisted-source assertion can give more confidence than many narrow unit assertions.
A test that stays green when the feature is disconnected is not useful primary coverage. For a user-facing or cross-layer feature, a private helper unit test must not be the only coverage.
Answer these questions first:
Test type is determined by the boundary, not the directory or runner. A Bun test can be an integration test, while a Playwright test can still be coupled to implementation details.
Use these primary boundaries:
| Change | Preferred primary coverage |
|---|---|
| Studio interaction | Playwright action followed by a visible, source-file, persistence, URL, or rendered outcome |
| Studio Server or codemod | Realistic source fixture through the route or handler, followed by the resulting source or event |
| Core, Player, or Media | Mount or render the real component with real neighboring modules and representative media |
| Web renderer or visual effect | Browser render, image comparison, pixel probe, or produced frame |
| CLI | Execute the command and assert exit code, output, and filesystem effects |
| Renderer, SSR, templates, or packaging | Public API through packages/it-tests or a realistic temporary project |
| Lambda or cloud orchestration | Full orchestration with only the remote provider mocked or faked |
| Pure math, parser, serializer, protocol transformation, or state machine | Focused, preferably table-driven unit tests |
Do not force every change through Playwright. A route-level or filesystem integration is often the best stable boundary. Reserve full E2E tests for important risks that cross browser, server, filesystem, HMR, or process boundaries.
test cases passed, vs 10 / 17 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.