Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →saffron-health/writing-tests
Write and edit test files that model real user flows.
Passed 18 of 22 test cases with the skill, and 10 of 22 without it.
and 10 more in the full report
8 fixed10 passed either way4 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 8 test cases the skill fixed. Same model, same prompt, with and without the skill.
We are writing tests for an encryption library `crypto-vault`. Should our test call internal state machine methods like `_transitionToState('KEY_EXCHANGE')` and `_deriveEphemeralKey()`, or should it only call `vault.encrypt(data)` and `vault.decrypt(data)`?
| Checked | Without skill | With skill |
|---|---|---|
| The response specifies calling the public API methods encrypt and decrypt. | Passed | Passed |
| The response advises against calling internal state machine methods directly. | Missed | Passed |
The instructions your model reads when the skill loads.
Each test should represent a single, concrete flow that a real user would follow when interacting with the system. Write tests at the same abstraction level as the user experiences the system.
The test should call the system the same way a user would. If the system is a CLI, the test should invoke the CLI with argument strings. If it's a library, the test should call the public API. Don't wrap the interface in test-specific helpers that hide what's actually happening.
Good — the test reads like what a user would do:
const result = await librettoCli("setup --skip-browsers");
expect(result.exitCode).toBe(0);
const result2 = await librettoCli("--help");
expect(result2.stdout).toContain("Usage:");
Bad — extra abstraction obscures the actual interaction:
const result = await setupWithDefaults(); // hides the real CLI invocation
assertSuccess(result); // hides what "success" means
Don't introduce helpers for brevity. The value of a test is that you can read it top-to-bottom and understand exactly what's happening. Repetition across tests is fine — each test should be self-contained and readable on its own.
Each test should exercise one coherent user scenario from setup through assertion. Don't combine unrelated flows into a single test, and don't split one logical flow across multiple tests.
The full set of tests for a system should cover the sufficiently complete set of flows a user might follow. Think about:
test cases passed, vs 10 / 22 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.