Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →failproofai/failproofai-sdk
The way to make an AI agent report what it did to Failproof AI — planning what to record, writing the instrumentation, and proving the events land. Reach for it on vague phrasing too: "add observability to my agent", "why isn't my agent showing up?"
Passed 23 of 24 test cases with the skill, and 5 of 24 without it.
and 34 more in the full report
18 fixed5 passed either way1 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
This benchmark measured an earlier version of the skill.
A newer attempt on gemini-3.6-flash on 10/2/2026 did not produce a publishable comparison (not comparable). The snapshot above is the previously measured run.
One of 18 test cases the skill fixed. Same model, same prompt, with and without the skill.
When our agent encounters an unhandled exception, our error handler calls the SDK's run termination method with outcome='failure'. However, on the telemetry platform, these runs are still listed with green success statuses. Why is outcome='failure' not marking the run as failed?
| Checked | Without skill | With skill |
|---|---|---|
| The response points out that outcome='failure' is not recognized as a failed status. | Passed | Passed |
| The response specifies setting outcome to failed, error, timeout, or rejected. | Missed | Passed |
The instructions your model reads when the skill loads.
Two packages, one pipe: failproofai-sdk (Python, imported as failproofai_sdk)
and @failproofai/sdk (TypeScript/JavaScript). They write the same 15 events in
the same wire format into the same spool directory. Everything in this file is
language-neutral unless it says otherwise, and code blocks are Python.
For a TypeScript or JavaScript agent, read references/typescript.md alongside
it: it has the camelCase names, the adapters, bundler and Next.js setup, the
no-framework wiring, shutdown and verification. Both packages also ship the
evaluator worker that scores finished sessions on your own infrastructure (§7,
references/evaluator.md).
The SDK records what your agent did, from inside your agent. You call it at points
you choose; it appends structured events to local .jsonl files. A separate
collector ships those files to the platform.
your agent calls failproofai_sdk.event.*
→ SDK queues it in memory
→ flush thread writes <base_dir>/events/event-<timestamp>.jsonl
→ collector picks the file up and ships it
→ visible as sessions / events / errors / evals
The SDK's job ends at the file. That boundary is the most useful thing to know
about it: everything up to the .jsonl is yours to get right and yours to verify,
and it is verifiable on a laptop with no server, no API key, and no network.
The API is small — 15 event methods, all keyword-only. The hard parts are deciding where to call them and knowing which silences are bugs, because this SDK does not raise when you get it wrong. Sections 1-3 are the plan, 4 is the code, 5-6 are the proof, 7 is scoring the runs.
TypeScript/JavaScript: npm install @failproofai/sdk — zero dependencies, Node ≥
20.9, Bun or Deno, ESM and CommonJS. The npm name has no lookalike trap; the rest of
this section is Python. See references/typescript.md.
test cases passed, vs 5 / 24 without
More skills from the same source and nearby in the registry, with their headline benchmark lift.