Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Execute AssemblyAI production deployment checklist and rollback procedures. Use when deploying AssemblyAI integrations to production, preparing for launch, or implementing go-live procedures for transcription services. Trigger with phrases like "assemblyai production", "deploy assemblyai", "assemblyai go-live", "assemblyai launch checklist".
.claude/skills/jeremylongshore-assemblyai-prod-checklist/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-11 | ✗→✓ | ▲ Improved | 68% | 0% |
| case-02 | ✗→✓ | ▲ Improved | 18% | 0% |
| case-01 | ✗→✓ | ▲ Improved | 24% | 0% |
| case-07 | ✗→✓ | ▲ Improved | -20% | 0% |
| case-09 | ✗→✓ | ▲ Improved | -13% | 0% |
Gate production on evidence across the entire AssemblyAI lifecycle. Treat data, credentials, spend, deployment, rollback, and deletion as separately owned boundaries.
Production requires current REST and Streaming v3 contracts, explicit models, protected keys and tokens, authenticated idempotent callbacks, bounded retries, explicit streaming termination, deletion propagation, cost controls, monitoring, and tested rollback. One happy-path transcript is insufficient.
For live work, inject ASSEMBLYAI_API_KEY from an approved secret manager and send the raw value only in the AssemblyAI Authorization header to the configured first-party host. Never print, commit, place in a URL, or expose it to an untrusted client. Callback secrets and temporary streaming tokens are separate credentials.
Use Read, Glob, and Grep to inspect repository code, configuration, fixtures, and evidence. Use Write and Edit only for approved implementation or documentation changes. Do not call AssemblyAI, upload audio, open a streaming session, mint a token, replay a callback, deploy, rotate a key, or delete a transcript merely because this skill was invoked.
Require an accountable owner before live audio processing, production credential or endpoint changes, paid model or capacity changes, content retention, callback replay, deployment, or deletion. Read-only repository inspection and synthetic offline validation do not authorize live vendor actions.
Return the operation scope, environment, region, contract surface, authorization class, model and feature decisions, deterministic validation results, content-free identifiers, risks, cleanup or rollback state, and a concise pass/fail receipt. Exclude credentials, signed URLs, audio, transcript text, prompts, and customer-derived content.
Rerun the smallest relevant deterministic check, compare actual state with the requested outcome and current first-party contract, verify sensitive fields are absent from evidence, and confirm rollback, termination, or deletion state before reporting success.
Review the dated first-party evidence map before relying on any model, parameter, limit, price, region, or lifecycle claim.
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-11 | fail→pass | 14,419 | 13,801 | -4% | 1 | 1 | 0% | 2,433 | 4,095 | +68% | 0 | 0 | — |
case-12 | pass→pass | 12,935 | 11,589 | -10% | 1 | 1 | 0% | 2,287 | 3,546 | +55% | 0 | 0 | — |
case-10 | pass→pass | 11,543 | 7,942 | -31% | 1 | 1 | 0% | 2,136 | 2,833 | +33% | 0 | 0 | — |
case-02 | fail→pass | 17,177 | 12,231 | -29% | 1 | 1 | 0% | 3,333 | 3,929 | +18% | 0 | 0 | — |
case-03 | fail→fail | 25,410 | 26,055 | +3% | 1 | 1 | 0% | 4,412 | 5,940 | +35% | 0 | 0 | — |
case-01 | fail→pass | 17,639 | 11,942 | -32% | 1 | 1 | 0% | 2,911 | 3,598 | +24% | 0 | 0 | — |
case-04 | pass→pass | 16,030 | 13,910 | -13% | 1 | 1 | 0% | 2,654 | 4,130 | +56% | 0 | 0 | — |
case-05 | pass→pass | 12,673 | 11,574 | -9% | 1 | 1 | 0% | 2,329 | 3,475 | +49% | 0 | 0 | — |
case-06 | fail→fail | 12,857 | 4,029 | -69% | 1 | 1 | 0% | 2,046 | 2,153 | +5% | 0 | 0 | — |
case-07 | fail→pass | 17,917 | 6,852 | -62% | 1 | 1 | 0% | 3,438 | 2,745 | -20% | 0 | 0 | — |
case-08 | fail→fail | 18,161 | 12,604 | -31% | 1 | 1 | 0% | 3,394 | 3,687 | +9% | 0 | 0 | — |
case-09 | fail→pass | 14,778 | 5,031 | -66% | 1 | 1 | 0% | 2,595 | 2,262 | -13% | 0 | 0 | — |
case-13 | fail→pass | 14,020 | 12,047 | -14% | 1 | 1 | 0% | 2,459 | 3,531 | +44% | 0 | 0 | — |
case-14 | pass→pass | 10,965 | 4,672 | -57% | 1 | 1 | 0% | 1,979 | 2,258 | +14% | 0 | 0 | — |
case-15 | pass→pass | 17,976 | 18,742 | +4% | 1 | 1 | 0% | 3,079 | 4,932 | +60% | 0 | 0 | — |
case-16 | fail→pass | 17,073 | 11,740 | -31% | 1 | 1 | 0% | 3,023 | 3,453 | +14% | 0 | 0 | — |
case-17 | fail→pass | 8,598 | 2,791 | -68% | 1 | 1 | 0% | 1,393 | 1,821 | +31% | 0 | 0 | — |
case-18 | pass→pass | 7,245 | 2,556 | -65% | 1 | 1 | 0% | 994 | 1,802 | +81% | 0 | 0 | — |
case-19 | fail→fail | 17,797 | 11,999 | -33% | 1 | 1 | 0% | 3,121 | 3,457 | +11% | 0 | 0 | — |
case-20 | pass→pass | 17,283 | 16,894 | -2% | 1 | 1 | 0% | 3,280 | 4,905 | +50% | 0 | 0 | — |
case-21 | pass→pass | 13,849 | 9,236 | -33% | 1 | 1 | 0% | 2,497 | 3,270 | +31% | 0 | 0 | — |
case-22 | pass→pass | 9,574 | 7,320 | -24% | 1 | 1 | 0% | 1,867 | 2,822 | +51% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted. The headline lift of +36 percentage points is the difference between those two pass rates over the 22 comparable cases.
The publisher has shipped newer versions since this run, so these numbers describe v1, not the version currently listed.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.