Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Production readiness checklist for Clari API integrations. Use when launching a Clari data pipeline, validating export automation, or preparing for production forecast sync. Trigger with phrases like "clari production", "clari go-live", "clari checklist", "clari launch".
.claude/skills/jeremylongshore-clari-prod-checklist/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-01 | ✗→✓ | ▲ Improved | 23% | 0% |
| case-02 | ✗→✓ | ▲ Improved | 22% | 0% |
| case-04 | ✗→✓ | ▲ Improved | -15% | 0% |
| case-05 | ✗→✓ | ▲ Improved | 25% | 0% |
| case-07 | ✗→✓ | ▲ Improved | -3% | 0% |
Convert production readiness into a fail-closed evidence bundle. A green decision requires proof of provider access, contract compatibility, data correctness, quota behavior, observability, security, recovery, and operator ownership.
Confirm surface-specific credentials, effective scope, base URL, API version, schema fingerprint, and entitlement.
Reconcile counts, identifiers, time periods, totals, pagination, and empty-result behavior against an approved source.
Exercise timeout, 429, provider 5xx, aborted job, partial load, schema drift, and restart from checkpoint.
Prove secret redaction, least privilege, encryption, retention, deletion, mutation gates, and sensitive-content minimization.
Confirm dashboards, alerts, service-status dependency, runbook, support bundle, capacity headroom, and on-call ownership.
Record exact artifact hashes and approvers. Launch only when rollback is rehearsed and no required evidence is missing.
Production secrets must be injected from the approved manager into the matching client and never copied into release artifacts. Verify rotation and emergency revocation before launch.
Use Read and Grep to inspect configuration, provider contracts, fixtures, logs, schemas, and existing tests before proposing a change. Use Write or Edit only for the approved plan, implementation, test, or redacted receipt; do not issue, rotate, revoke, create, update, cancel, delete, export, ingest, or publish provider data without explicit operator approval.
Return the exact surface, environment, resource or job identifiers, contract fingerprint, evidence, unresolved risks, and final decision without exposing credentials or sensitive customer data.
A forecast pipeline passes reconciliation and restart tests but lacks a verified token revocation drill. The launch remains no-go until rotation and dependent-job recovery are demonstrated.
| Failure | Response | | --- | --- | | Required evidence is missing | Return no-go and name the owner and exact proof required. | | Provider limit has no headroom | Reduce workload or obtain approved capacity before launch. | | Rollback is untested | Rehearse it in non-production and retain the receipt before approval. |
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-17 | pass→pass | 13,984 | 11,343 | -19% | 1 | 1 | 0% | 2,202 | 2,845 | +29% | 0 | 0 | — |
case-01 | fail→pass | 19,146 | 16,093 | -16% | 1 | 1 | 0% | 3,045 | 3,740 | +23% | 0 | 0 | — |
case-02 | fail→pass | 16,039 | 13,708 | -15% | 1 | 1 | 0% | 3,263 | 3,974 | +22% | 0 | 0 | — |
case-03 | fail→fail | 12,709 | 9,515 | -25% | 1 | 1 | 0% | 2,176 | 2,562 | +18% | 0 | 0 | — |
case-04 | fail→pass | 9,372 | 2,727 | -71% | 1 | 1 | 0% | 1,709 | 1,455 | -15% | 0 | 0 | — |
case-05 | fail→pass | 12,021 | 8,786 | -27% | 1 | 1 | 0% | 1,821 | 2,271 | +25% | 0 | 0 | — |
case-06 | pass→pass | 10,226 | 2,629 | -74% | 1 | 1 | 0% | 1,605 | 1,513 | -6% | 0 | 0 | — |
case-07 | fail→pass | 10,981 | 5,299 | -52% | 1 | 1 | 0% | 1,804 | 1,747 | -3% | 0 | 0 | — |
case-08 | fail→pass | 31,248 | 11,039 | -65% | 1 | 1 | 0% | 1,330 | 2,979 | +124% | 0 | 0 | — |
case-09 | fail→pass | 13,098 | 7,029 | -46% | 1 | 1 | 0% | 2,064 | 2,045 | -1% | 0 | 0 | — |
case-10 | fail→pass | 12,907 | 9,958 | -23% | 1 | 1 | 0% | 2,153 | 2,593 | +20% | 0 | 0 | — |
case-11 | pass→pass | 12,392 | 10,701 | -14% | 1 | 1 | 0% | 1,879 | 2,711 | +44% | 0 | 0 | — |
case-12 | fail→pass | 12,999 | 6,807 | -48% | 1 | 1 | 0% | 2,103 | 2,098 | -0% | 0 | 0 | — |
case-13 | pass→pass | 13,627 | 10,894 | -20% | 1 | 1 | 0% | 2,270 | 2,743 | +21% | 0 | 0 | — |
case-14 | pass→pass | 11,449 | 6,886 | -40% | 1 | 1 | 0% | 1,831 | 2,055 | +12% | 0 | 0 | — |
case-15 | pass→pass | 14,396 | 12,218 | -15% | 1 | 1 | 0% | 2,600 | 3,596 | +38% | 0 | 0 | — |
case-16 | pass→pass | 13,711 | 17,258 | +26% | 1 | 1 | 0% | 2,458 | 3,836 | +56% | 0 | 0 | — |
case-18 | fail→pass | 6,536 | 5,470 | -16% | 1 | 1 | 0% | 1,111 | 1,494 | +34% | 0 | 0 | — |
case-19 | pass→pass | 9,826 | 2,710 | -72% | 1 | 1 | 0% | 1,198 | 1,342 | +12% | 0 | 0 | — |
case-20 | fail→fail | 14,047 | 19,193 | +37% | 1 | 1 | 0% | 2,239 | 3,650 | +63% | 0 | 0 | — |
case-21 | pass→pass | 17,360 | 14,818 | -15% | 1 | 1 | 0% | 2,846 | 3,577 | +26% | 0 | 0 | — |
case-22 | pass→pass | 16,013 | 11,873 | -26% | 1 | 1 | 0% | 3,185 | 3,233 | +2% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted, and 21 counted toward the lift figure. The other 1 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of +45 percentage points is the difference between those two pass rates over the 21 comparable cases.
The publisher has shipped newer versions since this run, so these numbers describe v1, not the version currently listed.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.