Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Execute Customer.io production deployment checklist. Use when preparing for production launch, auditing integration quality, or performing pre-launch validation. Trigger: "customer.io production", "customer.io checklist", "deploy customer.io", "customer.io go-live", "customer.io launch".
.claude/skills/jeremylongshore-customerio-prod-checklist/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-02 | ✗→✓ | ▲ Improved | 57% | 0% |
| case-11 | ✗→✓ | ▲ Improved | 121% | 0% |
| case-12 | ✗→✓ | ▲ Improved | 61% | 0% |
| case-16 | ✗→✓ | ▲ Improved | 14% | 0% |
| case-17 | ✗→✓ | ▲ Improved | 34% | 0% |
Comprehensive go-live checklist for Customer.io integrations: credentials audit, integration quality review, email deliverability setup, monitoring configuration, smoke tests, and staged rollout plan.
| Item | Check | How to Verify | |------|-------|---------------| | Production Site ID | Correct workspace | Settings > API & Webhook Credentials | | Production Track API Key | Different from dev/staging | Compare with staging .env | | Production App API Key | Set for transactional messages | Test with curl bearer auth | | Region setting | Matches account region (US/EU) | Settings > Workspace Settings | | Secrets storage | In secrets manager, not .env | Check deployment config | | Key rotation schedule | Documented (90-day cycle) | Calendar reminder set |
typescript// scripts/prod-audit.ts import { TrackClient, RegionUS } from "customerio-node"; async function auditIntegration() { const checks: { name: string; pass: boolean; detail: string }[] = []; // Check 1: Credentials exist const siteId = process.env.CUSTOMERIO_SITE_ID; const trackKey = process.env.CUSTOMERIO_TRACK_API_KEY; const appKey = process.env.CUSTOMERIO_APP_API_KEY; checks.push({ name: "Track credentials", pass: !!(siteId && trackKey), detail: siteId ? `Site ID: ${siteId.substring(0, 4)}...` : "MISSING", }); checks.push({ name: "App API credential", pass: !!appKey, detail: appKey ? "Set" : "MISSING (needed for transactional)", }); // Check 2: API connectivity if (siteId && trackKey) { const cio = new TrackClient(siteId, trackKey, { region: RegionUS }); try { await cio.identify("prod-audit-test", { email: "prod-audit@example.com", }); await cio.suppress("prod-audit-test"); checks.push({ name: "Track API", pass: true, detail: "Connected" }); } catch (err: any) { checks.push({ name: "Track API", pass: false, detail: `${err.statusCode}: ${err.message}`, }); } } // Report console.log("\n=== Customer.io Production Audit ===\n"); for (const check of checks) { const icon = check.pass ? "PASS" : "FAIL"; console.log(`[${icon}] ${check.name}: ${check.detail}`); } const passed = checks.filter((c) => c.pass).length; console.log(`\nResult: ${passed}/${checks.length} passed`); return checks.every((c) => c.pass); } auditIntegration().then((ok) => process.exit(ok ? 0 : 1));
| Item | Status | Dashboard Location | |------|--------|-------------------| | Sending domain verified | Required | Settings > Sending Domains | | SPF record published | Required | DNS TXT record | | DKIM record published | Required | DNS CNAME record | | DMARC policy set | Recommended | DNS TXT record _dmarc.yourdomain.com | | Return-Path configured | Recommended | Settings > Sending Domains | | Unsubscribe link in all marketing emails | Required (CAN-SPAM) | Template editor | | Physical address in marketing emails | Required (CAN-SPAM) | Template editor |
message_data fieldstypescript// Set up these alerts in your monitoring system: const ALERT_THRESHOLDS = { // API health api_error_rate: 0.01, // Alert if > 1% of API calls fail api_p99_latency_ms: 5000, // Alert if p99 > 5 seconds // Delivery health bounce_rate: 0.05, // Alert if > 5% bounce rate complaint_rate: 0.001, // Alert if > 0.1% spam complaints delivery_rate: 0.95, // Alert if < 95% delivery rate // Operational health queue_depth: 10000, // Alert if event queue > 10K pending webhook_error_rate: 0.05, // Alert if > 5% webhook processing failures };
bash#!/usr/bin/env bash set -euo pipefail echo "=== Customer.io Production Smoke Test ===" # Test 1: Track API connectivity TRACK_STATUS=$(curl -s -o /dev/null -w "%{http_code}" \ -u "${CUSTOMERIO_SITE_ID}:${CUSTOMERIO_TRACK_API_KEY}" \ -X PUT "https://track.customer.io/api/v1/customers/smoke-test-$(date +%s)" \ -H "Content-Type: application/json" \ -d '{"email":"smoke@example.com","_smoke_test":true}') if [ "$TRACK_STATUS" = "200" ]; then echo "[PASS] Track API: HTTP 200" else echo "[FAIL] Track API: HTTP ${TRACK_STATUS}" exit 1 fi # Test 2: App API connectivity APP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" \ -H "Authorization: Bearer ${CUSTOMERIO_APP_API_KEY}" \ "https://api.customer.io/v1/campaigns") if [ "$APP_STATUS" = "200" ]; then echo "[PASS] App API: HTTP 200" else echo "[FAIL] App API: HTTP ${APP_STATUS}" exit 1 fi echo "" echo "All smoke tests passed"
| Time | Action | Rollback Trigger | |------|--------|-----------------| | T-24h | Final staging test pass | Any test failure | | T-12h | Production smoke tests | Smoke test failure | | T-1h | Enable for internal team (feature flag) | Error reports | | T-0 | Enable for 10% of users | Error rate > 1% | | T+1h | Increase to 50% | Error rate > 1% or bounce rate > 5% | | T+2h | 100% traffic | Same thresholds | | T+24h | Post-launch review | N/A |
typescript// Feature flag integration for staged rollout function shouldUseCio(userId: string, rolloutPercent: number): boolean { // Deterministic — same user always gets same result const hash = createHash("md5").update(userId).digest("hex"); const bucket = parseInt(hash.substring(0, 8), 16) % 100; return bucket < rolloutPercent; }
| Issue | Solution | |-------|----------| | Smoke test fails in production | Verify production credentials (different from staging) | | High bounce rate post-launch | Check sending domain verification, review bounce logs | | Spam complaints spike | Review email content, verify unsubscribe links work | | Feature flag not working | Check feature flag service connectivity |
After production launch, proceed to customerio-upgrade-migration for SDK maintenance.
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-01 | fail→fail | 13,289 | 11,862 | -11% | 1 | 1 | 0% | 2,793 | 4,803 | +72% | 0 | 0 | — |
case-02 | fail→pass | 36,095 | 16,034 | -56% | 1 | 1 | 0% | 3,126 | 4,898 | +57% | 0 | 0 | — |
case-03 | fail→fail | 16,807 | 14,259 | -15% | 1 | 1 | 0% | 3,683 | 5,175 | +41% | 0 | 0 | — |
case-04 | pass→pass | 8,635 | 2,503 | -71% | 1 | 1 | 0% | 1,553 | 2,472 | +59% | 0 | 0 | — |
case-05 | pass→pass | 7,253 | 4,447 | -39% | 1 | 1 | 0% | 1,333 | 2,853 | +114% | 0 | 0 | — |
case-06 | pass→pass | 9,808 | 4,014 | -59% | 1 | 1 | 0% | 1,919 | 2,917 | +52% | 0 | 0 | — |
case-07 | pass→pass | 4,951 | 3,581 | -28% | 1 | 1 | 0% | 843 | 2,724 | +223% | 0 | 0 | — |
case-08 | pass→pass | 3,776 | 2,381 | -37% | 1 | 1 | 0% | 639 | 2,418 | +278% | 0 | 0 | — |
case-09 | pass→pass | 11,106 | 2,032 | -82% | 1 | 1 | 0% | 1,997 | 2,363 | +18% | 0 | 0 | — |
case-10 | fail→fail | 15,999 | 14,342 | -10% | 1 | 1 | 0% | 3,147 | 4,950 | +57% | 0 | 0 | — |
case-11 | fail→pass | 6,752 | 2,197 | -67% | 1 | 1 | 0% | 1,063 | 2,345 | +121% | 0 | 0 | — |
case-12 | fail→pass | 9,400 | 1,992 | -79% | 1 | 1 | 0% | 1,513 | 2,437 | +61% | 0 | 0 | — |
case-13 | pass→pass | 10,292 | 2,490 | -76% | 1 | 1 | 0% | 1,753 | 2,546 | +45% | 0 | 0 | — |
case-14 | pass→pass | 9,259 | 2,933 | -68% | 1 | 1 | 0% | 1,494 | 2,518 | +69% | 0 | 0 | — |
case-15 | pass→pass | 9,444 | 2,338 | -75% | 1 | 1 | 0% | 1,665 | 2,495 | +50% | 0 | 0 | — |
case-16 | fail→pass | 11,761 | 1,538 | -87% | 1 | 1 | 0% | 2,038 | 2,332 | +14% | 0 | 0 | — |
case-17 | fail→pass | 11,604 | 4,128 | -64% | 1 | 1 | 0% | 2,063 | 2,766 | +34% | 0 | 0 | — |
case-18 | pass→pass | 6,910 | 2,180 | -68% | 1 | 1 | 0% | 1,309 | 2,469 | +89% | 0 | 0 | — |
case-19 | fail→pass | 10,470 | 2,590 | -75% | 1 | 1 | 0% | 1,885 | 2,478 | +31% | 0 | 0 | — |
case-20 | pass→pass | 6,235 | 2,519 | -60% | 1 | 1 | 0% | 990 | 2,469 | +149% | 0 | 0 | — |
case-21 | pass→pass | 12,880 | 13,254 | +3% | 1 | 1 | 0% | 2,709 | 4,808 | +77% | 0 | 0 | — |
case-22 | pass→pass | 11,568 | 13,507 | +17% | 1 | 1 | 0% | 2,298 | 4,797 | +109% | 0 | 0 | — |
case-23 | pass→pass | 15,948 | 13,583 | -15% | 1 | 1 | 0% | 3,262 | 4,882 | +50% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 23 cases were attempted. The headline lift of +26 percentage points is the difference between those two pass rates over the 23 comparable cases.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.