Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Claude Codeセッションの正式な評価フレームワークで、評価駆動開発（EDD）の原則を実装します
.claude/skills/loulanyue-eval-harness/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-01 | ✗→✓ | ▲ Improved | 17% | 0% |
| case-03 | ✗→✓ | ▲ Improved | 23% | 0% |
| case-04 | ✗→✓ | ▲ Improved | -4% | 0% |
| case-06 | ✗→✓ | ▲ Improved | -14% | 0% |
| case-09 | ✗→✓ | ▲ Improved | -21% | 0% |
Claude Code 工作階段的正式評估框架，實作 eval 驅動開發（EDD）原則。
Eval 驅動開發將 evals 視為「AI 開發的單元測試」：
測試 Claude 是否能做到以前做不到的事：
markdown[CAPABILITY EVAL: feature-name] 任務：Claude 應完成什麼的描述 成功標準： - [ ] 標準 1 - [ ] 標準 2 - [ ] 標準 3 預期輸出：預期結果描述
確保變更不會破壞現有功能：
markdown[REGRESSION EVAL: feature-name] 基準：SHA 或檢查點名稱 測試： - existing-test-1: PASS/FAIL - existing-test-2: PASS/FAIL - existing-test-3: PASS/FAIL 結果：X/Y 通過（先前為 Y/Y）
使用程式碼的確定性檢查：
bash# 檢查檔案是否包含預期模式 grep -q "export function handleAuth" src/auth.ts && echo "PASS" || echo "FAIL" # 檢查測試是否通過 npm test -- --testPathPattern="auth" && echo "PASS" || echo "FAIL" # 檢查建置是否成功 npm run build && echo "PASS" || echo "FAIL"
使用 Claude 評估開放式輸出：
markdown[MODEL GRADER PROMPT] 評估以下程式碼變更： 1. 它是否解決了陳述的問題？ 2. 結構是否良好？ 3. 邊界案例是否被處理？ 4. 錯誤處理是否適當？ 分數：1-5（1=差，5=優秀） 理由：[解釋]
標記為手動審查：
markdown[HUMAN REVIEW REQUIRED] 變更：變更內容的描述 理由：為何需要人工審查 風險等級：LOW/MEDIUM/HIGH
「k 次嘗試中至少一次成功」
「所有 k 次試驗都成功」
markdown## EVAL 定義：feature-xyz ### 能力 Evals 1. 可以建立新使用者帳戶 2. 可以驗證電子郵件格式 3. 可以安全地雜湊密碼 ### 回歸 Evals 1. 現有登入仍可運作 2. 工作階段管理未變更 3. 登出流程完整 ### 成功指標 - 能力 evals 的 pass@3 > 90% - 回歸 evals 的 pass^3 = 100%
撰寫程式碼以通過定義的 evals。
bash# 執行能力 evals [執行每個能力 eval，記錄 PASS/FAIL] # 執行回歸 evals npm test -- --testPathPattern="existing" # 產生報告
markdownEVAL 報告：feature-xyz ======================== 能力 Evals： create-user: PASS (pass@1) validate-email: PASS (pass@2) hash-password: PASS (pass@1) 整體： 3/3 通過 回歸 Evals： login-flow: PASS session-mgmt: PASS logout-flow: PASS 整體： 3/3 通過 指標： pass@1: 67% (2/3) pass@3: 100% (3/3) 狀態：準備審查
/eval define feature-name在 .claude/evals/feature-name.md 建立 eval 定義檔案
/eval check feature-name執行當前 evals 並報告狀態
/eval report feature-name產生完整 eval 報告
在專案中儲存 evals：
.claude/
evals/
feature-xyz.md # Eval 定義
feature-xyz.log # Eval 執行歷史
baseline.json # 回歸基準markdown## EVAL：add-authentication ### 階段 1：定義（10 分鐘） 能力 Evals： - [ ] 使用者可以用電子郵件/密碼註冊 - [ ] 使用者可以用有效憑證登入 - [ ] 無效憑證被拒絕並顯示適當錯誤 - [ ] 工作階段在頁面重新載入後持續 - [ ] 登出清除工作階段 回歸 Evals： - [ ] 公開路由仍可存取 - [ ] API 回應未變更 - [ ] 資料庫 schema 相容 ### 階段 2：實作（視情況而定） [撰寫程式碼] ### 階段 3：評估 執行：/eval check add-authentication ### 階段 4：報告 EVAL 報告：add-authentication ============================== 能力：5/5 通過（pass@3：100%） 回歸：3/3 通過（pass^3：100%） 狀態：準備發佈
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-01 | fail→pass | 20,232 | 13,529 | -33% | 1 | 1 | 0% | 3,504 | 4,084 | +17% | 0 | 0 | — |
case-02 | fail→fail | 15,588 | 11,398 | -27% | 1 | 1 | 0% | 2,728 | 3,706 | +36% | 0 | 0 | — |
case-03 | fail→pass | 13,987 | 7,295 | -48% | 1 | 1 | 0% | 2,461 | 3,025 | +23% | 0 | 0 | — |
case-04 | fail→pass | 13,211 | 4,441 | -66% | 1 | 1 | 0% | 2,505 | 2,409 | -4% | 0 | 0 | — |
case-05 | pass→pass | 10,320 | 3,375 | -67% | 1 | 1 | 0% | 1,728 | 2,241 | +30% | 0 | 0 | — |
case-06 | fail→pass | 12,815 | 2,100 | -84% | 1 | 1 | 0% | 2,315 | 1,999 | -14% | 0 | 0 | — |
case-07 | pass→pass | 14,707 | 9,173 | -38% | 1 | 1 | 0% | 2,413 | 3,252 | +35% | 0 | 0 | — |
case-08 | pass→pass | 15,724 | 11,034 | -30% | 1 | 1 | 0% | 2,622 | 3,440 | +31% | 0 | 0 | — |
case-09 | fail→pass | 13,478 | 1,572 | -88% | 1 | 1 | 0% | 2,288 | 1,811 | -21% | 0 | 0 | — |
case-10 | fail→pass | 10,475 | 2,232 | -79% | 1 | 1 | 0% | 1,711 | 1,969 | +15% | 0 | 0 | — |
case-11 | fail→pass | 8,916 | 2,016 | -77% | 1 | 1 | 0% | 1,510 | 1,866 | +24% | 0 | 0 | — |
case-12 | pass→pass | 13,422 | 4,906 | -63% | 1 | 1 | 0% | 1,917 | 2,481 | +29% | 0 | 0 | — |
case-13 | fail→pass | 13,972 | 9,334 | -33% | 1 | 1 | 0% | 2,379 | 3,146 | +32% | 0 | 0 | — |
case-14 | pass→pass | 17,066 | 9,981 | -42% | 1 | 1 | 0% | 2,695 | 3,197 | +19% | 0 | 0 | — |
case-15 | pass→pass | 16,514 | 5,943 | -64% | 1 | 1 | 0% | 2,874 | 2,639 | -8% | 0 | 0 | — |
case-16 | pass→pass | 14,789 | 3,582 | -76% | 1 | 1 | 0% | 2,540 | 2,167 | -15% | 0 | 0 | — |
case-17 | pass→pass | 12,429 | 7,866 | -37% | 1 | 1 | 0% | 2,138 | 2,923 | +37% | 0 | 0 | — |
case-18 | pass→pass | 10,360 | 5,900 | -43% | 1 | 1 | 0% | 1,722 | 2,556 | +48% | 0 | 0 | — |
case-19 | fail→pass | 11,932 | 3,270 | -73% | 1 | 1 | 0% | 2,097 | 2,217 | +6% | 0 | 0 | — |
case-20 | pass→pass | 13,957 | 11,182 | -20% | 1 | 1 | 0% | 2,470 | 3,658 | +48% | 0 | 0 | — |
case-21 | pass→pass | 8,871 | 6,433 | -27% | 1 | 1 | 0% | 1,653 | 2,649 | +60% | 0 | 0 | — |
case-22 | pass→pass | 10,418 | 8,834 | -15% | 1 | 1 | 0% | 2,245 | 3,444 | +53% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted. The headline lift of +41 percentage points is the difference between those two pass rates over the 22 comparable cases.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.