Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Iterative skill tuning via execute-evaluate-improve feedback loop. Uses maestro delegate Claude to execute skill, Agy to evaluate quality, and Agent to apply improvements. Iterates until quality threshold or max iterations. Triggers on "skill iter tune", "iterative skill tuning", "tune skill".
.claude/skills/catlog22-skill-iter-tune/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-07 | ✗→✓ | ▲ Improved | 292% | 0% |
| case-08 | ✗→✓ | ▲ Improved | 224% | 0% |
| case-10 | ✗→✓ | ▲ Improved | 198% | 0% |
| case-11 | ✗→✓ | ▲ Improved | 290% | 0% |
| case-12 | ✗→✓ | ▲ Improved | 489% | 0% |
> Plan tracking: codex 无 TaskCreate/TaskUpdate/TodoWrite 任务板。进度清单用 update_plan({ explanation?, plan: [{ step, status }] }) 维护(整体提交步骤数组,status: pending | in_progress | completed),权威状态始终在 session 工件中;依赖/认领(addBlockedBy/owner)是工件字段,不是工具参数。
<required_reading> @~/.maestro/workflows/run-mode.md @~/.maestro/workflows/codex-run-mode.md </required_reading>
Iterative skill refinement through execute-evaluate-improve feedback loops. Each iteration runs the skill via Claude, evaluates output via Agy, and applies improvements via Agent.
┌──────────────────────────────────────────────────────────────────────────┐
│ Skill Iter Tune Orchestrator (SKILL.md) │
│ → Parse input → Setup workspace → Iteration Loop → Final Report │
└────────────────────────────┬─────────────────────────────────────────────┘
│
┌───────────────────┼───────────────────────────────────┐
↓ ↓ ↓
┌──────────┐ ┌─────────────────────────────┐ ┌──────────┐
│ Phase 1 │ │ Iteration Loop (2→3→4) │ │ Phase 5 │
│ Setup │ │ ┌─────┐ ┌─────┐ ┌─────┐ │ │ Report │
│ │─────→│ │ P2 │→ │ P3 │→ │ P4 │ │────→│ │
│ Backup + │ │ │Exec │ │Eval │ │Impr │ │ │ History │
│ Init │ │ └─────┘ └─────┘ └─────┘ │ │ Summary │
└──────────┘ │ ↑ │ │ └──────────┘
│ └───────────────┘ │
│ (if score < threshold │
│ AND iter < max) │
└─────────────────────────────┘Chain Mode (execution_mode === "chain"):
Phase 2 runs per-skill in chain_order:
Skill A → maestro delegate → artifacts/skill-A/
↓ (artifacts as input)
Skill B → maestro delegate → artifacts/skill-B/
↓ (artifacts as input)
Skill C → maestro delegate → artifacts/skill-C/
Phase 3 evaluates entire chain output + per-skill scores
Phase 4 improves weakest skill(s) in chainjavascript// ★ Auto mode detection const autoYes = /\b(-y|--yes)\b/.test($ARGUMENTS) if (autoYes) { workflowPreferences = { autoYes: true, maxIterations: 5, qualityThreshold: 80, executionMode: 'single' } } else { const prefResponse = request_user_input({ questions: [ { question: "选择迭代调优配置:", header: "Tune Config", multiSelect: false, options: [ { label: "Quick (3 iter, 70)", description: "快速迭代,适合小幅改进" }, { label: "Standard (5 iter, 80) (Recommended)", description: "平衡方案,适合多数场景" }, { label: "Thorough (8 iter, 90)", description: "深度优化,适合生产级 skill" } ] } ] }) const configMap = { "Quick": { maxIterations: 3, qualityThreshold: 70 }, "Standard": { maxIterations: 5, qualityThreshold: 80 }, "Thorough": { maxIterations: 8, qualityThreshold: 90 } } const selected = Object.keys(configMap).find(k => prefResponse["Tune Config"].startsWith(k) ) || "Standard" workflowPreferences = { autoYes: false, ...configMap[selected] } // ★ Mode selection: chain vs single const modeResponse = request_user_input({ questions: [{ question: "选择调优模式:", header: "Tune Mode", multiSelect: false, options: [ { label: "Single Skill (Recommended)", description: "独立调优每个 skill,适合单一 skill 优化" }, { label: "Skill Chain", description: "按链序执行,前一个 skill 的产出作为后一个的输入" } ] }] }); workflowPreferences.executionMode = modeResponse["Tune Mode"].startsWith("Skill Chain") ? "chain" : "single"; }
$ARGUMENTS → Parse:
├─ Skill path(s): first arg, comma-separated for multiple
│ e.g., ".claude/skills/my-skill" or "my-skill" (auto-prefixed)
│ Chain mode: order preserved as chain_order
├─ Test scenario: --scenario "description" or remaining text
└─ Flags: --max-iterations=N, --threshold=N, -y/--yes> ⚠️ COMPACT DIRECTIVE: Context compression MUST check update_plan phase status. > The phase currently marked in_progress is the active execution phase — preserve its FULL content. > Only compress phases marked completed or pending.
Read and execute: Ref: phases/01-setup.md
{run_dir}/outputs/skill-iter-tune-{ts}/Output: workDir, targetSkills[], testScenario, initialized state
javascript// Orchestrator iteration loop while (true) { // Increment iteration state.current_iteration++; state.iterations.push({ round: state.current_iteration, status: 'pending', execution: null, evaluation: null, improvement: null }); // Update update_plan update_plan(iterationTask, { subject: `Iteration ${state.current_iteration}/${state.max_iterations}`, status: 'in_progress', activeForm: `Running iteration ${state.current_iteration}` }); // === Phase 2: Execute === // Read: phases/02-execute.md // Single mode: one maestro delegate call for all skills // Chain mode: sequential maestro delegate per skill in chain_order, passing artifacts // Snapshot skill → construct prompt → maestro delegate --to claude --mode write // Collect artifacts // === Phase 3: Evaluate === // Read: phases/03-evaluate.md // Construct eval prompt → maestro delegate --to agy --mode analysis // Parse score → write iteration-N-eval.md → check termination // Check termination if (shouldTerminate(state)) { break; // → Phase 5 } // === Phase 4: Improve === // Read: phases/04-improve.md // Agent applies suggestions → write iteration-N-changes.md // Update update_plan with score // Continue loop }
Read and execute: Ref: phases/02-execute.md
iteration-{N}/skill-snapshot/maestro delegate "..." --to claude --mode write --cd "${iterDir}/artifacts"Read and execute: Ref: phases/03-evaluate.md
maestro delegate "..." --to agy --mode analysisiteration-{N}-eval.mdRead and execute: Ref: phases/04-improve.md
iteration-{N}-changes.mdRead and execute: Ref: phases/05-report.md
final-report.mdPhase Reference Documents (read on-demand when phase executes):
| Phase | Document | Purpose | Compact | |-------|----------|---------|---------| | 1 | phases/01-setup.md | Initialize workspace and state | update_plan 驱动 | | 2 | phases/02-execute.md | Execute skill via maestro delegate Claude | update_plan 驱动 + 🔄 sentinel | | 3 | phases/03-evaluate.md | Evaluate via maestro delegate Agy | update_plan 驱动 + 🔄 sentinel | | 4 | phases/04-improve.md | Apply improvements via Agent | update_plan 驱动 + 🔄 sentinel | | 5 | phases/05-report.md | Generate final report | update_plan 驱动 |
Compact Rules:
in_progress → 保留完整内容,禁止压缩completed → 可压缩为摘要Read() 恢复iteration-state.json is the only source of truthUser Input (skill paths + test scenario)
↓ (+ execution_mode + chain_order if chain mode)
↓
Phase 1: Setup
↓ workDir, targetSkills[], testScenario, iteration-state.json
↓
┌─→ Phase 2: Execute (maestro delegate claude)
│ ↓ artifacts/ (skill execution output)
│ ↓
│ Phase 3: Evaluate (maestro delegate agy)
│ ↓ score, dimensions[], suggestions[], iteration-N-eval.md
│ ↓
│ [Terminate?]─── YES ──→ Phase 5: Report → final-report.md
│ ↓ NO
│ ↓
│ Phase 4: Improve (Agent)
│ ↓ modified skill files, iteration-N-changes.md
│ ↓
└───┘ next iterationjavascript// Initial state update_plan({ subject: "Phase 1: Setup workspace", activeForm: "Setting up workspace" }) update_plan({ subject: "Iteration Loop", activeForm: "Running iterations" }) update_plan({ subject: "Phase 5: Final Report", activeForm: "Generating report" }) // Chain mode: create per-skill tracking tasks if (state.execution_mode === 'chain') { for (const skillName of state.chain_order) { update_plan({ subject: `Chain: ${skillName}`, activeForm: `Tracking ${skillName}`, description: `Skill chain member position ${state.chain_order.indexOf(skillName) + 1}` }) } } // During iteration N // Single mode: one score per iteration (existing behavior) // Chain mode: per-skill status updates if (state.execution_mode === 'chain') { // After each skill executes in Phase 2: update_plan(chainSkillTask, { subject: `Chain: ${skillName} — Iter ${N} executed`, activeForm: `${skillName} iteration ${N}` }) // After Phase 3 evaluates: update_plan(chainSkillTask, { subject: `Chain: ${skillName} — Score ${chainScores[skillName]}/100`, activeForm: `${skillName} scored` }) } else { // Single mode (existing) update_plan({ subject: `Iteration ${N}: Score ${score}/100`, activeForm: `Iteration ${N} complete`, description: `Strengths: ... | Weaknesses: ... | Suggestions: ${count}` }) } // Completed — collapse update_plan(iterLoop, { subject: `Iteration Loop (${totalIters} iters, final: ${finalScore})`, status: 'completed' })
javascriptfunction shouldTerminate(state) { // 1. Quality threshold met if (state.latest_score >= state.quality_threshold) { return { terminate: true, reason: 'quality_threshold_met' }; } // 2. Max iterations reached if (state.current_iteration >= state.max_iterations) { return { terminate: true, reason: 'max_iterations_reached' }; } // 3. Convergence: ≤2 points improvement over last 2 iterations if (state.score_trend.length >= 3) { const last3 = state.score_trend.slice(-3); if (last3[2] - last3[0] <= 2) { state.converged = true; return { terminate: true, reason: 'convergence_detected' }; } } // 4. Error limit if (state.error_count >= state.max_errors) { return { terminate: true, reason: 'error_limit_reached' }; } return { terminate: false }; }
| Phase | Error | Recovery | |-------|-------|----------| | 2: Execute | CLI timeout/crash | Retry once with simplified prompt, then skip | | 3: Evaluate | CLI fails | Retry once, then use score 50 with warning | | 3: Evaluate | JSON parse fails | Extract score heuristically, save raw output | | 4: Improve | Agent fails | Rollback from iteration-{N}/skill-snapshot/ | | Any | 3+ consecutive errors | Terminate with error report |
Error Budget: Each phase gets 1 retry. 3 consecutive failed iterations triggers termination.
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-05 | pass→fail | 7,979 | 8,389 | +5% | 1 | 1 | 0% | 1,420 | 4,435 | +212% | 0 | 0 | — |
case-01 | fail→fail | 14,635 | 12,038 | -18% | 1 | 1 | 0% | 2,222 | 4,722 | +113% | 0 | 0 | — |
case-02 | fail→fail | 23,760 | 7,592 | -68% | 1 | 1 | 0% | 3,817 | 4,255 | +11% | 0 | 0 | — |
case-03 | fail→fail | 32,719 | 8,172 | -75% | 1 | 1 | 0% | 6,222 | 4,308 | -31% | 0 | 0 | — |
case-04 | pass→fail | 12,938 | 6,165 | -52% | 1 | 1 | 0% | 2,347 | 4,073 | +74% | 0 | 0 | — |
case-06 | pass→fail | 10,596 | 9,234 | -13% | 1 | 1 | 0% | 1,794 | 4,287 | +139% | 0 | 0 | — |
case-07 | fail→pass | 14,254 | 3,281 | -77% | 1 | 1 | 0% | 1,105 | 4,327 | +292% | 0 | 0 | — |
case-08 | fail→pass | 22,016 | 5,602 | -75% | 1 | 1 | 0% | 1,461 | 4,731 | +224% | 0 | 0 | — |
case-09 | fail→fail | 7,445 | 3,885 | -48% | 1 | 1 | 0% | 1,314 | 4,005 | +205% | 0 | 0 | — |
case-10 | fail→pass | 9,335 | 2,623 | -72% | 1 | 1 | 0% | 1,402 | 4,173 | +198% | 0 | 0 | — |
case-11 | fail→pass | 7,157 | 2,313 | -68% | 1 | 1 | 0% | 1,046 | 4,084 | +290% | 0 | 0 | — |
case-12 | fail→pass | 4,694 | 2,712 | -42% | 1 | 1 | 0% | 719 | 4,232 | +489% | 0 | 0 | — |
case-13 | pass→pass | 9,417 | 5,579 | -41% | 1 | 1 | 0% | 1,458 | 4,746 | +226% | 0 | 0 | — |
case-14 | fail→fail | 8,698 | 3,653 | -58% | 1 | 1 | 0% | 1,310 | 4,302 | +228% | 0 | 0 | — |
case-15 | pass→fail | 10,425 | 5,802 | -44% | 1 | 1 | 0% | 1,541 | 4,187 | +172% | 0 | 0 | — |
case-16 | fail→pass | 8,089 | 3,609 | -55% | 1 | 1 | 0% | 1,060 | 4,283 | +304% | 0 | 0 | — |
case-17 | fail→pass | 7,494 | 2,504 | -67% | 1 | 1 | 0% | 1,102 | 4,113 | +273% | 0 | 0 | — |
case-18 | pass→pass | 5,336 | 3,810 | -29% | 1 | 1 | 0% | 751 | 4,418 | +488% | 0 | 0 | — |
case-19 | fail→fail | 9,553 | 3,041 | -68% | 1 | 1 | 0% | 1,380 | 4,252 | +208% | 0 | 0 | — |
case-20 | fail→pass | 13,431 | 2,555 | -81% | 1 | 1 | 0% | 1,943 | 4,146 | +113% | 0 | 0 | — |
case-21 | fail→pass | 13,323 | 1,866 | -86% | 1 | 1 | 0% | 1,923 | 4,050 | +111% | 0 | 0 | — |
case-22 | pass→pass | 4,839 | 2,562 | -47% | 1 | 1 | 0% | 672 | 4,164 | +520% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted, and 14 counted toward the lift figure. The other 8 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of +23 percentage points is the difference between those two pass rates over the 14 comparable cases. 4 cases got worse with the skill loaded, and they are included in that figure.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.