Agentic Evaluation Patterns

Agentic Evaluation Patterns

Patterns for self-improvement through iterative evaluation and refinement.

Workspace-baseline context: This skill supports the template quality workflow. Use alongside @Code Review Agent (quality checklist validation), @SpecKit Analyze (cross-artifact consistency), and the Quality Checklist in copilot-instructions.md to evaluate generated templates against baseline standards.

Overview

Evaluation patterns enable agents to assess and improve their own outputs, moving beyond single-shot generation to iterative refinement loops.

Generate → Evaluate → Critique → Refine → Output
    ↑                              │
    └──────────────────────────────┘

When to Use

• Quality-critical generation: Code, reports, analysis requiring high accuracy
• Tasks with clear evaluation criteria: Defined success metrics exist
• Content requiring specific standards: Style guides, compliance, formatting
• Template validation: Checking new instructions/agents/prompts/skills against baseline Hard Rules and Quality Checklist

Pattern 1: Basic Reflection

Agent evaluates and improves its own output through self-critique.

def reflect_and_refine(task: str, criteria: list[str], max_iterations: int = 3) -> str:
    """Generate with reflection loop."""
    output = llm(f"Complete this task:\n{task}")
    
    for i in range(max_iterations):
        # Self-critique
        critique = llm(f"""