LLM & Retrieval Evals

LLM & Retrieval Evals

Evals are the core discipline of AI engineering. A model is a stochastic component you do not control: a prompt tweak, a temperature change, a model version bump, or a vendor silently re-routing your traffic can move quality in any direction, and none of it shows up in a stack trace. The only way to know whether a change helped or hurt is to measure task performance on a fixed set of inputs you trust. "It looks better" is a hypothesis, not a result. Vibes don't ship — a passing eval suite does.

Evals are to LLM systems what tests are to ordinary software, with one twist: the output is non-deterministic and often open-ended, so the grader is itself a system you must build and validate. This guide covers what to measure (eval types, generation metrics, retrieval metrics, grounding), how to grade (exact match, embeddings, LLM-as-judge, humans), how to build and maintain an eval set, and how to wire evals into your delivery process so regressions get caught before users do.

The mental model: inputs → system → outputs → grader → score → decision. Every section below is one link in that chain. If any link is weak — a stale eval set, a biased judge, a metric that doesn't track the task — the score lies, and you ship blind.

1. Why evals, concretely

• You can't improve what you can't measure. Iteration without a scorecard is a random walk. With a scorecard, every change is a labeled experiment.
• Silent regressions are the default failure. LLM output degrades gracefully-looking: still fluent, still confident, subtly wrong. No exception, no 500. Only an eval catches "accuracy dropped from 88% to 71% after the model upgrade."
• Model swaps are not drop-in. GPT-4o → GPT-4.1, Sonnet → a cheaper Haiku, an open-weights migration — each shifts the output distribution. The eval is the acceptance test for the swap.
• Prompts are code with no type system. A one-word prompt edit can break JSON formatting on 12% of inputs. Regression evals are your compiler.
• Evals turn arguments into experiments. "Should we use few-shot here?" is answerable in an hour with an eval set, versus a week of stakeholder opinion.

Capability evals ≠ product evals. Public benchmarks (MMLU, GSM8K, HumanEval, MT-Bench) measure a model's general capability and help you pick a base model. They tell you almost nothing about your task on your data — a model can top a leaderboard and fail your extraction schema. They are also heavily contaminated (leaked into training sets), so a high public score is partly memorization. Build product evals on your own inputs; treat benchmarks only as a coarse shortlist for which models to even try.

Rule of thumb: if you cannot state the metric and the threshold a change must clear, you are not ready to make the change.

2. Eval types — what to build

Golden sets are the backbone: small, high-quality, human-verified. Each item carries the input, the expected output (or a rubric/checklist when output is open-ended), and metadata (category, difficulty, source). Quality over quantity — 80 well-chosen cases beat 5,000 noisy ones.

Regression sets exist to fail loudly. They are golden cases promoted to a CI gate: if a change drops the pass rate below threshold, the build is red. No regression gate = a prompt tweak can quietly break prod and nobody knows until a customer complains.

Adversarial sets probe the unhappy path on purpose: prompt injection ("ignore previous instructions…"), jailbreaks, PII exfiltration attempts, out-of-scope requests that should trigger refusal, ambiguous or contradictory inputs, and inputs in the wrong language/format. Track refusal correctness in both directions — over-refusal (declining valid requests) is as much a failure as under-refusal (complying with harmful ones).